Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
mueslix
@mueslix:matrix.org
[m]
I actively use knoxite myself tho, so it'll see updates for the foreseeable future.
Domink Heinz
@dominikheinz
Fair enough! I have been spending the past 2 hours or so playing around with it, and I do like it. I am trying to find a backup tool that suites my needs (Turns out that is trickier than expected)

I actively use knoxite myself tho, so it'll see updates for the foreseeable future.

Thats great news!

mueslix
@mueslix:matrix.org
[m]
curious, what are you particular needs?
Domink Heinz
@dominikheinz

So, essentially I want to backup my personal data with it.
My setup consists of two different storage places:

  • One cold storage backup (An HDD that I occasionally connect to my PC, that is only used for backups).
  • One is online storage/cloud storage. (Just a VPS with a bunch of storage space).

The needs are that I can do backups are restores easiely via the commandline. I also want the tool to be deduping to save disk space. And also I want the backup data to be properly encrypted so I dont have to worry about handling this manually with e.g. LUKS.

I was looking into other backup solutions and most of them were missing some things.

mueslix
@mueslix:matrix.org
[m]
restic might be a popular alternative that fits the bill
Domink Heinz
@dominikheinz
Yea I was looking into restic and borgbackup.
From what I remember borgbackup's encryption is fairly poor, they even mention that on their website. For restic, teh way of handling keys and passphrases seemed unnecessarily conmplicated.
I do like that for knoxite all I have to do is remember the password and I am able to decrpyt and restore my data. No need to manage any keys that are bound to my backups etc.
mueslix
@mueslix:matrix.org
[m]
good points, yes. I'd like to think restic also lets you use passwords, but I'm not entirely sure
Domink Heinz
@dominikheinz
I will recheck for sure
Domink Heinz
@dominikheinz
knoxite/knoxite#224 @mueslix:matrix.org
mueslix
@mueslix:matrix.org
[m]
🙌
Domink Heinz
@dominikheinz
As for storing on a remote server, what would you recommend? Mounting the remote filesystem via sshfs and then proceed as if it is a normal local repo, or utilize the SSH functionality of knoxite instead?
mueslix
@mueslix:matrix.org
[m]
I would use the SSH functionality inside knoxite, as the transparent sshfs layer causes a lot of unnecessary network traffic.
Domink Heinz
@dominikheinz
Okay, good to know.
Another question, if you dont mind me asking: Is there anything I need to keep in my when having multiple backup repositories (e.g. in my case on on a VPS and one on an external HDD). Ideally I would like them to be synchonized.
Is it safe to copy one knoxite repo from one storage device to another and use them to my liking?
Or can that cause problems?
mueslix
@mueslix:matrix.org
[m]
Copying the data would absolutely work fine.
You could also use both the SSH server and external drive concurrently during backups and set fault redundancy to 1. That would let you restore your data as long as one of the backup targets is available.
Domink Heinz
@dominikheinz
Okay that is good to know. Because in that case I would probably use the VPS are a regular backup location, and occasionally just copy the entire backup repo to my HDD

You could also use both the SSH server and external drive concurrently during backups and set fault redundancy to 1. That would let you restore your data as long as one of the backup targets is available.

That sounds good, is there any documentation on how to do that>

1 reply
mueslix
@mueslix:matrix.org
[m]
The downside is that both targets needs to be always available during the backup (not the restore tho)
It basically combines your external drive and SSH server to a RAID-1 array
Domink Heinz
@dominikheinz
Okay. So just an exact copy - basically the same as if I would manually copy over the repo?
2 replies

You could also use both the SSH server and external drive concurrently during backups and set fault redundancy to 1. That would let you restore your data as long as one of the backup targets is available.

That is not a problem since I prefer to trigger my backups manually and not automated. Does fault redundancy 2 then mean, the backup would only be started if both targets are available?

1 reply
Okay, yea so fault tolerance levels might be overkill for my scenario then
How does knoxite deal with connection loss during a backup or a restore?
mueslix
@mueslix:matrix.org
[m]
indeed, just thought I'll point it out as it's probably knoxite's unique selling point. I don't think anyone else does that
During a backup that should be a fatal error, as it can't guarantee your data is safely stored.
During a restore you can let it try continue restoring data
There's also the --pedantic flag, which makes it abort on the first error
Domink Heinz
@dominikheinz
Good to know! Does knoxite automatically clean up corrupted files that were created during the failed backup on the backup destination?
So say for example a big file gets backed up, and halfway the connection drops. Then obviously on the remote repo, there is a corrupted backup or at least corrupted chunks.
mueslix
@mueslix:matrix.org
[m]
All good questions, really! Only if it tries to store the same chunk of data again, but often that's the common scenario: the backup operation aborts with an error, you restart it, it would clean up after itself
if you just ignore the error, then yeah, it would obviously be left in a dangling state. A restore or verify operation would also detect that tho.
Domink Heinz
@dominikheinz
But the verify just informs that the snapshot/repo is corrupted, it doesnt repair the error, does it?
mueslix
@mueslix:matrix.org
[m]
it wouldn't know where the original data stems from, so yeah, it can only inform at that point
Domink Heinz
@dominikheinz
So I guess in that case, the right approach would be to delete the snapshot that was attempted and then retry?
mueslix
@mueslix:matrix.org
[m]
the snapshot wouldn't be created in that case, as that's finished as the last part of the transaction
so technically, depending on when exactly it aborts you couldn't even try to restore or verify
retrying the same backup operation would detect the existing chunks in storage however and would not overwrite the existing data. It pretty much picks up where it left off during the last backup operation.
Domink Heinz
@dominikheinz
Hm, interesting. Yea, I am jus tthinking of edge cases haha. I am curious how resilient knoxite is, to minimize possible unwanted surprises when using it ;)
mueslix
@mueslix:matrix.org
[m]
fully understand :)
Domink Heinz
@dominikheinz
I got a couple more questions :D If you dont mind
mueslix
@mueslix:matrix.org
[m]
glad you do!
of course
Domink Heinz
@dominikheinz
Knoxite has volumes and snapshots. Snapshots are clear, but are volumes kinda like partitions on a disk?
So the way I understand them right now, it makes sense to use them to separate different content in the same repo. For example have one volume music, another one personal data, and one for movies or something like that.
ANd then for each of them I manage snapshots
Is that correct?
mueslix
@mueslix:matrix.org
[m]
Yep, that's how I use it. Also if you backup a farm of servers, you could have a volume per machine
(and still benefit from data deduplication across multiple machines)