Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
mueslix
@mueslix:matrix.org
[m]
There's also the --pedantic flag, which makes it abort on the first error
Domink Heinz
@dominikheinz
Good to know! Does knoxite automatically clean up corrupted files that were created during the failed backup on the backup destination?
So say for example a big file gets backed up, and halfway the connection drops. Then obviously on the remote repo, there is a corrupted backup or at least corrupted chunks.
mueslix
@mueslix:matrix.org
[m]
All good questions, really! Only if it tries to store the same chunk of data again, but often that's the common scenario: the backup operation aborts with an error, you restart it, it would clean up after itself
if you just ignore the error, then yeah, it would obviously be left in a dangling state. A restore or verify operation would also detect that tho.
Domink Heinz
@dominikheinz
But the verify just informs that the snapshot/repo is corrupted, it doesnt repair the error, does it?
mueslix
@mueslix:matrix.org
[m]
it wouldn't know where the original data stems from, so yeah, it can only inform at that point
Domink Heinz
@dominikheinz
So I guess in that case, the right approach would be to delete the snapshot that was attempted and then retry?
mueslix
@mueslix:matrix.org
[m]
the snapshot wouldn't be created in that case, as that's finished as the last part of the transaction
so technically, depending on when exactly it aborts you couldn't even try to restore or verify
retrying the same backup operation would detect the existing chunks in storage however and would not overwrite the existing data. It pretty much picks up where it left off during the last backup operation.
Domink Heinz
@dominikheinz
Hm, interesting. Yea, I am jus tthinking of edge cases haha. I am curious how resilient knoxite is, to minimize possible unwanted surprises when using it ;)
mueslix
@mueslix:matrix.org
[m]
fully understand :)
Domink Heinz
@dominikheinz
I got a couple more questions :D If you dont mind
mueslix
@mueslix:matrix.org
[m]
glad you do!
of course
Domink Heinz
@dominikheinz
Knoxite has volumes and snapshots. Snapshots are clear, but are volumes kinda like partitions on a disk?
So the way I understand them right now, it makes sense to use them to separate different content in the same repo. For example have one volume music, another one personal data, and one for movies or something like that.
ANd then for each of them I manage snapshots
Is that correct?
mueslix
@mueslix:matrix.org
[m]
Yep, that's how I use it. Also if you backup a farm of servers, you could have a volume per machine
(and still benefit from data deduplication across multiple machines)
Domink Heinz
@dominikheinz
Perfect, yea that sounds good !
How would I remove a snapshot with knoxite? The documentation only mentions removal of volumes
(Because I can see removing a snapshot not being trivial because of all the deduplication functionality)
mueslix
@mueslix:matrix.org
[m]
There is knoxite snapshot remove [id]. If that's missing from the docs, that's an oversight :)
Domink Heinz
@dominikheinz
Gotcha !
mueslix
@mueslix:matrix.org
[m]
This will indeed only delete the metadata. If you want to also remove chunks that are now not associated with any snapshot anymore (data deduplication), you would have to use knoxite repo pack manually.
Domink Heinz
@dominikheinz
How would I specify more complex rules what to include and exclude in a backup. Say all files in dir1, except for *.txt files for example etc etc. (I am kinda thinking of a similar ruleset that one can specify with a .gitignore file for example.

This will indeed only delete the metadata. If you want to also remove chunks that are now not associated with any snapshot anymore (data deduplication), you would have to use knoxite repo pack manually.

Perfect!

mueslix
@mueslix:matrix.org
[m]
A .gitignore like feature would be super neat indeed, but doesn't exist yet.
Domink Heinz
@dominikheinz
Gotcha.
mueslix
@mueslix:matrix.org
[m]
You can however filter the files beforehand, and then pass the file-set to knoxite
Domink Heinz
@dominikheinz
You mean manually filter the files beforehand?
mueslix
@mueslix:matrix.org
[m]
yeah
Domink Heinz
@dominikheinz
Yea, but that could be really tedious when there are many files :D But yep, thats a way to do it
mueslix
@mueslix:matrix.org
[m]
I wonder if there's a cli tool that parses .ignore files and returns the resulting list of filtered fiels/dirs
if not, there totally should be one :D
Domink Heinz
@dominikheinz
A quick google search told me there are gitignore parsers, yea
Not sure if there are any for go tho
mueslix
@mueslix:matrix.org
[m]
I meant a pure, universal CLI tool actually
for Go, I actually wrote one myself: https://github.com/muesli/gitcha
Domink Heinz
@dominikheinz
Ah nice. Yea, you could use that then if you ever decide to add some sort of .knoxiteignore file :P
mueslix
@mueslix:matrix.org
[m]
yep :)
Domink Heinz
@dominikheinz
Another thing that came to my mind: Does knoxite have some sort of retention functionality. E.g. keep 3 snapshots each week, 1 per month and 4 per year or something like that
I know that restic and borg have that kinda feature
Not really needed for me, just curiouys
mueslix
@mueslix:matrix.org
[m]
it doesn't, sorry
but certainly something that might be useful
Domink Heinz
@dominikheinz
Fair
Yea, but overall knoxite seems really good. Simple and straight to the point. The only feature I am missing is the diff between snapshots. But I can see myself using knoxite for my personal backups for sure
mueslix
@mueslix:matrix.org
[m]
That's lovely to hear :)