Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
Domink Heinz
@dominikheinz

So, I created my repo initially on an external hdd that was mounted to my system.
Let's it was mounted as /mnt/a/.
I then created my repository, created volumes, snapshots etc - all good.

Now the next day, I wanted to access my snapshots again. However, everytime I tried to access my snapshots (list them) I got an error: unable to load snapshot from any storage backend.
I figured out, that this has to do with how the repository is mounted when it was crated.
Because on that day, my drive with the repository was mounted as /mnt/b/. And then there was a mismatch between the path that is saved in the knoxite configuration and the one that is specified.
Even tho I specified the correct path when I wanted to list the snapshots (/mnt/b/), it failed. Most likely because internally it still uses the path from the knoxite configuration that was set upon repository creation (/mnt).
When I umounted my harddrive and remounted it as /mnt/a/, everything worked fine.

So, to recreate this problem, try this:

  1. Connect external hdd and mount it to your system with mountpoint: /mnt/a/.
  2. Create a folder on that drive, something like backups or so. Initialize a new repo in there.
  3. Create a volume and backup some data to it.
  4. See if you can list the snapshots with knoxite -r /mnt/a/backups snapshot list <volume_id>. Should work fine.
  5. Unmount the drive.
  6. Remount the drive under a different path, for example /mnt/b/.
  7. See if you can list the snapshots with knoxite -r /mnt/b/backups snapshot list <volume_id>. It will fail with the error "unable to load snapshot from any storage backend".
  8. Weirdly enough, it is possible to list volumes, but not snapshots.
@mueslix:matrix.org
mueslix
@mueslix:matrix.org
[m]
oh wow
thanks for the detailed description, trying to reproduce and fix tonight
Domink Heinz
@dominikheinz
I was able to see that by looking into the knoxite configuration: knoxite repo cat -r /mnt/b/backups, it still had the old storage path saved in it. /mnt/a/backups

thanks for the detailed description, trying to reproduce and fix tonight

Awesome! Let me know if you have any trouble reproducing it

By the way, have you looked at #225 ? Were you able to reproduce the problem?
5 replies
mueslix
@mueslix:matrix.org
[m]
I can kinda see what's going on already.
What happens is that knoxite stores all the backup targets in the repo config, so it can reconnect to all targets without you having to manually specify each and every single one of them
e.g. when you store your repo on Dropbox and S3, it doesn't matter which target you specify, it'll open the repo config and discover the other configured targets
Domink Heinz
@dominikheinz
:D
Btw, is master the right branch to use? Beacuse there are other branches that are more recent
mueslix
@mueslix:matrix.org
[m]
Yes, master is the right branch. Branches will eventually feed into a PR that lands in master
Domink Heinz
@dominikheinz
Gotcha
Btw, I had a random idea, not sure what you think about it. the repository.knoxite file is very important, since without it, all access to volumes and snapshots is lost right. To make it a bit more robust against bit-corruption or other sorts of damage, you could have the file structured so all the information is written twice. But the second half is written mirror'd. That way, if there is a bit corruption, it is possible to repair the file. The same concept is used for the MBR/GPT partition schemes for example
Just a random thought :D
mueslix
@mueslix:matrix.org
[m]
Interesting
So far we're storing a copy of it on each target
Domink Heinz
@dominikheinz
What do you mean by target?
mueslix
@mueslix:matrix.org
[m]
Guess we could also use the reed solomon encoding we already employ
if you setup a knoxite repository to store data in multiple locations, e.g. Dropbox & S3 parallely
Domink Heinz
@dominikheinz
Ah okay.
But yea, would be neat to also have some resilience against corruption when using a single target for backups
mueslix
@mueslix:matrix.org
[m]
thanks @dominikheinz!
Domink Heinz
@dominikheinz
@mueslix:matrix.org Any updates? Did you manage to fix the bug?
mueslix
@mueslix:matrix.org
[m]
I think I see what's going on, but I'll have to find some more time to properly fix this. Probably not before this weekend, sorry.
Domink Heinz
@dominikheinz
Okay, no problem.
mueslix
@mueslix:matrix.org
[m]
I'll ping you here as soon as I got something to test :)
Domink Heinz
@dominikheinz
Hell yea, sounds good :D
mueslix
@mueslix:matrix.org
[m]
@dominikheinz: just a quick heads up, that I'm making a ton of progress and improvements in the fuse department right now.
funnily it affects a wider range of projects that are all using my code there :D
1 reply
Domink Heinz
@dominikheinz
@mueslix:matrix.org Thanks for the feedback. I assume the bug in the mounting (with files appearing as empty directories) is what you are referring to here?
And probably also the fact that the repo can't be modified when the path changed, as outlined in the other issue
Can you already give any ETA when you expect the bugfix to be live on master for knoxite?
Domink Heinz
@dominikheinz
@mueslix:matrix.org No updates yet? :P