Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    tionis
    @tionis:matrix.org
    [m]
    Do you have a sponsor page or something like that?
    andrewchambers
    @andrewchambers
    Not currently
    andrewchambers
    @andrewchambers
    @tionis:matrix.org one thing to keep in mind is bupstash currently does linear lookup for --pick and also for listing backups
    it hasn't yet been an issue, my largest repository has a few thousand snapshots and a linear lookup can easily search that in an instant
    tionis
    @tionis:matrix.org
    [m]
    Ok, so if I want to read a specific file from snapshots I should ensure that it is one of the lexicographic first for faster lookup?
    andrewchambers
    @andrewchambers
    How many files do you plan to put in each snapshot?
    1 reply
    andrewchambers
    @andrewchambers
    In the future I might improve bupstash to do something more efficient in that case
    but I can't really say without knowing what you are doing
    andrewchambers
    @andrewchambers
    @tionis:matrix.org btw - there is no sponsor page, the best way would be to create an empty repository on bupstash.io
    it would just charge you the minimum
    tionis
    @tionis:matrix.org
    [m]
    Ok
    andrewchambers
    @andrewchambers
    So, a new release is coming soon, it is almost time for 1.0 or at least 1.0 release candidates
    1 reply
    1.0 will mean even stronger backwards compatibility guarantees - not that I ever deliberately broke a repository
    1.0 should coincide with wider access to the bupstash.io service - so we really don't want to break bupstash for people using versions packaged with their distro package manager.
    andrewchambers
    @andrewchambers
    Though we still need a logo / fav icon :P
    tionis
    @tionis:matrix.org
    [m]
    Is there some way to force bupstash new-key to print the key to stdout instead of a file
    tionis
    @tionis:matrix.org
    [m]
    :point_up: Edit: Is there some way to force bupstash new-key to print the key to stdout instead of a file?
    andrewchambers
    @andrewchambers
    it seems not
    we couldatt it I suppose
    add
    andrewchambers
    @andrewchambers
    The main reason it doesn't is it sets the permissions on the key
    andrewchambers
    @andrewchambers
    so you don't accidentally make a key readable by other users, but stdout with -o - might be ok
    tionis
    @tionis:matrix.org
    [m]
    Would make it a tiny bit easier to wrap bupstash in scripts to store the key in a secure place like the keychain
    piegames
    @piegames:matrix.org
    [m]
    Does using /dev/stdout as "file" work?
    Paul
    @ptman:kapsi.fi
    [m]
    isn't that like a bashism?
    oh, it isn't?
    symlinks to /proc/self/fd/1 (and 0 and 2)
    maybe I was thinking of bash /dev/tcp
    andrewchambers
    @andrewchambers
    It currently doesn't work since it doesn't overwrite keys
    tionis
    @tionis:matrix.org
    [m]
    Does bupstash currently do checksumming etc on the server side to prevent corruption of data?
    andrewchambers
    @andrewchambers
    @tionis:matrix.org all data is hashed and signed so if it got corrupted on the server
    the client would alert you
    however if you want checksums on your repository data too, I considered adding more, but I would also say you could use the btrfs or zfs file systems
    those filesystems verify checksums on read automatically for all data
    andrewchambers
    @andrewchambers
    for my personal backups I use btrfs so I can use btrfs scrub
    or I can just read the data
    and it will check for errors
    andrewchambers
    @andrewchambers
    That being said, in the future I might add a data storage mode that adds parity information and checksums as an extra
    for now I would advise using a filesystem with checks or RAID
    tionis
    @tionis:matrix.org
    [m]
    Ok, thanks
    nh2
    @nh2:matrix.org
    [m]

    @andrewchambers: can you remind me, what exactly is bupstash's behaviour with small files, and does it depend on if they are in the same dir or not?
    In https://github.com/andrewchambers/bupstash/issues/26#issuecomment-730882871 you write

    small files are packed into single chunks

    but that is the only reference I can find on the Internet for this feature

    andrewchambers
    @andrewchambers
    @nh2:matrix.org sure
    so bupstash packs small files within the same directory into a chunk
    bupstash internally uses directory boundaries as a sort of deduplication boundary
    4 replies
    that helps the dedup resync
    as a consequence it means it must split the chunk at directory boundaries and isn't able to pack more small files into it
    I am considering other heuristics that can group directories
    technically it would be possible to pack many directories into a single chunk
    it doesn't affect the repository format - it is something we can change
    bupstash concatenates all files into a giant stream