Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    andrewchambers
    @andrewchambers
    dirs are just metadata
    and it can skip files that match exclude rules i suppose
    + /some/file
    c /another/file
    s /some/path
    or maybe a for add
    so add, cached and skipped
    piegames
    @piegames:matrix.org
    [m]
    What would "cached" mean? And will there be a distinction between "added" and "updated"?
    andrewchambers
    @andrewchambers
    well bupstash works by remembering what it sent last time
    and doesn't resend that
    so cached is just a file it knows it already send last time
    piegames
    @piegames:matrix.org
    [m]
    Maybe rename "cached" to "skipped" and call "skipped" "excluded"?
    andrewchambers
    @andrewchambers
    yeah
    that makes sense
    put /dir/added
    skip /dir/skipped
    exclude /dir/excluded
    maybe
    piegames
    @piegames:matrix.org
    [m]
    If you add padding :)
    andrewchambers
    @andrewchambers
    haha
    do you think times are useful?
    piegames
    @piegames:matrix.org
    [m]
    Not really.
    I mean, sometimes. But eeh
    andrewchambers
    @andrewchambers
    I'll give adding it a try, i think you are right and its not as complicated as I was making it out to be.
    andrewchambers
    @andrewchambers
    hmm, theres another thing
    the file actions are not quite so straight forward, since only files are ever 'skipped', as they are the only things that have data that can be skipped
    piegames
    @piegames:matrix.org
    [m]
    Because you always walk all directories (except when they're excluded)?
    andrewchambers
    @andrewchambers
    yeah, it stats everything in a dir that isn't excluded, then it hashes the stats, then looks in the cache for a matching entry and uses that entry to skip reading the file contents
    then it pushes all dirs into the work list
    the current progress bar just shows when it is processing a whole directory
    its definitely easy to just print the paths as we process them
    just harder to exactly summarize it
    i can see why exclusions are especially useful to see
    andrewchambers
    @andrewchambers
    in which case it might be something like...
    d  /dir
    f  /file
    d* /dir-contributing-to-cache-hit
    f* /cached-file
    x  /excluded-path
    this way you can kind of see how effective the stat cache is being
    and what is being excluded
    piegames
    @piegames:matrix.org
    [m]
    You could try doing dx and fx for excluded paths.
    andrewchambers
    @andrewchambers
    yeah that might look ok
    and maybe ! or something for modification detected
    thats the case where you are uploading a file
    and it changes while you are uploading it
    piegames
    @piegames:matrix.org
    [m]
    How does bupstash handle that anyways?
    andrewchambers
    @andrewchambers
    it just carries on and updates the file size to match whatever it read
    but it doesn't cache that file
    if bupstash stats a file, but then you delete it
    bupstash pretends that file never existed and skips it
    so, it tries to carry on
    originally bupstash would abort and restart your backup
    but that caused a lot of other problems too
    so, in practice its like tar, you shouldn't really do a backup on live data
    but its gonna try it's best
    perhaps a slightly smeared backup is better than nothing
    piegames
    @piegames:matrix.org
    [m]
    That sounds pretty sane