Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    andrewchambers
    @andrewchambers
    and to get s3 support ready for people to use
    Paul
    @ptman:kapsi.fi
    [m]
    let's hope you get it
    so I was going to ask about access patterns
    high-latency dumb protocols like sftp and s3 are probably clearly worse than bupstash over ssh?
    andrewchambers
    @andrewchambers
    bupstash can pipeline a lot of file access
    its not too bad, though I need to add some tuning options for the user
    w.r.t. s3 - the way it will work is it will be like a storage plugin for the repository
    so if you ssh server is located close to your s3 server it will be fast
    but yeah, normal bupstash doesn't do many round trips
    it basically pushes data to the server in a stream and gets the 'ack' once every 20 GB
    and when it downloads data the server pushes it, without any round trips or w.e.
    also - regarding bupstash post 1.0
    I mainly want to fine tune things and make it perform as fast as possible
    and be very stable
    So we will see what sort of extra features become necessary or not
    I pushed the next release back a few times more than I should have - had a few things to deal with
    andrewchambers
    @andrewchambers
    so the project looks more dormant than it is
    Sebastien Marie
    @semarie
    one difficulty with posix lock is that to be effective all tools needs to use it too. else you ends with one tool reading/writing file (without using lock) whereas the file is (properly) locked by another tool
    andrewchambers
    @andrewchambers
    yep - though bupstash provides exec-with-locks to help you
    generally programs don't mess around around in the internal file trees of other tools
    Sebastien Marie
    @semarie
    with the query-language, is it possible to get the "latest" entry (from timestamp tag point of vue) ? I am looking to extract (bupstash restore) the latest snapshot (name=backup.tar and ...)
    doing external processing on "bupstash list" output isn't a very good way for me, as I am using BUPSTASH_KEY_COMMAND with explicit password asking, and it means asking password twice (one for "list", another for "restore"), or locally store the key
    andrewchambers
    @andrewchambers
    @semarie sorry for the late reply, was away. I don't think there is another way at the moment.
    I considered adding some post processing to the queries
    like limit 1
    It's definitely something I will consider adding
    andrewchambers
    @andrewchambers
    @semarie another thing is to use something like gpg agent to retain keys in memory for some duration
    tionis
    @tionis:matrix.org
    [m]
    How can I specify a port for ssh repositories?
    andrewchambers
    @andrewchambers
    @tionis:matrix.org I think the normal way should work
    :port
    if not you can go : BUPSTASH_REPOSITORY_COMMAND='ssh opts... bupstash serve /path/to/repository'
    3 replies
    sogaiu
    @sogaiu:matrix.org
    [m]
    is the ssh url port style supported? e.g. https://serverfault.com/questions/218256/specify-ssh-port-for-git/218408#218408
    1 reply
    tionis
    @tionis:matrix.org
    [m]
    or to be more specific it tries to looks up "host.tld:port" over dns
    andrewchambers
    @andrewchambers
    Ah - @tionis:matrix.org i was mistaken
    it doesn't yet support :PORT
    but the REPOSITORY_COMMAND should certainly work
    I have been thinking of changing it to a true URI
    however then it would require uri escaping for spaces and a few other things
    tionis
    @tionis:matrix.org
    [m]
    Well I'm currently doing something like this:
    BUPSTASH_REPOSITORY_COMMAND="ssh -p 222" bupstash init --repository ssh://host/path/on/host
    andrewchambers
    @andrewchambers
    @tionis:matrix.org that is not what I said to do
    I said to do this
    tionis
    @tionis:matrix.org
    [m]
    Ah I see, sorry, I kinda read over that part
    andrewchambers
    @andrewchambers
    export BUPSTASH_REPOSITORY_COMMAND="ssh -p 222 host bupstash serve /path/on/host"
    then do bupstash init
    tionis
    @tionis:matrix.org
    [m]
    Yes this does seem to work
    andrewchambers
    @andrewchambers
    its quite low level but at least it gives total control of how the io pipes are established
    tionis
    @tionis:matrix.org
    [m]
    ok
    andrewchambers
    @andrewchambers
    I think the ssh:// thing should still be improved to at least include port though
    tionis
    @tionis:matrix.org
    [m]
    that would be great
    thanks for the help!