Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Francesc Alted
    @FrancescAlted
    Hi, this is a test
    Peter Taylor
    @emersont1
    Hi, is this channel used at all?
    Francesc Alted
    @FrancescAlted
    Yes, chatting about Blosc and ecosystem here is welcome!
    Alan Somers
    @asomers
    I just published a Rust package for Blosc bindings: https://crates.io/crates/blosc
    esc
    @esc
    hi
    esc
    @esc
    I discovered today, that c-blosc doesn't seem to like the empty string anymore
    esc
    @esc
    'ello
    esc
    @esc
    python-blosc got a nice upgrade on PyPi with some included docs via README: https://pypi.org/project/blosc/
    it also lists this gitter channel now as a point of contact
    Alecks Gates
    @agates
    Hi. I just finished a rough port of python-blosc to use a CFFI backend. It still needs some work, but here it is if anyone's interested. https://github.com/agates/python-blosc-cffi
    Fair warning, I'm 100% not a C programmer :)
    Alecks Gates
    @agates
    I tried to keep the high-level interface (toplevel.py and __init__.py) as close to the original as possible
    esc
    @esc
    @agates cool!
    @agates not sure if you saw already, but we may have such a thing already
    Francesc Alted
    @FrancescAlted
    Hi @agates I was going to say the same than @esc . Still, I suppose that adding the toplevel.py would be a nice exercise :-)
    Alecks Gates
    @agates
    Yes, I saw that and wanted the high level interface. Also learning CFFI and doing some C was good effort :)
    So I have all of the original tests of python-blosc passing, it's mostly a drop-in replacement for that library
    Francesc Alted
    @FrancescAlted
    Good point. Could you provide a hint on how the compare-pack-ptr.py benchmarks fares in the different libraries?
    Alecks Gates
    @agates
    I'll try to get some results, yes.
    Alecks Gates
    @agates
    @FrancescAlted I have some initial results here https://github.com/agates/python-blosc-cffi/tree/master/bench/results
    esc
    @esc
    @agates do you think you could graph those? Or maybe you have a take home message?
    Alecks Gates
    @agates
    I can try to graph it. A more statistically sound comparison will be useful too as many of these differences are probably within a margin or error...
    But from what I have seen, some things seem a little slower and some seem a little faster. No explanation as to why :)
    esc
    @esc
    @agates interesting!
    Alecks Gates
    @agates
    I made a quick spreadsheet of compress_ptr.py results after I got my version working with pypy -- in general it looks like the CFFI version is faster on pypy and a tad slower on CPython. Also, the CFFI version tends to have more consistent compression between pypy and CPython which is interesting
    https://github.com/agates/python-blosc-cffi/blob/master/bench/results/compress_ptr_comparison.ods?raw=true
    I'll look at compare-pack-ptr.py again hopefully this weekend
    esc
    @esc
    @agates cool!
    esc
    @esc
    @FrancescAlted it seems like the new python-blosc release is a bit broken? 😉
    esc
    @esc
    hi
    Francesc Alted
    @FrancescAlted
    Hi @esc. For some reason I was not informed about your messages. Hopefully the new python-blosc release fixed the existing issues.
    esc
    @esc
    @FrancescAlted thank you, that would be good, hopefully then I can release a new version of bloscpack
    @FrancescAlted I think the problem might be, that the context objects don't read the value of the global setting
    I think the compression and decompression objects probably have a copy of the number of threads and so setting the value globally does not have an effect.
    Francesc Alted
    @FrancescAlted
    yeah, maybe. in fact, next generation of c-blosc2 comes with contexts, that should prevent this to happen.
    you might want to check https://github.com/Blosc/python-blosc2 which is providing a backward compatible API with python-blosc and see how it goes
    esc
    @esc
    @FrancescAlted it might be worth a shot, yes, thank you for the suggestion
    Alexandr Romanenko
    @waralex
    Hello everyone. Do you know if anyone is already working on a wrapper for Julia? I was thinking of spending the weekend exploring the possibility of writing such a wrapper, but if someone is already writing it, it will be a waste of time