Reminder we have our bi-weekly community call tonight @ 23:30 IST/20:00 CEST/19:00 BST/14:00 EDT/11:00 PDT. 🔔
If you want to discuss any topic at the meeting or want the Zarr community to discuss it please add it to the agenda here: https://hackmd.io/tFcffc__QRuo1UdGcGUbsg
Link to join: https://zoom.us/j/300670033?pwd=OFhjV0FHQmhHK2FYbGFRVnBPMVNJdz09
@ctr26 : you cannot. Zarr calls the storage classes in sync code. Although the storage classes might have async internals (like fsspec), that is done using a separate thread. We could, in theory, rewrite zarr to bubble up async IO calls when doing arr[:], and this might be worthwhile, but some considerable work.
The only thing you could do right now, would be to download all of the zarr chunks you will need to the virtual filesystem presented by pyodide using JS's async HTTP calls, and then read that with zarr in sync-only mode.
Hm, that gives me an idea! Maybe we can wrap a zarr array with an async zarr array that does exactly those two stages and then allows you to do z await arr[:]?
6 replies
28 replies
Hello Everyone,
I am Kshitij Verma pursuing Computer Science and Engineering from Indian Institute of Information Technology Tiruchirappalli, India. I would really like to start contributing to the Zarr community but don't understand how and where to begin.
I have read the contributing guide and the Zarr tutorials and have familiarized myself a bit with the package and am wondering what to do next.
I was wondering you guys could help me on how to start contributing.
2 replies
Tweet: https://twitter.com/zarr_dev/status/1554193857335554048
n5.py -- as has been suggested before, the clean way to implement an N5Store is by wrapping an existing store with an n5 compatibility layer. But I couldn't see a simple way to express the zarr -> n5 transformation, so instead we have two implementations (N5Store and N5FSStore) that just use inheritance. It's not very satisfying, and there's a lot of code duplication
7 replies
zarr-developers/zeps#13
This also comes with a PR for the spec:
zarr-developers/zarr-specs#152
I'll also provide an implementation PR for zarr-python next week. @joshmoore @MSanKeys963 et al, is this the right way forward for sharding? Please let me know if there is something missing or unclear in the ZEP. I assume the sharding spec PR will be under review until the ZIC decides upon the ZEP, right?
7 replies
10 replies
Hey everyone! I was curious if anyone knows about how to run optical flow algortihms on zarrs/H5 files with Dask. Our lab is using SLEAP to process some animal behavior and it uses opencv's flow calculation to maintain identities of subjects. You can see what I think is relevant here. It takes quite long time to run on their .slp files, which are based on H5 from what I can tell.
This seems like a great usecase for zarr and dask so read/write can be done in parallel to these files! Are there any examples of using dask for this kind of thing that any of you know about?
6 replies
Just wanted to share a small update. All the info. related to Zarr bi-weekly community calls is available at the new URL: https://zarr.dev/community-calls/
The new webpage also offers a
search bar at the top which can be used to search discussion topics from previous community meetings.
23:30 IST/20:00 CEST/19:00 BST/14:00 EDT/11:00 PDT. 🔔Agenda, joining instructions and previous meeting notes available here: https://zarr.dev/community-calls/
