Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Ella Bahry
    @bellonet
    @tomka @aschampion, do i have a way of getting the pixel value of the label layer when clicking?
    Andrew Champion
    @aschampion
    Yes, in the feature/label-stack-manager branch StackLayer has a method pixelValueInScaleLevel: https://github.com/catmaid/CATMAID/blob/features/label-stack-manager/django/applications/catmaid/static/js/layers/stack-layer.js#L520
    it takes stack space coordinates
    For the purposes of getting something working you can get a stack layer via project.getStackViewers()[0].getLayer('StackLayer')
    Ella Bahry
    @bellonet
    thanks!! will try it tonight.
    Ella Bahry
    @bellonet
    hi andrew, im here with tom, and will try to see if we can upload labels to catmaid. should the n5 format be exactly in the format i've sent you for the previous data?
    Ella Bahry
    @bellonet
    when we're specifying image mirrors in "image base" path - what do we need to specify for a local path?
    Tom Kazimiers
    @tomka
    I only know about the h2n5 setup of the ortho-stacks for v14. If you could point us to an example with a label stack, we could use this as an example. The stack mirror path for the ortho views looks like this: http://bocklab.hhmi.org/h2n5/tile/volumes/raw/%SCALE_DATASET%/0_2/256_26/%AXIS_0%/%AXIS_2%/%AXIS_1%. And we started h2n5 just in its most simple form: h2n5 <path-to-n5>.
    Andrew Champion
    @aschampion
    Label stacks work the same. The H2N5 tile source path in CATMAID is just the relative path from the root of the n5 you're running h2n5 on. So http://localhost:8088/group/dataset/etc/%SCALE_DATASET%/[slicing dimensions]/[tile size]/%AXIS_0%/%AXIS_1%/%AXIS_2%/, permuting the AXIS substitution parameter order as necessary for xyz, xzy, etc.
    Tom Kazimiers
    @tomka
    Copying from Slack: We made some progress. We could get h2n5 to show the image data using this URL: http://127.0.0.1:8088/tile/raw/data/%SCALE_DATASET%/0_1/256_256/%AXIS_0%/%AXIS_1%/%AXIS_2%, but struggle to get the labels shown. With http://127.0.0.1:8088/tile/labels/unique-labels/%SCALE_DATASET%/0_1/256_256/%AXIS_0%/%AXIS_1%/%AXIS_2%`. We seem to get binary data back and the browser complains with code 501 (Data type does not have an image renderer implemented), which sounds promising. We tried to use the Randomised Label Color Map, but this doesn't seem to change anything.
    The browser also only seems to attemt the first request which returns with 501, and after that all further requests are canceled.
    Andrew Champion
    @aschampion
    What dtype is the dataset?
    Tom Kazimiers
    @tomka
    We are not entirely sure, is there a way we can check?
    Andrew Champion
    @aschampion
    Look at the dataType key in labels/unique-labels/s0/attributes.json
    err `n5gest ls <n5_directory.n5>
    Tom Kazimiers
    @tomka
    Okay, thanks. It is uint64
    Ella Bahry
    @bellonet
    thread 'arbiter:4ea5f3ed-0373-40cf-9ad8-14bd1042bc07:actix-net-worker-43' panicked at 'TODO: block ndarray failed: ShapeError/IncompatibleShape: incompatible shapes', libcore/result.rs:1009:5
    Tom Kazimiers
    @tomka
    This is what h2n5 logs on requests
    Andrew Champion
    @aschampion
    uint64 labels don't work yet, see the tested/supported table here: https://github.com/catmaid/CATMAID/issues/1801#issuecomment-438747065
    Ella Bahry
    @bellonet
    what should they be?
    Andrew Champion
    @aschampion
    Further, that table is for the features/image-block-layer, which is accessing N5 directly, bypassing H2N5
    And I think the unique-labels dataset from Paintera is an index of labels, not a label map
    Which is why the shapes are incompatible
    If you're using H2N5, the only way is to still use float32 labels
    Tom Kazimiers
    @tomka
    Oh okay. So we probably can't make it work with this dataset?
    Andrew Champion
    @aschampion
    And you'll need the features/channel-packing branch of H2N5, which will pack the 32bit values into the RGBA channels (with some changes to the URL)
    Actually the dev branch of h2n5 already includes the channel packing
    Tom Kazimiers
    @tomka
    Thanks! But I guess we can't use this (Paintera) dataset then, but would need a different one, correct?
    Andrew Champion
    @aschampion
    Philipp changed paintera so that it can export scalar labels
    So if you can figure that out, then truncate the labels to 32 bits, you could either try to use h2n5 with channel packing, or use the features/image-block-layer CATMAID branch to access the N5 directly.
    Both of those have been tested.
    Tom Kazimiers
    @tomka
    Okay, thanks
    Andrew Champion
    @aschampion
    I don't recall what the next steps for 64 bit support were, got sidetracked into fixing the performance problems with webworkers and haven't had time for it the past two weeks.
    Tom Kazimiers
    @tomka
    This N5 also has a /labels/data/ with uint8 data, does this happen to look like that? Otherwise we would try to reexport from Paintera.
    Andrew Champion
    @aschampion
    That's not actually uint8, it's a serialized multilabel representation.
    Tom Kazimiers
    @tomka
    I see
    Andrew Champion
    @aschampion
    conda update -c hanslovsky paintera
    This should install paintera-0.8.1. It will also install paintera-conversion-helper-0.4.0, I made a separate package for that tool for better versioning. I tested very briefly and as far as I can tell, scalar label support works now. Let me know if you encounter any issues.
    This is how I created a data set with scalar labels, notice the --winner-takes-all-downsampling option:
    paintera-conversion-helper -r -d /home/hanslovskyp/Downloads/sample_A_padded_20160501.hdf,volumes/raw,raw -d /home/hanslovskyp/Downloads/sample_A_padded_20160501.hdf,volumes/labels/neuron_ids,label -o /data/hanslovskyp/sample_A_padded_20160501-bs=64-winner-takes-all-n5-lookup2.n5 -b 64,64,64 -s 2,2,1 2,2,1 2,2,1 2,2,2 2,2,2 2,2,2 --winner-takes-all-downsampling --label-block-lookup-backend-n5=10000
    Those are his instructions for the scalar labels
    Tom Kazimiers
    @tomka
    Awesome, thanks
    Andrew Champion
    @aschampion
    For my own future reference: the next steps for 64 bit support were updating the ObjectLabelColorMap shader to shader version 300, making shaders generic/specializable to the datatype of image stacks, and setting the right GL parameters in the image block layer
    Stephan Gerhard
    @unidesigner
    (I'm on dev branch and start the devserver): After a few seconds, I get an " File "....python3.7/site-packages/asgi_ipc/store.py", line 45, in mutate_value
    raise self.deadlock_error
    RuntimeError: Semaphore appears to be deadlocked. Kill all channels processes and remove files named like asgi-messages in /dev/shm
    "
    What could be the problem here?
    Nevermind. I removed the locks from /dev/shm and things seem to work again.
    Tom Kazimiers
    @tomka
    Yeah, unfortunately runserver does start a minimal ASGI server as well that uses shared memory (/dev/shm) for inter-process communication. I typically run in therefore typically as runserver --noasgi and the deadlocks are gone. In production we use Daphne for our WebSockets/ASGI needs.
    Stephan Gerhard
    @unidesigner
    Ok, thanks for clarification. Useful switch to know. What are you using WebSockets/ASGI for in production?
    Can you also briefly check to edit a "data view" in the admin interface on dev? I always get a ""render() got an unexpected keyword argument 'renderer'"" ... "... File \"/.../python3.7/site-packages/django/forms/boundfield.py\", line 93, in as_widget\n renderer=self.form.renderer,\nTypeError: render() got an unexpected keyword argument 'renderer'\n"
    Tom Kazimiers
    @tomka
    Currently, WebSockets are used mainly to refresh widgets that queue async jobs if there are status updates. These are mainly the Neuron Similarity Widget where NBLAST ranking is performed in the back-end and similarity matrices are created. I believe Will also uses WebSockets to do the same thing for updating the status on his GPU back-end jobs. The "Refresh" button in each widget would of course work too, but it's nice if it updates automatically. We also use for the regular message updates (i.e. a cropping job has finished). Currently it is not used for a real feature isn't needed for CATMAID to work. If widgets want to register own WebSockets message handlers they can use the websocketHandlers option when registering the widget using CATMAID.registerWidget().
    And thanks for the pointer to the broken data view edit page, I will fix it.
    Tom Kazimiers
    @tomka
    @unidesigner: The data view admin interface is fixed both in master (after the tests on the maintenance branch succeed) and dev.
    Stephan Gerhard
    @unidesigner
    Thanks for the explanation and fixing the issue.