by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • May 20 08:23
  • May 15 17:36
    Erotemic opened #3250
  • May 15 16:26
    Erotemic starred girder/girder
  • May 15 16:21
  • May 09 03:05
    yayima starred girder/girder
  • May 06 17:21
  • May 04 15:21
    zachmullen commented #3249
  • May 04 15:21

    dchiquit on 3242-readme-plugin

    (compare)

  • May 04 15:21

    dchiquit on master

    Add girder-readme plugin The g… Remove debug code Unclobbering folder API object and 6 more (compare)

  • May 04 15:21
    dchiquit closed #3249
  • May 04 15:21
    dchiquit closed #3242
  • Apr 30 17:37
    dchiquit synchronize #3249
  • Apr 30 17:37

    dchiquit on 3242-readme-plugin

    Moved readme test data to prope… (compare)

  • Apr 30 17:11
    dchiquit synchronize #3249
  • Apr 30 17:11

    dchiquit on 3242-readme-plugin

    Move python readme test from ct… (compare)

  • Apr 30 15:30
    dchiquit synchronize #3249
  • Apr 30 15:30

    dchiquit on 3242-readme-plugin

    Use index for readme query (compare)

  • Apr 30 14:51
    dchiquit synchronize #3249
  • Apr 30 14:51

    dchiquit on 3242-readme-plugin

    Fix readme plugin package depen… (compare)

  • Apr 30 14:44
    danlamanna commented #3249
alasla
@alasla
Yeah. I ended up modifying the node/12-buster dockerfile to just install girder & all plugins, large_image & all plugins, histomicstk and histomicsUI from the various github master branches. Did a bunch of testing yesterday afternoon and seems to work fine. Does that seem like a reasonable approach?
I guess also slicer_cli_web and girder_worker too.
David Manthey
@manthey
Its a fine approach. If you are running algorithms (for instance, nuclei detection), there can be some issues with Docker permissions. One of the main benefits of using the dockers built using the deploy_docker.py command is that they can resolve some of these issues.
alasla
@alasla
:thumbsup: I think I’ve got most of the docker stuff with cli plugins sorted, and being able to just build a docker image and deploy to my kubernetes cluster is pretty convenient.
David Manthey
@manthey
Great.
alasla
@alasla
I can’t quite seem to figure out how to use the API Key I generate for a user. The documentation mentions using them, but its not apparant how I either use the API-Key to make authenticated http requests to the api, or how I use it to log in and get a session token.
David Manthey
@manthey
You can use the python girder-client library to facilitate this. An api-key grants limited rights on the behalf of a user and is revokable. When you authenticate with a user or an api-key, you get an session access token. For an api-key, you get a token via POST api/v1/api_key/token?key=<apikey>. This token can then be passed to other requests via the "Girder-Token" header.
alasla
@alasla

Ok. Lots of my ML pipeline working now. Last issue (I hope). I’m trying to take a mask image (with integers as values 0-5). I’ve scaled that image so that its its in base pixels. I’m trying to turn this back into DSA style annotations using some of the methods provided into histomicstk.

contours = histomicstk.annotations_and_masks.masks_to_annotations_handler.get_contours_from_mask(
    fullsize_mask,
    gtcod,
    background_group="Ignore",
    get_roi_contour=False,
    discard_nonenclosed_background=True,
)
contours

Which seems to work ok but not perfectly. Issues I have:

  • discard_nonenclosed_background=True doesn’t seem to work - I still get a background annotation from the ignore group that touches all the edges and has holes.
  • I get a few annotation elements that have negative indices… and I’m not sure how that is possible given that fullsize_mask is just a (43008, 39936) dtype=int8 numpy array.

I can probably manually throw away any background (Ignore) annotations that touch all the edges, and I guess I can toss the ones with negative indicies, but I’m curious if I’ve done something wrong here.

alasla
@alasla
image.png
                  group     ymin     ymax     xmin     xmax  has_holes
5    Ashcroft Score - 0  18538.0  22125.0  34684.0  37673.0        1.0
6    Ashcroft Score - 0  17940.0  24517.0  13156.0  22125.0        1.0
7    Ashcroft Score - 0  14950.0  39467.0   3588.0  29899.0        1.0
10   Ashcroft Score - 0  12558.0  17939.0  19734.0  25115.0        1.0
15   Ashcroft Score - 0   7774.0  19135.0  28106.0  37075.0        1.0
20   Ashcroft Score - 0   2990.0   8969.0  13754.0  19135.0        1.0
21   Ashcroft Score - 0   1794.0  12557.0  16146.0  29899.0        1.0
22   Ashcroft Score - 0    -50.0  43057.0    -50.0  39985.0        1.0
100  Ashcroft Score - 1   5980.0   9567.0  19734.0  22125.0        1.0
195  Ashcroft Score - 4  14950.0  19135.0   7774.0  13753.0        1.0
222              Ignore      0.0  43007.0      0.0  39935.0        1.0
The offending elements are 22 (which goes from -50 to 43057 (which is 50 beyond the right edge of the image), and 222 which is fullsize and background, but not filtered out by discard_nonenclosed_background=True.
alasla
@alasla
Actually, maybe 222 is considered enclosed by 22, which wraps it by 50px on every side.
So the real question is whats up with 22 ?
David Manthey
@manthey
It may take me a bit to answer this adequately. A margin is added internally of 50 pixels to be able to detect edge contours later. I'm not sure why 50 pixels was chosen, and I would have expected it to have been excluded.
alasla
@alasla
lmk if you want test data
i can export my mask and ground truths
David Manthey
@manthey
Yes, please, that would make it quicker for me to figure things out.
I won't get to it until tomorrow, though.
alasla
@alasla
no worries
i’ve worked around it for now by filtering out any contours with negative indices or that touch all edges
but I’d still like to understand what happened
Lee Cooper
@cooperlab
@alasla Can you create an issue on the Github and upload a minimal example and code that causes this?
This would be an issue on the HistomicsTK repo
David Manthey
@manthey
Just to check, what version of opencv-python do you have installed?
alasla
@alasla
Yeah I can try to do that today.
Also I now realize that the docker container I was running that in didn’t have the most recent build of things. So I’ll look at that too.
opencv-python 4.2.0.32
David Manthey
@manthey
So it turns out this is caused by adding padding around the mask to detect edge conditions. Specifically, if there is a GT Code with a value of 0, adding padding with a value of 0 causes the issue.
In our tests, apparently we never have a GT Code with a value of 0, so didn't detect this.
alasla
@alasla
oh that makes sense
you pad the mask with zeros to make edge detection work better
David Manthey
@manthey
The solution will be to select a padding value that isn't present in the GT Codes -- it doesn't matter what value.
Right.
alasla
@alasla
but since one of my mask values was also 0, it just selected the whole thing
ok
fair enough
David Manthey
@manthey
I just manually set the padding to a number not used, and your shared example works as expected.
The routine where padding is added can be modified to be smarter to avoid this issue.
alasla
@alasla
ya’ll still need a github issue? or since you know what it is you can just make the code pad with some other value (not contained in the gtcodes dataframe).
David Manthey
@manthey
No need to add a GitHub issue. I can make a much smaller test case now that I know what the problem is.
alasla
@alasla
Cool.
I wonder if my “background” value had been 0 if it would have just worked
somehow I ended up with my background as 5, because I didn’t really think the values mattered
David Manthey
@manthey
Unless we clearly document that they matter, they really shouldn't.
alasla
@alasla
Ok. Well it sounds like you have a fix in mind, and I have a work-around that seems to be reliable at the moment (worked for 50-ish images I classified last night)
David Manthey
@manthey
Great. I'll try to get a PR in shortly.
alasla
@alasla
Do you prefer me to ask for support here? Or discourse? Or github? Or do you not care which I use.
David Manthey
@manthey
I try to watch them all, so it really doesn't matter. More people watch Discourse and Github, I think.
David Manthey
@manthey
@alasla The HistomicsTK was updated with a fix.
alasla
@alasla
thanks man - i’ll rebuild my docker containers later this week, and try removing the workarounds.