Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Oct 21 02:03
    jma256 starred girder/girder
  • Oct 15 09:23
    SolarisYan starred girder/girder
  • Oct 12 17:32
    zachmullen closed #3228
  • Oct 12 17:31

    zachmullen on master

    Introduce User.verifyLogin Thi… Ensure that login policy is enf… Merge pull request #3229 from X… (compare)

  • Oct 12 17:31
    zachmullen closed #3229
  • Oct 12 17:22
    Xarthisius edited #3229
  • Oct 12 17:07
    Xarthisius synchronize #3229
  • Oct 12 17:04
    Xarthisius synchronize #3229
  • Oct 12 16:54
    zachmullen commented #3229
  • Oct 12 16:36
    Xarthisius synchronize #3229
  • Oct 12 16:15
    Xarthisius synchronize #3229
  • Oct 12 15:35
    Xarthisius synchronize #3229
  • Oct 12 06:27
    tomodian starred girder/girder
  • Oct 09 12:52

    zachmullen on v3.1.3

    (compare)

  • Oct 08 18:09
    nitesh201 starred girder/girder
  • Oct 02 20:24

    zachmullen on no-token

    (compare)

  • Oct 02 20:24

    zachmullen on master

    Add environment variables to up… Merge pull request #3312 from g… (compare)

  • Oct 02 20:24
    zachmullen closed #3312
  • Oct 02 19:58
    jbeezley synchronize #3312
  • Oct 02 19:58

    jbeezley on no-token

    Add environment variables to up… (compare)

Alex Laslavic
@alasla
Any suggestions on how to debug why the “girder mount” process dies from time to time? It’ll run for days at a time before shutting down. Nothing in the log from that process aside from the usual startup stuff..
David Manthey
@manthey
In the DSA we send all output from girder mount to logs/mount.log. I'd expect something there. If you aren't using deply_docker.sh, then this is just "girder mount >logfile 2>&1".
Alex Laslavic
@alasla
yeah i have log output there, but nothing interesting
just the standard connecting to mongo, loading plugins stuff
running: girder mount /opt/girder/mnt 2>&1 >mount.log
David Manthey
@manthey
Is there anything is /var/log/syslog? That might record something if the process was killed externally. I'm not sure where else you'd get any messages.
Alex Laslavic
@alasla
yeah i’ve been hunting to see if OOM killer or something like that killed it
Alex Laslavic
@alasla
I’ve also (separately) had the Java Bridge die a few times and had to restart girder...
Its not seeming to give me the java exception. I suppose it could also be OOM. Let me watch memory for a bit and see if thats it.
David Manthey
@manthey
Perhaps Java is using a lot of memory. We cache references to a number of open WSI files, since opening them has a non-trivial time cost, and only run the bioformats java package once. If it has either a memory leak or consumes too much memory per file that we can't cache more than a few of them, then that could explain it. We'll have to watch what happens when opening numerous images.
David Manthey
@manthey
If you change the Girder config file to log debug messages, girder mount will emit copious messages. I don't know if they will tell much if the problem is OOM.
Alex Laslavic
@alasla
Ok. I’ll watch memory
Curtis Lisle
@curtislisle
I am trying to recreate a system I built with girder & girder_worker initially in May 2020. Now when I install girder_worker, I see the message "girder-worker version xxx doesn't support girder_io". I believe I need girder_io, which versions of girder_worker support it, or do I not need girder_io after all? (I tried with girder_worker master, v0.74, v0.6, etc.)
David Manthey
@manthey
girder_io is part of girder-worker-utils. I'd have to dig into it, but girder_worker master, and anythin 0.6.0 and later (with girder-work-utils installed) should certainly support it.
Curtis Lisle
@curtislisle
Thanks @manthey , it is working for me at the moment. I'll explore versions more carefully in a bit to make sure I have a repeatable installation.
Curtis Lisle
@curtislisle
I have created a Docker container that installs girder. I am not using ansible. I would like to programmatically create an assetstore. From looking at the girder ansible role code and the girder API itself, it looks like I should create an admin user, authenticate as admin and do a POST to girder to create an assetstore. Is there any example script available that does this? I assume the creation of the admin would have to be at the REST level, the remaining operations could be either REST or through the girder client. Does this sound right?
Curtis Lisle
@curtislisle
I found a way to simplify my particular use case. I can already access an admin level account on my new girder instance. In this case, I can just authenticate and do a POST to create the assetstore, correct?
David Manthey
@manthey
There are two approaches to this. One is to use girder-client or another REST method; authenticate through it and use the appropriate rest calls. The second is to have a python script that connects to the database itself (just import girder models and use them directly).
Often, if I'm deploying via docker, I'll do the second with something like "adminUser = User().findOne({'admin': True})" to get any admin user in order to be able to perform other actions appropriately.
Curtis Lisle
@curtislisle
Thanks for these suggestions. I was wondering about the mongo approach.
Alex Laslavic
@alasla
So I can add {"images": ["label", "macro", "*”]} to the large image plugin configs to show associated images, which is super useful. This definately contains the label and slide preview images. Unfortunately they are extremely dark. Is there a way to add any histogram stretch to these images?
Also, is there any way to make the thumbnail shown on the collection page (or folder) to be the slide label or the preview image, rather than a scaled down version of the first frame of a czi file?
sai6a
@sai6a
Hi,
I was just trying the docker scripts provided in the Digital Slide Archive Github code.While running the deploy.sh/build.sh, the docker script is supposed to mount the code of HistomicsUI available in the host system into the docker container.This is not happening.Could someone help please?
I also wanted to understand where does the front end code and how is deployed inside the docker container.
David Manthey
@manthey
There isn't currently a way to change the dynamic range of associated images. We could certainly add the ability to do so. For the CZI files, perhaps auto-adjusting these should be the default.
Currently, the thumbnail in the folder list is always the scaled down image of the first frame of the file. You can also add the label and/or macro images. It wouldn't be hard to add an option to that setting to not show the scaled image.
David Manthey
@manthey
@sai6a, Do you have the source files in the appropriate location or have you specified their location via an environment variable (see https://github.com/DigitalSlideArchive/digital_slide_archive/tree/master/devops)?
The front end code is in the HistomicsUI repository, with components in Girder, large_image, and other plugins.
Alex Laslavic
@alasla

For some reason the “label” and “macro” images in the CZI files aren’t being shown as “label” and “macro” but as “Image28” and “Image29”, so there isn’t any way to show them on the index page that I can figure out.

As for showing them on the item page, it works but the scaling is unusable. I’ll see what I can do on our end. Its possible that we’re taking those images with the wrong camera (the monochrome 16bit one vs the color 8 bit one).

David Manthey
@manthey
I haven't found where the extra images are marked as the label and macro images. We can probably assume that the first two associated images are always label and macro. With the update to the bioformats module, perhaps this information is exposed.
Alex Laslavic
@alasla
Let me see what it looks like with bioformats 4+
i’m pushing the bioformats 4 image now, so I’ll let you know
I’m also asking our Zeiss contact if we can take those label and macro images with the 8bit color camera instead of the 16bit mono camera. But if we need to keep the 16bit mono images it might be nice to have some automatic histogram scaling on any 16bit associated images.
sai6a
@sai6a

@sai6a, Do you have the source files in the appropriate location or have you specified their location via an environment variable (see https://github.com/DigitalSlideArchive/digital_slide_archive/tree/master/devops)?

Yes.I have put the source files in the appropriate location.Both HistomicsUI and Digital Slide Archive I have cloned in the same directory.But if I change anything in the HistomicsUI code in the local repo the changes does not get reflected the code inside the docker container.I was able to make changes in the source code inside the docker container and see the changes getting reflected on the frontend.Even I tried with the setting up of environment variables.It does not work.It would be great help if you could assist furthe.

David Manthey
@manthey
@sai6a, Can you share a screenshot of your directory structure? What is your host OS?
sai6a
@sai6a

@sai6a, Can you share a screenshot of your directory structure? What is your host OS?

I have Ubuntu 16.04 as host OS.Here is the screenshot of directory structure: https://github.com/sai6a/DSA/blob/master/Screenshot%20from%202020-09-14%2011-09-05.png

David Manthey
@manthey
@sai6a, What error are you seeing? If you type "docker inspect dsa_girder" does it show the mounts to the source directories?
sai6a
@sai6a

@sai6a, What error are you seeing? If you type "docker inspect dsa_girder" does it show the mounts to the source directories?

Actually I am making changes to the backend code. To create my own rest service, I made changes to the python files inside the docker container. The thing is I am finding it difficult to deploy my code into the container everytime. Everytime I make a small change in the code, and test its working, I have to copy the changed file to the container, only then i would be able to see my changes. I am thinking that whenever I make a code change locally, it should be mounted into the container.

This is what I get as response when i run "docker inspect dsa_girder"
https://raw.githubusercontent.com/sai6a/DSA/master/dockerinspect.log

David Manthey
@manthey
Based on the docker inspect command, the source directories aren't mounted. The deploy.sh script checks if the directories exist and only mounts them if they so. bash's check for a directories existence must be returning false (e.g., -d "$HISTOMICS_SOURCE_FOLDER" returns false).
If, from a bash prompt, running if [ -d "$HISTOMICS_SOURCE_FOLDER" ]; then echo "good"; else echo "bad"; fi prints "bad" then your environment variable or your folder is not set as expected.
sai6a
@sai6a

If, from a bash prompt, running if [ -d "$HISTOMICS_SOURCE_FOLDER" ]; then echo "good"; else echo "bad"; fi prints "bad" then your environment variable or your folder is not set as expected.

do i run it in the host os or inside the container?
If i run it in the host os, I am getting GOOD.
If I run it inside the container, meaning after I connect to the docker container with the connect.sh docker script, I am getting BAD.
How do I fix this?

David Manthey
@manthey
On the host. If you see GOOD on the host, then try removing your existing containers and recreating them (deploy.sh rm followed by deploy.sh).
sai6a
@sai6a

The issue is resolved! Thank you so much for the help @manthey.

On the host. If you see GOOD on the host, then try removing your existing containers and recreating them (deploy.sh rm followed by deploy.sh).

The issue is resolved! Thank you so much for the help @manthey.

sai6a
@sai6a

Hi @manthey ,
I am not able to run build.sh docker script.It is halted in middle.I tried many times but the code is not built.
tele@tele-H97M-D3H159:~$ digital_slide_archive/devops/build.sh
digital_slide_archive/devops/build.sh: line 9: ${@@Q}: bad substitution

@girder/meta-build@ build /opt/girder/girder/web_client
grunt "--girder-version=3.1.3.dev6+g317962d15" "--static-path=/home/ubuntu/.virtualenvs/girder/share/girder/static" "--static-public-path=/static" "--no-progress=false" "--watch" "webpack:plugin_histomicsui" "--env=dev"

Running "webpack:plugin_histomicsui" (webpack) task
Version: webpack 2.7.0
Asset Size Chunks Chunk Names
plugin.min.js 1.73 MB 0 [emitted] [big] plugins/histomicsui/plugin
plugin.min.css 21.4 kB 0 [emitted] plugins/histomicsui/plugin
plugin.min.js.map 960 kB 0 [emitted] plugins/histomicsui/plugin
plugin.min.css.map 91 bytes 0 [emitted] plugins/histomicsui/plugin
favicon.png 5.07 kB [emitted]
^X^C
Aborted!

Daniel Romberg
@djromberg
Hi, how can I invoke the unit tests for plugins? I am currently extending the LDAP plugin for arbitrary filter operations and extended its unit test. Before sending a PR, I want to ensure the test passes :)
I forked, modified the Dockerfile for RUN pip install -e plugins/ldap for a locally built image (which worked well) and it seems that the tests are also included. I just can't find the information how to run them.
David Manthey
@manthey
There are two sorts of tests in Girder's code base. There are some older tests that are run through cmake, and newer tests run through tox. The ldap plugin has the older tests. See https://girder.readthedocs.io/en/latest/development.html?highlight=cmake#legacy-unittest-tests for brief instructions. Once you've run cmake to configure everything, you can run just the tests for the ldap plugin via "ctest -V -R server_ldap.ldap".
Daniel Romberg
@djromberg
I see, I will try that, thanks!
RazerEdge64
@RazerEdge64
Hi, I've been through the api docs for girder and saw how it creates a user. But, is there a way to add extra field in the user collection? like login, email, firstname, lastname and maybe some other field to add ? Couldn't find it in the api : https://girder.readthedocs.io/en/stable/api-docs.html#module-girder.models.user
Brandon Davis
@subdavis

@RazerEdge64 a bit more detail may be helpful, but this might get you started. The User model (mongo collection) already has login, email, firstName, and lastName [1]. You've likely already noticed this, because you provide this info when you register for an account.

If you wan to extend the User model with additional properties, you'd need to create a small Girder plugin. Plugins are created as standalone python packages, you can check the docs and take a look at the cookiecutter [2].

As far as adding new properties to existing models, how you go about this will likely depend on your use case. Typically, you'll write a new endpoint that modifies some new property and use that endpoint from the client app. Here's the general docs on plugin development [3]. You'll have to make sure to expose the new field with exposeField [4].

This isn't the most straightforward thing to do, so the more specific info you can give about what you're trying to accomplish, the more helpful we can probably be.

I did notice above that you are trying to create a login form for a react app. If this is related, I would recommend creating a plugin that adds a new endpoint for user creation. You could basically copy the code from [5] and add your own custom properties all at once.

[1] https://github.com/girder/girder/blob/master/girder/api/v1/user.py#L149-L152
[2] https://github.com/girder/cookiecutter-girder-plugin
[3] https://girder.readthedocs.io/en/stable/plugin-development.html#defining-a-plugin-descriptor-class
[4] https://girder.readthedocs.io/en/stable/api-docs.html?highlight=exposefield#girder.models.model_base.Model.exposeFields
[5] https://github.com/girder/girder/blob/master/girder/api/v1/user.py#L160-L183

RazerEdge64
@RazerEdge64
@subdavis thanks !