Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Marius van den Beek
    @mvdbeek
    Why that doesn’t happen in tmpdir is not clear to me, so many of the singularity defaults are pretty odd
    M Bernt
    @bernt-matthias
    And this path I also set in job_conf: <param id=“container_image_cache_path”>/path/to/images</param>?
    Marius van den Beek
    @mvdbeek
    That is supposed to work, but if you have just one path I’d set it in galaxy.yml
    because I don’t think that is properly implemented on the destination level
    M Bernt
    @bernt-matthias
    OK. I kind of succeeded with SINGULARITY_CACHEDIR .. but I need to figure out a way to configure it such that it can be used by all users. In my real user setup the job's singularity exec will initialize the cache dir and so it will belong to only a single user.
    Marius van den Beek
    @mvdbeek
    It sounds like you’d want to fetch the containers before the run, so that containers won’t be fetched during execution
    M Bernt
    @bernt-matthias
    But maybe this is not an issue once I'm able to access the container dependency page in the admin UI.
    (which still times out).
    Marius van den Beek
    @mvdbeek
    yes, I think that would work. You can also use the API to fetch containers if it still times out
    M Bernt
    @bernt-matthias
    I guess the container would then by pulled by the galaxy process itself. So I would need to set SINGULARITY_CACHEDIR for the galaxy process, or?
    Marius van den Beek
    @mvdbeek
    yes
    you should also be able to set <env id="SINGULARITY_CACHEDIR”>-/some/path/</env> in your destination, but that will not pull the image to container_image_cache_path (during job execution)
    M Bernt
    @bernt-matthias
    Do you have a link for me how to do this via the api?
    Marius van den Beek
    @mvdbeek
    POST to /api/container_resolvers/resolve/install with tool_id in the payload
    M Bernt
    @bernt-matthias
    would it be a good idea to clean the cache regularly, e.g. with singularity cache clean .. alternatively I could put it on a path where files have a 30d lifetime.
    Marius van den Beek
    @mvdbeek
    Yes, those are needed only once. I really don’t understand why singularity doesn’t do this automatically
    M Bernt
    @bernt-matthias
    True, but only if you use the images. One could also go a different path and just let the jobs call singularity exec reusing a shared cache. Potential downside is that you always get the latest build which might break as we now. Also there is a warning in the docs regarding parallel use of the cache ..
    On the positive side admins do not need to take care of installing containers ...
    jacorvar
    @jacorvar_gitlab

    Yes and no. We influence our community a lot with our playbooks.

    Please @bgruening @natefoo @hexylena let us know as soon as you reach an agreement regarding the chosen OS. We're about to upgrade the OS of our Galaxy servers and would like to know the community consensus about the OS's for Galaxy.

    Helena
    @hexylena
    @jacorvar_gitlab please choose whatever works for you. Everything works with the playbooks, I'm not sure you'll see one community opinion. Many of the community sites used centos, but, nowhere was it written "it is recommended to use centos". it has always been up to individual sites.
    hexylena
    @hexylena:matrix.org
    [m]
    Also some used debian. AU i think used ubuntu. It's always been a mix! :)
    Nicola Soranzo
    @nsoranzo
    BTW, there seems to be an alternative for CentOS coming up: https://rockylinux.org/
    jacorvar
    @jacorvar_gitlab

    BTW, there seems to be an alternative for CentOS coming up: https://rockylinux.org/

    sure, but we still don't know how far they will work for that and the future community engagement on that project. We look expectant at it.

    Nicola Soranzo
    @nsoranzo
    Yep, just wanted to share the news if anyone has missed it
    Jennifer Hillman-Jackson
    @jennaj
    Interesting question that I don't know the answer to. Anyone running a galaxy server for a commercial organization over 200 people? Willing to share advice re "In 2020, anaconda’s TOS changed and organizations larger than 200 people will have to pay for the use of the anaconda defaults repository."?? Q&A here at Galaxy Help: https://help.galaxyproject.org/t/anaconda-defaults-repository-commercial-edition/4993 Thanks!
    Björn Grüning
    @bgruening
    I will take that.
    cat-bro
    @cat-bro
    Seeking help getting all tool IDs for a galaxy instance: We have conda_auto_install: false in galaxy.yml and we run our jobs on a slurm cluster. For galaxy built-in tools we need to create the environments because they do not auto_install. I'm trying to find all of these tools through the API so that I can set up conda environments for them. However, /api/tools does not return all of these. CONVERTER_bam_to_qname_sorted_bam is not returned. api/tools/CONVERTER_bam_to_qname_sorted_bam/requirements does return the tool's requirements. Is there a method I could use to get all of the tools including those not returned by /api/tools?
    Nate Coraor
    @natefoo:matrix.org
    [m]
    @cat: Try /api/tools?in_panel=false
    wm75 (Wolfgang Maier)
    @wm75:matrix.org
    [m]
    Nate Coraor: yeah, was going to suggest that, too, but when I tested it gainst .au it didn't help :(
    Nate Coraor
    @natefoo:matrix.org
    [m]
    Interesting, I tested it against .org to make sure I wasn't making things up and it worked there.
    wm75 (Wolfgang Maier)
    @wm75:matrix.org
    [m]
    I mean, it worked in terms of showing additional results, but not this specific tool.
    Marius van den Beek
    @mvdbeek
    That would suggest the converter is not loaded
    Nate Coraor
    @natefoo:matrix.org
    [m]
    Ah indeed.
    Marius van den Beek
    @mvdbeek
    (maybe ebcause it’s missing dependencies)
    Nate Coraor
    @natefoo:matrix.org
    [m]
    Might be in the install DB (and therefore appears in the Admin UI) but fails to load?
    Marius van den Beek
    @mvdbeek
    This is a built-in converter, they shoudln’t be in the install db
    Nate Coraor
    @natefoo:matrix.org
    [m]
    Ok, wasn't sure, thought it might be from the TS.
    wm75 (Wolfgang Maier)
    @wm75:matrix.org
    [m]
    wait, when I'm trying this with .org, it doesn't show CONVERTER_bam_to_qname_sorted_bam either.
    Nate Coraor
    @natefoo:matrix.org
    [m]
    I am not sure if that's loaded on .org
    Marius van den Beek
    @mvdbeek
    It should be
    https://usegalaxy.org/root?tool_id=CONVERTER_bam_to_qname_sorted_bam
    Nolan Woods
    @innovate-invent
    @cat-bro are you wanting all installed tools or all installed repositories? there is an important difference
    ah, never mind, I misunderstood what you are trying to get a list of
    Torfinn Nome
    @torfinnnome
    Anyone using Ubuntu, Ansible and Singularity? What is the recommended way of installing Singularity? (I only find a really old version using NeuroDebian, and would prefer not to compile it.)
    Helena Rasche
    @hexylena:matrix.org
    [m]
    We have a role for it
    one sec
    huh, we don't. nevermind. I thought we did at one point.
    Torfinn Nome
    @torfinnnome
    Ok. :)
    Helena Rasche
    @hexylena:matrix.org
    [m]
    https://github.com/CyVerse-Ansible/ansible-singularity
    ah, it was removed because centos I guess now ships new enough singularity for @gmauro :)