Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    pvanheusden
    @pvanheusden:matrix.org
    [m]
    doesn't look like its really possible... reading https://whoosh.readthedocs.io/en/latest/parsing.html the query matches any of these fields...
    martenson
    @martenson:matrix.org
    [m]
    I don't think you can search by revision
    pvanheusden
    @pvanheusden:matrix.org
    [m]
    nope and the response doesn't include a revision either
    martenson
    @martenson:matrix.org
    [m]
    it does, in the lineage
    pvanheusden
    @pvanheusden:matrix.org
    [m]
    aha! the trick is to use repositories.search_repositories from bioblend, not tools.search_tools
    martenson
    @martenson:matrix.org
    [m]
    the latter probably searches Galaxy's toolpanel
    pvanheusden
    @pvanheusden:matrix.org
    [m]
    no, it definitely interfaces with the toolshed
    but I think it might be an outdated approach
    martenson
    @martenson:matrix.org
    [m]
    Galaxy proxies the toolshed search, but parsec does not need to I guess.
    with significant overlap between them in what they do. anyway I want the repository search...
    pvanheusden
    @pvanheusden:matrix.org
    [m]
    got it! my "latest" trimmomatic is the one from testtoolshed. (0.38.1 from testtoolshed is revision 79f875f98770)
    martenson
    @martenson:matrix.org
    [m]
    the endpoints mostly differ on what fields the indeces include in search
    pvanheusden
    @pvanheusden:matrix.org
    [m]
    the other trimmomatic - not show in my admin panel - is from main toolshed (revision 51b771646466)
    so in the admin panel it is showing 1 tool - trimmomatic - and the lineage from the toolshed it originated in - testtoolshed - and not showing anything about the other tool (from main toolshed)
    martenson
    @martenson:matrix.org
    [m]
    at this point I think you have enough to fill a Galaxy bug
    the tool ids are probably equivalent and possibly masking each other
    pvanheusden
    @pvanheusden:matrix.org
    [m]
    I'm just going to dig down a little more to understand what Ephemeris is doing
    pvanheusden
    @pvanheusden:matrix.org
    [m]
    before I file a bug... is the admin panel in 21.01 Vue? or still old school?
    pvanheusden
    @pvanheusden:matrix.org
    [m]
    ok... well I submitted an issue... galaxyproject/galaxy#12335
    pvanheusden
    @pvanheusden:matrix.org
    [m]
    I've discovered another problem with the "version matcher" in the workflow editor but I need to dig more to describe it properly...
    Curtis Ross
    @cross12tamu
    Does anyone have an idea on how to permit longer request times for uwsgi/Galaxy? I have some workflows that are bombing from long requests to apollo. The only errors I have extracted from this internal server error 500 is:
    uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 341] during GET /galaxy/api/w
    orkflows/38a667d82ec1b866/download?style=run (50.24.22.31)
    OSError: write error
    Maiken Pedersen
    @maikenp

    Hi there. Any tips on why put() and get() are not available in jupyterlab notebook?

    Using this tool wrapper:

    <tool id="interactive_tool_jupyter_notebook" tool_type="interactive" name="General Jupyter Notebook" version="0.1">
      <description>Default empty notebook, reuse a previous one, or upload a new</description>
      <requirements>
        <container type="docker">quay.io/bgruening/docker-jupyter-notebook:2021-03-05</container>
      </requirements>
        <entry_points>
            <entry_point name="Jupyter Interactive Tool" requires_domain="True">
                <port>8888</port>
                <url>lab</url>
            </entry_point>
        </entry_points>
        <environment_variables>
            <environment_variable name="HISTORY_ID">$__history_id__</environment_variable>
            <environment_variable name="REMOTE_HOST">$__galaxy_url__</environment_variable>
            <environment_variable name="GALAXY_WEB_PORT">8080</environment_variable>
            <environment_variable name="GALAXY_URL">$__galaxy_url__</environment_variable>
            <environment_variable name="API_KEY" inject="api_key" />
        </environment_variables>
        <command detect_errors="aggressive"><![CDATA[
            #import re
            export GALAXY_WORKING_DIR=`pwd` &&
            mkdir -p ./jupyter/outputs/ &&
            mkdir -p ./jupyter/data &&
    
            #set $cleaned_name = re.sub('[^\w\-\.]', '_', str($input.element_identifier))
            ln -sf '$input' './jupyter/data/${cleaned_name}' &&
    
            ## change into the directory where the notebooks are located
            cd ./jupyter/ &&
            export PATH=/home/jovyan/.local/bin:\$PATH &&
    
            #if $mode.mode_select == 'scratch':
                ## copy default notebook
                cp '$__tool_directory__/default_notebook.ipynb' ./ipython_galaxy_notebook.ipynb &&
                jupyter trust ./ipython_galaxy_notebook.ipynb &&
                jupyter lab --allow-root --no-browser  --ServerApp.token=''  &&
                cp ./ipython_galaxy_notebook.ipynb '$jupyter_notebook'
    
            #else:
                #set $cleaned_name = re.sub('[^\w\-\.]', '_', str($input.element_identifier))
                cp '$mode.ipynb' ./${cleaned_name}.ipynb &&
                jupyter trust ./${cleaned_name}.ipynb &&
    
                #if $mode.run_it:
                    jupyter nbconvert --to notebook --execute --output ./ipython_galaxy_notebook.ipynb --allow-errors  ./*.ipynb &&
                #else:
                    jupyter lab --allow-root --no-browser  --ServerApp.token='' &&
                #end if
                cp ./ipython_galaxy_notebook.ipynb '$jupyter_notebook' 
    
            #end if
        ]]>
        </command>
        <inputs>
    
            <conditional name="mode">
                <param name="mode_select" type="select" label="Do you already have a notebook?" help="If not, no problem we will provide you with a default one.">
                    <option value="scratch">Start with a fresh notebook</option>
                    <option value="previous">Load a previous notebook</option>
                </param>
                <when value="scratch"/>
                <when value="previous">
                    <param name="ipynb" type="data" format="ipynb" label="IPython Notebook"/>
                    <param name="run_it" type="boolean" truevalue="true" falsevalue="false" label="Execute notebook and return a new one."
                        help="This option is useful in workflows when you just want to execute a notebook and not dive into the webfrontend."/>
                </when>
            </conditional>
            <param name="input" type="data" optional="true" label="Include data into the environment"/>
    
        </inputs>
        <outputs>
            <data name="jupyter_notebook" format="ipynb" label="Executed Notebook"></data>
        </outputs>
        <tests>
            <test expect_num_outputs="1">
                <param name="mode" value="previous" />
                <param name="ipynb" value="test.ipynb" />
                <param name="run_it" value="true" />
                <output name="jupyter_notebook" file="test.ipynb" ftype="ipynb"/>
            </test>
        </tests>
        <help>
        The Jupyter Notebook is an open-source web application that allows you to create
    24 replies
    Screen Shot 2021-08-12 at 15.48.02.png
    Nate Coraor
    @natefoo:matrix.org
    [m]
    Is anyone aware of a per-destination (or remote Pulsar-side) way to specify container resolvers?
    Nate Coraor
    @natefoo:matrix.org
    [m]
    Bummer, Pulsar unstructured path rewrites don't work on the Singularity image path =(
    @jmchilton: any clever ideas?
    I want to replace /cvmfs/singularity.galaxyproject.org/all/... resolved on the Galaxy side with docker://quay.io/biocontainers/... in the Pulsar job script.
    natefoo
    @natefoo:matrix.org
    [m]
    Not really sure how to make use of that though.
    Torfinn Nome
    @torfinnnome
    We have been using Galaxy Reports to manually monitor tool errors. At some point (not exactly sure when), the page "Jobs In Error Per Month->Month->Jobs in error" started failing (perhaps with 21.01?). Is this a known issue, or did we do anything stupid? Here's the log, if useful: https://gist.github.com/torfinnnome/370a5822828b43d6cfaf923e1301a6e2
    Nicola Soranzo
    @nsoranzo:matrix.org
    [m]
    Reports are not widely used on usegalaxy.* and may sometimes break without developers noticing. I'd suggest to open an issue on the galaxy GitHub repo.
    Torfinn Nome
    @torfinnnome
    Ok, thanks.
    bgruening
    @bgruening:matrix.org
    [m]
    The get/put only works in pthon
    If you use the R kernel or any other kernel please use the commandline tool 'put --help'
    Maiken Pedersen
    @maikenp

    The get/put only works in pthon

    Ok, maybe it is not using the python kernel. Is there any way of ensuring that the python kernel is used by default - i.e. something to add in the tool wrapper? In this tutorial: https://training.galaxyproject.org/training-material/topics/galaxy-interface/tutorials/jupyterlab/tutorial.html it is mentioned that the python kernel is used automatically.

    Ah, but in fact, I see Python (Python3) is used as kernel. So it is supposed to work then.

    Tom
    @tomraulet
    Hi, there ! Playing around with galaxy ansible role version (upgrade and then downgrade; bad idea) led me to be unable to deploy my test galaxy instance. uWSGI does not want to launch. Launching it manually, I had this error : FileNotFoundError: [Errno 2] No such file or directory: '/srv/galaxy/server/lib/galaxy/config/../config_schema.yml' Does anyone have an idea ?
    2 replies
    Fred
    @FredericBGA
    Hello, I play around with Ansible as well.
    I follow the tutorial . I try now to set up the systemd monitoring. But the playbook fails with: {"changed": false, "msg": "AnsibleUndefinedVariable: '__galaxy_user_group' is undefined"}
    Fred
    @FredericBGA
    I don't understand where I need to set this, it seems related to galaxy_group so I tried to add this to group_vars/galaxyservers.yml or galaxy.yml without any success so far. Could you give my a hint?
    13 replies
    Maiken Pedersen
    @maikenp

    Hi there, another question from me related to our custom setup galaxy instance. I have some storage used by the galaxy server mounted from one of the compute nodes in the cluster. Since there are several compute nodes I opted to soft-link the paths to the galaxy storage, so typically I have

    lrwxrwxrwx. 1 galaxy galaxy  22 Aug 23 11:45 galaxy -> /storage/hepp02/galaxy

    The first directory giving problems in the loop checking directories in /srv/galaxy/server/lib/galaxy/config/init.py is the first path in the list which lives on the /storage/galaxy softlinked directory, which is the tmp directory. This is how the folder looks like, as you see the tmp is there.

    [galaxy@galaxy-hepp /]$ ls -lhrt /storage/galaxy/
    total 116K
    drwxr-xr-x. 2 galaxy galaxy   10 Apr 21 15:25 slurm
    drwxr-x---. 2 galaxy galaxy   10 Apr 21 15:43 backups
    drwxr-xr-x. 3 galaxy galaxy   29 May 20 11:14 data
    drwxr-xr-x. 7 galaxy galaxy 4.0K Jun 17 11:41 compiled_templates
    drwxr-x---. 4 galaxy galaxy   40 Aug 10 10:32 files
    drwxr-x---. 4 galaxy galaxy   40 Aug 10 10:32 jobs_directory
    drwxr-xr-x. 9 galaxy galaxy  68K Aug 20 13:36 tmp

    but is not recognized as a directory, and therefore goes into the if-clause in the _ensure_directory method in /srv/galaxy/server/lib/galaxy/config/init.py and the directory is attempted to be created.

    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: maiken checking path: /storage/galaxy/tmp
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: maiken path which is trying to be made: /storage/galaxy/tmp - isdir? False
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: maiken - whoami? galaxy
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: Traceback (most recent call last):
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]:   File "/srv/galaxy/server/lib/galaxy/config/__init__.py", line 447, in _ensure_directory
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]:     os.makedirs(path)
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]:   File "/srv/galaxy/venv/lib64/python3.6/os.py", line 220, in makedirs
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]:     mkdir(name, mode)
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: PermissionError: [Errno 13] Permission denied: '/storage/galaxy/tmp'
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: During handling of the above exception, another exception occurred:
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: Traceback (most recent call last):
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]:   File "/srv/galaxy/server/lib/galaxy/webapps/galaxy/buildapp.py", line 48, in app_factory
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]:     app = galaxy.app.UniverseApplication(global_conf=global_conf, **kwargs)
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]:   File "/srv/galaxy/server/lib/galaxy/app.py", line 77, in __init__
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]:     self.config.check()
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]:   File "/srv/galaxy/server/lib/galaxy/config/__init__.py", line 935, in check
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]:     self._ensure_directory(path)
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]:   File "/srv/galaxy/server/lib/galaxy/config/__init__.py", line 449, in _ensure_directory
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]:     raise ConfigurationError("Unable to create missing directory: {}\n{}".format(path, unicodify(e)))
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: galaxy.exceptions.ConfigurationError: Unable to create missing directory: /storage/galaxy/tmp
    Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: [Errno 13] Permission denied: '/storage/galaxy/tmp'

    So first of all, the soflink is not detected as a directory, second, the directory is attempted to be created, but fails due to permissions. The second part I could probably figure out myself. But the main question is: Why is the softlink not recognized as a directory? In my simple tests it is.

    4 replies
    Gildas Le Corguillé
    @lecorguille
    Hi here,
    Do you know if it’s possible to bypass the registered_user_concurrent_jobs limit for a specific user? With dynamic destinations?
    We have a special bot user which will process automatically [we expect] a lot of jobs and we don’t want it to reach the same limit as a regular registered user.
    Fred
    @FredericBGA
    @lecorguille : we had the same issue: a workflow that launches of lot of Blast, so the user was then unable to launch a single Blast in his history. So we managed to work with dynamic destinations. Blast launched over a workflow go to another destination now.
    Gildas Le Corguillé
    @lecorguille
    So you "move" all your user jobs on a specific destination using dynamic destinations and set destination_user_concurrent_jobs ?
    7 replies
    Martin
    @martinwolst

    Hello,
    Our Galaxy is having issues with accessing reference genomes in /cvmfs/data.galaxyproject.org

    I configured cvmfs with ansible following instructions https://training.galaxyproject.org/training-material/topics/admin/tutorials/cvmfs/tutorial.html

    The problem has been observed when running RNA STAR:

    EXITING because of FATAL ERROR: could not open genome file /cvmfs/data.galaxyproject.org/managed/rnastar/2.7.4a/hg38/hg38/dataset_fdfceb86-2b00-45f3-a3f4-63794ea8182d_files//genomeParameters.txt
    SOLUTION: check that the path to genome files, specified in --genomeDir is correct and the files are present, and have user read permsissions

    I have checked and the file it complains about is present and can be read by all users.

    I have independently reproduced the error with a different dataset using this tutorial: https://training.galaxyproject.org/training-material/topics/transcriptomics/tutorials/ref-based/tutorial.html and it fails at RNA STAR running with the same message as above.

    I am not a domain expert and I don't know how RNA STAR is supposed to work; however, with the 'could not open file' error, it feels like something cvmfs related.

    One thing I have noticed, one of the galaxy project cvmfs servers is trying to connect back approximately every hour three times e.g.:

    Aug 24 08:24:37 bioinfbrad kernel: [UFW BLOCK] IN=enp1s0 OUT= MAC=88:51:fb:5a:62:e6:00:18:71:b2:04:00:08:00 SRC=128.118.250.10 DST=REDACTED LEN=40 TOS=0x00 PREC=0x00 TTL=41 ID=20210 DF PROTO=TCP SPT=80 DPT=60588 WINDOW=0 RES=0x00 RST URGP=0
    Aug 24 08:24:39 bioinfbrad kernel: [UFW BLOCK] IN=enp1s0 OUT= MAC=88:51:fb:5a:62:e6:00:18:71:b2:04:00:08:00 SRC=128.118.250.10 DST=REDACTED LEN=40 TOS=0x00 PREC=0x00 TTL=41 ID=20211 DF PROTO=TCP SPT=80 DPT=60588 WINDOW=0 RES=0x00 RST URGP=0
    Aug 24 08:24:41 bioinfbrad kernel: [UFW BLOCK] IN=enp1s0 OUT= MAC=88:51:fb:5a:62:e6:00:18:71:b2:04:00:08:00 SRC=128.118.250.10 DST=REDACTED LEN=40 TOS=0x00 PREC=0x00 TTL=41 ID=20212 DF PROTO=TCP SPT=80 DPT=60588 WINDOW=0 RES=0x00 RST URGP=0

    128.118.250.10 = cvmfs1-psu0.galaxyproject.org

    As you can see this is being blocked by the firewall.
    1) Why is this happening? I understood cvmfs does not make incoming connections.
    2) is it likely to be related to the problem described above? It doesn't seem to coincide in time

    Suggestions gratefully received!

    2 replies
    Martin
    @martinwolst
    Update: this https://help.galaxyproject.org/t/rna-star-reference-genome-at-usegalaxy-org-resolved/3156/7 suggests that there may (have been?) an issue with RNA STAR and reference data in cvmfs, does anyone know of that is still the case? I'm on Galaxy 20.09, using: toolshed.g2.bx.psu.edu/repos/iuc/rgrnastar/rna_star/2.7.8a+galaxy0
    pvanheusden
    @pvanheusden:matrix.org
    [m]
    is there something wrong with toolshed? I've trying to install data_manager_snpeff_download and its stuck on Cloning for hours.
    pvanheusden
    @pvanheusden:matrix.org
    [m]
    what's the most straightforward way to go from a tools.yaml file which containers tools organised by name / owner & their revisions to a similar list but where the tool version is mentioned for each revision?
    do you have a Jupyter running, which I can look at?