repositories.search_repositories from bioblend, not tools.search_tools
internal server error 500 is:uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 341] during GET /galaxy/api/w
orkflows/38a667d82ec1b866/download?style=run (50.24.22.31)
OSError: write error
Hi there. Any tips on why put() and get() are not available in jupyterlab notebook?
Using this tool wrapper:
<tool id="interactive_tool_jupyter_notebook" tool_type="interactive" name="General Jupyter Notebook" version="0.1">
<description>Default empty notebook, reuse a previous one, or upload a new</description>
<requirements>
<container type="docker">quay.io/bgruening/docker-jupyter-notebook:2021-03-05</container>
</requirements>
<entry_points>
<entry_point name="Jupyter Interactive Tool" requires_domain="True">
<port>8888</port>
<url>lab</url>
</entry_point>
</entry_points>
<environment_variables>
<environment_variable name="HISTORY_ID">$__history_id__</environment_variable>
<environment_variable name="REMOTE_HOST">$__galaxy_url__</environment_variable>
<environment_variable name="GALAXY_WEB_PORT">8080</environment_variable>
<environment_variable name="GALAXY_URL">$__galaxy_url__</environment_variable>
<environment_variable name="API_KEY" inject="api_key" />
</environment_variables>
<command detect_errors="aggressive"><![CDATA[
#import re
export GALAXY_WORKING_DIR=`pwd` &&
mkdir -p ./jupyter/outputs/ &&
mkdir -p ./jupyter/data &&
#set $cleaned_name = re.sub('[^\w\-\.]', '_', str($input.element_identifier))
ln -sf '$input' './jupyter/data/${cleaned_name}' &&
## change into the directory where the notebooks are located
cd ./jupyter/ &&
export PATH=/home/jovyan/.local/bin:\$PATH &&
#if $mode.mode_select == 'scratch':
## copy default notebook
cp '$__tool_directory__/default_notebook.ipynb' ./ipython_galaxy_notebook.ipynb &&
jupyter trust ./ipython_galaxy_notebook.ipynb &&
jupyter lab --allow-root --no-browser --ServerApp.token='' &&
cp ./ipython_galaxy_notebook.ipynb '$jupyter_notebook'
#else:
#set $cleaned_name = re.sub('[^\w\-\.]', '_', str($input.element_identifier))
cp '$mode.ipynb' ./${cleaned_name}.ipynb &&
jupyter trust ./${cleaned_name}.ipynb &&
#if $mode.run_it:
jupyter nbconvert --to notebook --execute --output ./ipython_galaxy_notebook.ipynb --allow-errors ./*.ipynb &&
#else:
jupyter lab --allow-root --no-browser --ServerApp.token='' &&
#end if
cp ./ipython_galaxy_notebook.ipynb '$jupyter_notebook'
#end if
]]>
</command>
<inputs>
<conditional name="mode">
<param name="mode_select" type="select" label="Do you already have a notebook?" help="If not, no problem we will provide you with a default one.">
<option value="scratch">Start with a fresh notebook</option>
<option value="previous">Load a previous notebook</option>
</param>
<when value="scratch"/>
<when value="previous">
<param name="ipynb" type="data" format="ipynb" label="IPython Notebook"/>
<param name="run_it" type="boolean" truevalue="true" falsevalue="false" label="Execute notebook and return a new one."
help="This option is useful in workflows when you just want to execute a notebook and not dive into the webfrontend."/>
</when>
</conditional>
<param name="input" type="data" optional="true" label="Include data into the environment"/>
</inputs>
<outputs>
<data name="jupyter_notebook" format="ipynb" label="Executed Notebook"></data>
</outputs>
<tests>
<test expect_num_outputs="1">
<param name="mode" value="previous" />
<param name="ipynb" value="test.ipynb" />
<param name="run_it" value="true" />
<output name="jupyter_notebook" file="test.ipynb" ftype="ipynb"/>
</test>
</tests>
<help>
The Jupyter Notebook is an open-source web application that allows you to create
/cvmfs/singularity.galaxyproject.org/all/... resolved on the Galaxy side with docker://quay.io/biocontainers/... in the Pulsar job script.
The get/put only works in pthon
Ok, maybe it is not using the python kernel. Is there any way of ensuring that the python kernel is used by default - i.e. something to add in the tool wrapper? In this tutorial: https://training.galaxyproject.org/training-material/topics/galaxy-interface/tutorials/jupyterlab/tutorial.html it is mentioned that the python kernel is used automatically.
Ah, but in fact, I see Python (Python3) is used as kernel. So it is supposed to work then.
FileNotFoundError: [Errno 2] No such file or directory: '/srv/galaxy/server/lib/galaxy/config/../config_schema.yml' Does anyone have an idea ?
systemd monitoring. But the playbook fails with: {"changed": false, "msg": "AnsibleUndefinedVariable: '__galaxy_user_group' is undefined"}
Hi there, another question from me related to our custom setup galaxy instance. I have some storage used by the galaxy server mounted from one of the compute nodes in the cluster. Since there are several compute nodes I opted to soft-link the paths to the galaxy storage, so typically I have
lrwxrwxrwx. 1 galaxy galaxy 22 Aug 23 11:45 galaxy -> /storage/hepp02/galaxyThe first directory giving problems in the loop checking directories in /srv/galaxy/server/lib/galaxy/config/init.py is the first path in the list which lives on the /storage/galaxy softlinked directory, which is the tmp directory. This is how the folder looks like, as you see the tmp is there.
[galaxy@galaxy-hepp /]$ ls -lhrt /storage/galaxy/
total 116K
drwxr-xr-x. 2 galaxy galaxy 10 Apr 21 15:25 slurm
drwxr-x---. 2 galaxy galaxy 10 Apr 21 15:43 backups
drwxr-xr-x. 3 galaxy galaxy 29 May 20 11:14 data
drwxr-xr-x. 7 galaxy galaxy 4.0K Jun 17 11:41 compiled_templates
drwxr-x---. 4 galaxy galaxy 40 Aug 10 10:32 files
drwxr-x---. 4 galaxy galaxy 40 Aug 10 10:32 jobs_directory
drwxr-xr-x. 9 galaxy galaxy 68K Aug 20 13:36 tmpbut is not recognized as a directory, and therefore goes into the if-clause in the _ensure_directory method in /srv/galaxy/server/lib/galaxy/config/init.py and the directory is attempted to be created.
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: maiken checking path: /storage/galaxy/tmp
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: maiken path which is trying to be made: /storage/galaxy/tmp - isdir? False
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: maiken - whoami? galaxy
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: Traceback (most recent call last):
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: File "/srv/galaxy/server/lib/galaxy/config/__init__.py", line 447, in _ensure_directory
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: os.makedirs(path)
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: File "/srv/galaxy/venv/lib64/python3.6/os.py", line 220, in makedirs
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: mkdir(name, mode)
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: PermissionError: [Errno 13] Permission denied: '/storage/galaxy/tmp'
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: During handling of the above exception, another exception occurred:
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: Traceback (most recent call last):
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: File "/srv/galaxy/server/lib/galaxy/webapps/galaxy/buildapp.py", line 48, in app_factory
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: app = galaxy.app.UniverseApplication(global_conf=global_conf, **kwargs)
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: File "/srv/galaxy/server/lib/galaxy/app.py", line 77, in __init__
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: self.config.check()
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: File "/srv/galaxy/server/lib/galaxy/config/__init__.py", line 935, in check
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: self._ensure_directory(path)
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: File "/srv/galaxy/server/lib/galaxy/config/__init__.py", line 449, in _ensure_directory
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: raise ConfigurationError("Unable to create missing directory: {}\n{}".format(path, unicodify(e)))
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: galaxy.exceptions.ConfigurationError: Unable to create missing directory: /storage/galaxy/tmp
Aug 23 11:31:40 galaxy-hepp.hpc.uio.no uwsgi[4191392]: [Errno 13] Permission denied: '/storage/galaxy/tmp'So first of all, the soflink is not detected as a directory, second, the directory is attempted to be created, but fails due to permissions. The second part I could probably figure out myself. But the main question is: Why is the softlink not recognized as a directory? In my simple tests it is.
registered_user_concurrent_jobs limit for a specific user? With dynamic destinations?Hello,
Our Galaxy is having issues with accessing reference genomes in /cvmfs/data.galaxyproject.org
I configured cvmfs with ansible following instructions https://training.galaxyproject.org/training-material/topics/admin/tutorials/cvmfs/tutorial.html
The problem has been observed when running RNA STAR:
EXITING because of FATAL ERROR: could not open genome file /cvmfs/data.galaxyproject.org/managed/rnastar/2.7.4a/hg38/hg38/dataset_fdfceb86-2b00-45f3-a3f4-63794ea8182d_files//genomeParameters.txt
SOLUTION: check that the path to genome files, specified in --genomeDir is correct and the files are present, and have user read permsissionsI have checked and the file it complains about is present and can be read by all users.
I have independently reproduced the error with a different dataset using this tutorial: https://training.galaxyproject.org/training-material/topics/transcriptomics/tutorials/ref-based/tutorial.html and it fails at RNA STAR running with the same message as above.
I am not a domain expert and I don't know how RNA STAR is supposed to work; however, with the 'could not open file' error, it feels like something cvmfs related.
One thing I have noticed, one of the galaxy project cvmfs servers is trying to connect back approximately every hour three times e.g.:
Aug 24 08:24:37 bioinfbrad kernel: [UFW BLOCK] IN=enp1s0 OUT= MAC=88:51:fb:5a:62:e6:00:18:71:b2:04:00:08:00 SRC=128.118.250.10 DST=REDACTED LEN=40 TOS=0x00 PREC=0x00 TTL=41 ID=20210 DF PROTO=TCP SPT=80 DPT=60588 WINDOW=0 RES=0x00 RST URGP=0
Aug 24 08:24:39 bioinfbrad kernel: [UFW BLOCK] IN=enp1s0 OUT= MAC=88:51:fb:5a:62:e6:00:18:71:b2:04:00:08:00 SRC=128.118.250.10 DST=REDACTED LEN=40 TOS=0x00 PREC=0x00 TTL=41 ID=20211 DF PROTO=TCP SPT=80 DPT=60588 WINDOW=0 RES=0x00 RST URGP=0
Aug 24 08:24:41 bioinfbrad kernel: [UFW BLOCK] IN=enp1s0 OUT= MAC=88:51:fb:5a:62:e6:00:18:71:b2:04:00:08:00 SRC=128.118.250.10 DST=REDACTED LEN=40 TOS=0x00 PREC=0x00 TTL=41 ID=20212 DF PROTO=TCP SPT=80 DPT=60588 WINDOW=0 RES=0x00 RST URGP=0128.118.250.10 = cvmfs1-psu0.galaxyproject.org
As you can see this is being blocked by the firewall.
1) Why is this happening? I understood cvmfs does not make incoming connections.
2) is it likely to be related to the problem described above? It doesn't seem to coincide in time
Suggestions gratefully received!
data_manager_snpeff_download and its stuck on Cloning for hours.
tools.yaml file which containers tools organised by name / owner & their revisions to a similar list but where the tool version is mentioned for each revision?