galaxyproject/Lobby

Data intensive science for everyone. http://galaxyproject.org | Use now! http://usegalaxy.org

Marius van den Beek

@mvdbeek

@mshamash that’s no problem, just reference them as from_work_dir=“path_to_file” or using mv path_to_file ‘$output'

Michael Shamash

@mshamash

@mvdbeek thanks a lot! I will try that out shortly.

Marius van den Beek

@mvdbeek

Sure, no problem. fwiw from_work_dir is documented at https://docs.galaxyproject.org/en/latest/dev/schema.html#id65 and there’s an example at https://planemo.readthedocs.io/en/latest/writing_standalone.html#wrapping-a-script

Michael Shamash

@mshamash

I'll keep those handy, look like good future references!

Greg Von Kuster

@gregvonkuster

I'm working with a lab doing research with bacterial genomes and I'm working on introducing their pipeline into Galaxy. In many cases the bacterial data consists of a combination of species, so mapping to a specific genome is tricky.

Mapping bacterial data to a referencce genome has 3 options:

the genome is known, so is selected from local cache as is often done in Galaxy
a custom reference is selected - these custom references are a composition of files, and I'm not aware if/how this is currently handled in Galaxy. For example, for the custom reference for Brucella_abortus1 consists of the following 7 files:
- Bab1_define_filter.xlsx
- Bab1_remove_from_analysis.xlsx
- NC_006932-NC_006933.fasta
- NC_006932-NC_006933.gbk
- NC_006932-NC_006933.genome
- NC_006932-NC_006933.gff
The input data is inspected to determine the optimal genome to be mapped.

Does anyone know if this is currently being done in Galaxy? I've done some searching and only found this so far https://help.galaxyproject.org/t/mapping-rna-seq-data-to-a-composite-build-of-bacterial-genomes/1443.

I can wrap the tool for Galaxy that the lab is using, but I want to make sure I'm not re-inventing the wheel.

galaxybot

@galaxybot

[unknown]
Local Galaxy installation: Tool Shed dependencies as packages are not installed [Uncategorized]
Dear Galaxy community,

I am quite new to Galaxy and have set up my own local Galaxy server for learning.

This is installed on Centos, conf…

galaxybot

@galaxybot

[unknown]
Downsampling all regions to specified maximum number of reads [usegalaxy.org support] galaxy-local
Hello,

I see that Samtools View can downsample by specifying a fraction of the total reads to keep or by specifying a target number of tota…

galaxybot

@galaxybot

[unknown]
Installation of Rstudio in Galaxy Europe [usegalaxy.eu support]
Hi everyone,

I’ve tried to install Rstudio in Galaxy Europe a week ago, but the installation is still not finished yet. Is it a general sit…

Thomas Raulet

@tomraulet_gitlab

@nsoranzo Sure I'll open a pull request ;)

Sid Advani

@sadvani6

Hi there! I'm using trimmomatic and would like to QC the data being outputted from there using FastQC followed by MultiQC. When using MultiQC, I have the option of choosing where the data came from - either FastQC or Trimmomatic. Right now I have picked FastQC and supplied it FastQC data. However, my source files were processed by Trimmomatic and could lead to differences in the output. How should I proceed? Thanks.

galaxybot

@galaxybot

[unknown]
merge multiple VCF files [usegalaxy.eu support] workflow
I have multiple VCF files corresponding to 40 different patients. I want to run a batch annotation and GEMINI analysis on them. Can I merge/…

Sid Advani

@sadvani6

Hi there! I'm also stuck trying to get BWA-MEM to work on Trimmomatic files. I get the following errors in the history:
[W::bseq_read] the 1st file has fewer sequences.
[M::mem_pestat] analyzing insert size distribution for orientation FF...
[M::mem_pestat] (25, 50, 75) percentile: (45, 150, 272)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1,

galaxybot

@galaxybot

[unknown]
Galaxy Tool Wrappers for GATK 4? [usegalaxy.org support] tool-dev
Hi -

Just wondering if there are plans or timeline to support GATK 4 + in Galaxy tools?

Thank you!

Gabriel Benavidez

@gabrielbenavidez

Does anyone know how job and file directories get created. At first I only had a "database/files/000" directory and now I see my files are being sent to "database/files/001", do new directories get created when when we create a new history?

Gabriel Benavidez

@gabrielbenavidez

My bigger issue is that the $output.files_path and $output.extra_files_path are pointing to the jobs directory causing errors so I had to manually add the output to 'files/000' but now that it changed to 'files/001' randomly im getting errors.

Jeffrey Thiessen

@JeffreyThiessen

I'm working on updating the outdated blend4j api call importWorkflow(String json)to use the dict version like in bioblend https://bioblend.readthedocs.io/en/latest/api_docs/galaxy/all.html#bioblend.galaxy.workflows.WorkflowClient.import_workflow_dict
Where could I find the information on how that dict is supposed to be formatted?

James Taylor

@jxtx

@gabrielbenavidez The directories are created based on the dataset IDs in the database. You shouldn't have to ever know that though, You should just write to those paths and Galaxy will take care of moving the data back to the database/files directory when the job finishes.

@gabrielbenavidez So, maybe there is a different cause of / way to fix the errors you are seeing. Manually writing to database/files could cause all kinds of problems.

@sadvani6 Are you using trimmomatic with paired inputs, and only trying to align the paired outputs? bwa is unhappy because you are giving it two files with different numbers of reads -- can't be proper paired ends.

galaxybot

@galaxybot

[unknown]
DESeq2 or EdgeR or Limma without replication [usegalaxy.org support] error
Hello, I am trying to do different expression using DESeq2, or EdgeR, or Limma but it always obtain error.

Can I use these tool without rep…

Robert Leach

@hepcat72

Is there a data manager that could essentially do the following to create a reference from ncbi: esearch -db nuccore -query "biomol_rRNA[prop]" | efetch -format fasta? I would like a reference that contains all known rRNAs to use in a QC step for RNA-Seq data. I looked at the existing ncbi fetch data manager tools and they don't appear to support this type of query.

FYI, I've never used a data manager before. This is my first shot. I watched a presentation from the docs on adding references and read through some forum posts.

galaxybot

@galaxybot

[unknown]
$output.files_path points to wrong directory is there a fix? [Uncategorized] tool-dev
Many of my Qiime2 tools are failing in Galaxy because the wrapper outputs additional html files to $output.files_path, however ‘files_path’ …

Patamarerk

@Patamarerk

Hi all, my disk status in cloudman console is full (320/320G) but in my Analyze data page it is just 200 GB. I have purge deleted my old data but it doesn't free more space in the console. How can I free more space? Please help. Thank you very much.

Valid95

@Valid95

Hi everyone, I would like to ask a question about the extraction and annotation of differentially expressed genes procedure. I would like to run Filter data on any column using simple expressions on Galaxy and put the DEseq2 result file but I cannot find my DEseq2 result file while as an input file in the filter box, it doesn't show that file but shows other files. How can I solve that issue? Thank you very much in advance.

galaxybot

@galaxybot

[unknown]
computeMatrix Deeptools [usegalaxy.org support] admin, workflow
How long does it take to complete a job for computeMatrix from Deeptools.

I am not able to get anything even after a day.

Valid95

@Valid95

I have a question related to the normalization of counts. When I run DEseq2 I got lots of zero in my normalized count table. Do you know how can I properly normalize reads and look at if my samples are significantly different compared to the genes?

Gabriel Benavidez

@gabrielbenavidez

@jxtx Thank you for your help. In my case $output.files_path is pointing to 'database/job_directory/###/datasetId/dataset_Id_files' but $output goes to 'files/###/datasetId.dat'. I was wondering about this ### number and how that changes. For me it was 000 then it went to 001 now its at 002. So every job I run is in the 002 directory. So my concern is why does this number keep changing when it's different than the dataset Id?

Nicola Soranzo

@nsoranzo

@gabrielbenavidez That's how Galaxy organises the datasets to not put all the files in the same directory, you really shouldn't rely on any particular naming scheme.

Gabriel Benavidez

@gabrielbenavidez

@nsoranzo Thanks, so my goal is to output an index.html file so the full command is :

qiime demux summarize --i-data=$idata
if str($pn):
--p-n=$pn
end if
--o-visualization=output.qzv
;
qiime tools export --input-path output.qzv --output-path out
&& mkdir -p '$output.files_path'
&& cp -r out/* '$output.files_path'
&& mv '$output.files_path/index.html' '$output'

The logs show :

qiime demux summarize --i-data=/home/manager/galaxy/database/files/002/dataset_2034.dat --p-n=10000 --o-visualization=output.qzv ; qiime tools export --input-path output.qzv --output-path out && mkdir -p '/home/manager/galaxy/database/jobs_directory/001/1918/dataset_2098_files' && cp -r out/* '/home/manager/galaxy/database/jobs_directory/001/1918/dataset_2098_files' && mv '/home/manager/galaxy/database/jobs_directory/001/1918/dataset_2098_files/index.html' '/home/manager/galaxy/database/files/002/dataset_2098.dat'

As you can see the issue is that im getting an error for creating the directory because it is read-only. I just need to get the index.html and its accompanying files in the same output directory.

ps: I apologize if my post is a bit long or if im asking in the wrong place as im new to Galaxy .

Nicola Soranzo

@nsoranzo

@gabrielbenavidez Can you try replacing $output.files_path with $output.extra_files_path ?

Gabriel Benavidez

@gabrielbenavidez

@nsoranzo it's giving me the same directory not sure why.

Nicola Soranzo

@nsoranzo

@gabrielbenavidez If you can share the full tool XML, e.g. on https://gist.github.com/ , it may be easier to find the issue

Gabriel Benavidez

@gabrielbenavidez

@nsoranzo Sure thing I just did. https://gist.github.com/gabrielbenavidez/a873930c11fd46c979770bad16286fb9

galaxybot

@galaxybot

[unknown]
Galaxy migration from sqlite to postgres almost worked [Uncategorized]
Good Day. This is my first post and I’m completely new to Galaxy (2 weeks) so please be gentle. I’ve been given the task to migrate a mult…

galaxybot

@galaxybot

[unknown]
custom data source tool error [Uncategorized] tool-dev
I’m trying to create a custom data source tool that links to our site and when a button is pressed it returns a csv to Galaxy, however I got…

Nicola Soranzo

@nsoranzo

@gabrielbenavidez I don't see anything specifically wrong in your tool, it works fine when tested with planemo. I only made the following changes and added a test:

--- qiime_demux_summarize.xml    2020-01-15 23:58:08.167977529 +0000
+++ qiime_demux_summarize.xml.new    2020-01-15 23:46:40.220604152 +0000
@@ -1,28 +1,33 @@
 <?xml version="1.0" ?>
-<tool id="qiime_demux_summarize" name="qiime demux summarize">
+<tool id="qiime_demux_summarize" name="qiime demux summarize" version="1.0">
     <description> - Summarize counts per sample.</description>
     <requirements>
         <container type="docker">qiime2/core</container>
     </requirements>
     <command ><![CDATA[
-qiime demux summarize --i-data=$idata 
+qiime demux summarize --i-data='$idata'
 #if str($pn):
     --p-n=$pn 
 #end if
 --o-visualization=output.qzv
-;
-qiime tools export --input-path output.qzv --output-path out
-&& mkdir -p '$output.files_path'
-&& cp -r out/* '$output.files_path'
-&& mv '$output.files_path/index.html' '$output'
+&& qiime tools export --input-path output.qzv --output-path out
+&& mkdir -p '$output.extra_files_path'
+&& cp -r out/* '$output.extra_files_path'
+&& mv '$output.extra_files_path/index.html' '$output'
     ]]></command>
     <inputs>
        <param format="qza,no_unzip.zip" label="--i-data: ARTIFACT SampleData[SequencesWithQuality | PairedEndSequencesWithQuality | JoinedSequencesWithQuality] The demultiplexed sequences to be summarized. [required]" name="idata" optional="False" type="data"/>

If you can share how you are testing the tool and the full error you are seeing.

galaxybot

@galaxybot

[unknown]
TRINITY run time [Uncategorized]
I am trying to assembled data using TRINITY. It has been well over 24hrs and it still shows that the analysis job is still running. Is that …

[unknown]
RNA-STAR has been queued [Uncategorized]
I’m rather new to Galaxy. I’m trying to run RNA-STAR using mapped.bam file, The job has been queued now for more than 24 hrs. Is this norma…

BlizeGit

@BlizeGit

when I run "sh run.sh", it shows this error: ERROR: Invalid requirement: u'lib/galaxy/dependencies/pinned-requirements.txt' (from line 1 of requirements.txt)

does anyone know what is the cause

Nicola Soranzo

@nsoranzo

@BlizeGit Is that under which operating system? Windows?

galaxybot

@galaxybot

[unknown]
New Galaxy account [Uncategorized]
Hello,

Please I would like to ask if it is okay and allowed to create another account on Galaxy because I used almost all the available spa…

galaxybot

@galaxybot

[unknown]
Trim Galore latest version update on usegalaxy.eu? [Uncategorized]
Hello,

I am wondering whether it would be possible to update the ‘Trim Galore’ to the latest version. It would be appreciated in advance.

R…

galaxybot

@galaxybot

[nekrut] @Marius van den Beek: it is still impossible to run multiqc on fastqc output ifg fastqc was run on a paired collection. Are you aware of any efforts to address that?

Gabriel Benavidez

@gabrielbenavidez

@nsoranzo Thanks again for trying to help as I've been stuck on it for a while now. I'm testing just by using the Galaxy UI uploading a .qza file and using that as input for the tool. I have changed my script to mirror yours by changing from files_path to extra_files_path.

The command:

galaxy.jobs.command_factory INFO 2020-01-16 11:38:44,838 [p:30062,w:1,m:0] [LocalRunner.work_thread-1] Built script [/home/manager/galaxy/database/jobs_directory/001/1939/tool_script.sh] for tool command 
[qiime demux summarize --i-data='/home/manager/galaxy/database/files/002/dataset_2116.dat' --p-n=10000 --o-visualization=output ; 
qiime tools export --input-path output.qzv --output-path out 
&& mkdir -p '/home/manager/galaxy/database/jobs_directory/001/1939/dataset_2119_files' 
&& cp -r out/* '/home/manager/galaxy/database/jobs_directory/001/1939/dataset_2119_files' 
&& mv '/home/manager/galaxy/database/jobs_directory/001/1939/dataset_2119_files/index.html' '/home/manager/galaxy/database/files/002/dataset_2119.dat']
galaxy.jobs.runners DEBUG 2020-01-16 11:38:44,876 [p:30062,w:1,m:0] [LocalRunner.work_thread-1] (1939) command is: rm -rf working; 
mkdir -p working; 
cd working; 
docker inspect qiime2/core > /dev/null 2>&1
[ $? -ne 0 ] && docker pull qiime2/core > /dev/null 2>&1

docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v /home/manager/galaxy:/home/manager/galaxy:ro -v /home/manager/galaxy/tools/qiime2:/home/manager/galaxy/tools/qiime2:ro -v /home/manager/galaxy/database/jobs_directory/001/1939:/home/manager/galaxy/database/jobs_directory/001/1939:ro -v /home/manager/galaxy/database/jobs_directory/001/1939/working:/home/manager/galaxy/database/jobs_directory/001/1939/working:rw -v /home/manager/galaxy/database/files:/home/manager/galaxy/database/files:rw -w /home/manager/galaxy/database/jobs_directory/001/1939/working --net none --rm --user 1000:1000 qiime2/core /home/manager/galaxy/database/jobs_directory/001/1939/tool_script.sh > ../tool_stdout 2> ../tool_stderr; return_code=$?; cd '/home/manager/galaxy/database/jobs_directory/001/1939';
[ "$GALAXY_VIRTUAL_ENV" = "None" ] && GALAXY_VIRTUAL_ENV="$_GALAXY_VIRTUAL_ENV"; _galaxy_setup_environment True
python "metadata/set.py"; sh -c "exit $return_code"

The error:

execution finished: /home/manager/galaxy/database/jobs_directory/001/1939/galaxy_1939.sh
galaxy.jobs.output_checker DEBUG 2020-01-16 11:39:08,271 [p:30062,w:1,m:0] [LocalRunner.work_thread-1] 
job failed, detected state generic_error, standard error is -
 [mkdir: cannot create directory 
‘/home/manager/galaxy/database/jobs_directory/001/1939/dataset_2119_files’: Read-only file system
]
galaxy.jobs DEBUG 2020-01-16 11:39:08,320 [p:30062,w:1,m:0] [LocalRunner.work_thread-1] (1939) setting dataset 2119 state to ERROR
galaxy.jobs INFO 2020-01-16 11:39:08,405 [p:30062,w:1,m:0] [LocalRunner.work_thread-1] Collecting metrics for Job 1939 in /home/manager/galaxy/database/jobs_directory/001/1939
galaxy.jobs DEBUG 2020-01-16 11:39:08,414 [p:30062,w:1,m:0] [LocalRunner.work_thread-1] job 1939 ended (finish() executed in (138.691 ms))
galaxy.tools.error_reports DEBUG 2020-01-16 11:39:08,423 [p:30062,w:1,m:0] [LocalRunner.work_thread-1] Bug report plugin <galaxy.tools.error_reports.plugins.sentry.SentryPlugin object at 0x7f91e30937d0> generated response None

I'm assuming the error is referring to the mkdir command, so possibly the extra_files_path is pointing to the wrong directory for me.

Nicola Soranzo

@nsoranzo

@gabrielbenavidez This is weird, in my case it is mkdir -p '/tmp/tmpyasl800v/job_working_directory/000/2/working/dataset_ab8002bd-f6c1-4b38-a1d3-dd56b4ce6f7f_files' , where the main difference is that in your case the working subdirectory is missing. What Galaxy version are you using?

I'm using planemo with an updated release_19.09

Gabriel Benavidez

@gabrielbenavidez

@nsoranzo Thats strange, I am using Galaxy version 19.05. Is this something updating my Galaxy instance might help with?

Where communities thrive

People

Repo info

Activity