IMO these catalogs will become much more relevant when they include 1) containers, 2)programatically access/format to the command line options 3) format for type of data in/out.
This will allow the super fast deployment of tools into workflows such as NF.
. fromFilePairs
. Or other ways of grouping inputfiles. My use case is that I have samples sequenced over multiple lanes and want to submit them all to one process. eg. 3_160412_AC8B7GANXX_P4701_1001_1.fastq.gz 3_160412_AC8B7GANXX_P4701_1001_2.fastq.gz 4_160412_AC8B7GANXX_P4701_1001_1.fastq.gz 4_160412_AC8B7GANXX_P4701_1001_2.fastq.gz 5_160412_AC8B7GANXX_P4701_1001_1.fastq.gz 5_160412_AC8B7GANXX_P4701_1001_2.fastq.gz
.fromFilePairs
channel.fromFilePairs('?_*_{1,2}.fastq.gz ', size:6 )
should work
delhommet@X130121:~/Documents/GitHub_repos/nextflow_toys$ cat test_params.nf
#!/usr/bin/env nextflow
/*
* Try to change an input parameter keeping same name
*/
params.foo = null
if (params.foo == "x") { params.foo = "new_foo1"; log.info "I am trying to set foo as new_foo1" }
else if (params.foo == "y") { params.foo = "new_foo2" ; log.info "I am trying to set foo as new_foo2"}
else {log.info "I don't want to change foo"}
log.info "The value of foo is : ${params.foo}"
delhommet@X130121:~/Documents/GitHub_repos/nextflow_toys$ nextflow run test_params.nf --foo=x
N E X T F L O W ~ version 0.24.4
Launching `test_params.nf` [small_hopper] - revision: e427052373
I am trying to set foo as new_foo1
The value of foo is : x
params
cannot be overridden
ERROR ~ Cannot find any reads matching: *P4701_1001_{1,2}.fastq.gz
Channel.fromFilePairs('{3,4,5}_*_{1,2}.fastq.gz', size:6).println()
[, [/Users/pditommaso/projects/nextflow/3_160412_AC8B7GANXX_P4701_1001_1.fastq.gz, /Users/pditommaso/projects/nextflow/3_160412_AC8B7GANXX_P4701_1001_2.fastq.gz, /Users/pditommaso/projects/nextflow/4_160412_AC8B7GANXX_P4701_1001_1.fastq.gz, /Users/pditommaso/projects/nextflow/4_160412_AC8B7GANXX_P4701_1001_2.fastq.gz, /Users/pditommaso/projects/nextflow/5_160412_AC8B7GANXX_P4701_1001_1.fastq.gz, /Users/pditommaso/projects/nextflow/5_160412_AC8B7GANXX_P4701_1001_2.fastq.gz]]
Before I put in a feature request: When a job fails, it would be really useful to get the work dir as is given when the job is submitted. The message for example:
WARN: Process `dpa_alignment (ltn - UPP - DPA - 1000)` terminated with an error exit status (126) -- Execution is retried (1)
makes it almost impossible to track back when I have thousands of jobs running (and failing).
I would propose something like:
WARN: Process `dpa_alignment (ltn - UPP - DPA - 1000)` [da/8e42c6] terminated with an error exit status (126) -- Execution is retried (1)
[da/8e42c6] WARN: Process `dpa_alignment (ltn - UPP - DPA - 1000)`terminated with an error exit status (126) -- Execution is retried (1)
params {
igenomes_base = './iGenomes/'
genomes {
'GRCh37' {
bed12 = { "${params.igenomes_base}/Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/genes.bed" }
${params.igenomes_base}
isn't substituting in to the file path as I expect
igenomes$_run_closure1$_closure2$_closure3$_closure28@669513d8
fromPath
channel we get the following:ERROR ~ org.codehaus.groovy.runtime.GStringImpl cannot be cast to java.nio.file.FileSystem
bed12 = "${params.igenomes_base}/Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/genes.bed"
/var/spool/slurmd/job206121/slurm_script: line 71: /lupus/proj/ngi2016003/nobackup/NGI/ANALYSIS/P8455/rnaseq_ngi_v1.3_test/${params.igenomes_base}/Mus_musculus/Ensembl/GRCm38/Sequence/STARIndex: bad substitution
println params
[igenomes_base:./iGenomes/, genomes:[GRCh37:[bed12:./iGenomes//Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/genes.bed]]]
.comman.run
sbatch file;ln -s /lupus/proj/ngi2016003/nobackup/NGI/ANALYSIS/P8455/rnaseq_ngi_v1.3_test/${params.igenomes_base}/Mus_musculus/Ensembl/GRCm38/Sequence/STARIndex STARIndex