afterScript
? I'd like to be able to read some of the files generated by the script:
block and store information in a map alongside other metadata to be pass down a channel rather than throwing more tiny files into the output channel.
docker
command, make sure there's docker.enabled=false
in your config (I guess this is a glitch with a nf-core pipeline conf ..)
docker.enabled=false
. That may well be it (now testing). It was set to true in our pipeline (globally), but only made a difference for multiqc
somehow.
multiqc
not found. This must be simple ... more coffee needed.
docker.enabled = true
and it fails with 'multiqc not found' if docker.enabled = false
nextflow run my_flow.nf
as background process, right? if i need multiple of them i just forked nf-process and let them run as background on this frontend til they exit?
sbatch nextflow run
something like that?
Allright, I found it ... it lived in our conf/base.config
that came from nf-core/rnaseq. It has this:
withName: multiqc {
executor = 'local'
}
After reading this old issue nextflow-io/nextflow#693 it prompted me to search for local executors ... there it was in the base config.
no
!
process download_reads {
/*
This process downloads fastq.gz files from ENA accessions. I run enaDataGet for accessions which point to
only one sequencing lane, i.e. one pair of fastq files. It fails for accessions that point to multiple
lanes, so in that case I use enaGroupGet instead. Both of these utilities are supplied by ENA:
https://github.com/enasequence/enaBrowserTools
*/
tag "$accession"
publishDir 'reads', mode: 'symlink'
cpus 1
time '1h'
errorStrategy 'retry'
maxRetries 3
memory {8.GB * task.attempt}
time {1.hour * task.attempt}
input:
val accession from accessions
output:
set accession, file('**.fastq.gz') into downloaded_reads
script:
"""
enaDataGet -f fastq $accession || enaGroupGet -f fastq $accession
"""
}
ls | wc -c
etc one-liner that returns a non-zero if one or more files are missing ?
find . -name '*.fastq.gz' | wc -l
should do the trick
Looks really good, @pditommaso !