work
dir to be somewhere else? I'm processing a long timeseries and I'm afraid I cannot fit it entirely on my disk
-w
cli option
nextflow.config
?
workDir = '/some/path'
fromFilePairs
nextflow does not return any file, but if is use the same pattern on my shell, I can find them
~/your-regex/
(if I'm not wrong) to fromPath
MergeBamAlignment
?
BwaMem_output
and FastqToSam_output
, right ?
pair_id
along with the output file
Bwa_mem
process, replace
output:
file "bwamem.sam" into BwaMem_output
output:
set pair_id, file("bwamem.sam") into BwaMem_output
FastqToSam
process
pair_id
Channel
.fromFilePairs( "/data/workspace/data/Samples/NA12878-rep7_S7_L00*_R{1,2}_001.fastq.gz", flat: true)
.into { reads; reads2 }
nextflow clean
Hi,
I managed to reproduce my storeDir
issue by an unfortunate coincidence... This time, the whole pipeline was terminated due to critical failure in some process. Because of that, some upstream processes that were copying results to storeDir
were also terminated/aborted etc. I relaunched the pipeline (actually without -resume
flag) and it took off with whatever was in the storeDir
folder - regrardless of whether the upstream was aborted, results were incomplete or whatever. It seems that the only thing nextflow
cares in such cases is the content of storeDir
. Is it expected behavior? Is there a way to make nextflow
check if copying to storeDir
went ok?
storeDir
storeDir
?
errorStrategy = 'finish'
bam
files go to the same storeDir
publishDir
in place of storeDir
whenever possible
errorStrategy = { task.attempt>2 ? 'finish' : 'retry' }
?
publishDir
behavior, and see if we could rewrite the pipeline in accordance with the best practices ...
storeDir
only when it's needed a cache across different execution of the pipeline
if i'd want to change some parameters and relaunch?
storeDir
-resume
mechanism
publishDir
to see the difference
-resume
check the contents of the work
directory?
storeDir
?
-resume
check things like .exitcode
?
storeDir
in the pipeline to begin with ...
distiller
When I run this:
./nextflow cloud create optitype-cluster -c 4
I get the following error:
> cluster name: optitype-cluster
> instances count: 4
> Launch configuration:
- driver: 'aws'
- imageId: 'ami-cc2d64da'
- instanceType: 'c4.xlarge'
- keyName: 'xxxxxxxxx'
- securityGroup: 'xxxxxxxxx'
- subnetId: 'xxxxxxxxx'
- userName: 'ubuntu'
Please confirm you really want to launch the cluster with above configuration [y/n] y
Fetching EC2 prices (it can take a few seconds depending your internet connection) ..
ERROR ~ Cannot cast object 'null' with class 'null' to class 'int'. Try 'java.lang.Integer' instead
Any thoughts on why this might be happening? This was working fine for me a couple weeks ago. Any help would be much appreciated!
@pditommaso Doing the following worked.
export NXF_VER=0.25.3-SNAPSHOT
Thanks!
Running this command on the master node
./nextflow run nmdp-bioinformatics/flow-Optitype \
--with-docker nmdpbioinformatics/flow-OptiType \
--outfile hli-optitype.csv \
--bamdir s3://bucket/s3/data \
--datatype dna
Returns the following errors:
N E X T F L O W ~ version 0.25.3-SNAPSHOT
Pulling nmdp-bioinformatics/flow-Optitype ...
downloaded from https://github.com/nmdp-bioinformatics/flow-OptiType.git
Launching `nmdp-bioinformatics/flow-Optitype` [sleepy_jang] - revision: 6fcb330fe1 [master]
---------------------------------------------------------------
NEXTFLOW OPTITYPE
---------------------------------------------------------------
Input BAM folder (--bamdir) : s3://bucket/s3/data
Sequence data type (--datatype) : dna
Output file name (--outfile) : hli-optitype.csv
[warm up] executor > ignite
ERROR ~ ip-xxxxx-xx: ip-1xxxx-xx-xx: Name or service not known
-- Check script 'main.nf' at line: 68 or see '.nextflow.log' file for more details
What's the best way to share log data? Thanks, Mike.