These are chat archives for nextflow-io/nextflow

23rd
Jan 2019
Yasset Perez-Riverol
@ypriverol
Jan 23 13:17
Hi all if I use errorStrategy ‘retry’ in my process, how many times by defaut NF will try to recover it?
Paolo Di Tommaso
@pditommaso
Jan 23 13:19
1
use maxRetries to change it
Shellfishgene
@Shellfishgene
Jan 23 14:21
Hi, recently I often get the error Missing output file(s) '.command.log' expected by process 'align'. I have a suspicion this may be related to file system slowness on our cluster, or would there be a better nextflow-related cause for this?
Forgot to say, when I check the file is there as expected, with all other output files.
rfenouil
@rfenouil
Jan 23 15:12
Question about a possible feature:
Is it possible for a run to search hashes from another run and cache results from there ?
I have a big list of files and I happen to run the same pipeline in separate folders for different subsets of this list. I would save time (and disk space) if it could cache results (computed on individual files) from previous runs.
If that makes any sense ;)
Stephen Kelly
@stevekm
Jan 23 20:15
the results from a previous Nextflow run are automatically cached, in a sense, in your 'work' directory. This is part of how the -resume feature work, I think
I think it might be easier for you if you run the same pipeline from the same directory each time, and just change the subset of samples in your pipeline inputs.
You can specify the location of the 'work' directory for Nextflow to use, among other things, however I am not sure how that works in combination with -resume. For example in one of my pipelines, I have a unique 'work' dir for each time I run it, along with outputting all the Nextflow logs there. The code for this is https://github.com/NYU-Molecular-Pathology/lyz-nf/blob/master/Makefile#L40
So maybe you could try something like that in combination with -resume? Not sure if it will work but might be worth a shot.