These are chat archives for nextflow-io/nextflow

28th
Jun 2018
Pierre Lindenbaum
@lindenb
Jun 28 2018 09:40

I think that could a nice addition to the nextflow DSL providing an helper method or something like that.

I will, thanks !

Pierre Lindenbaum
@lindenb
Jun 28 2018 09:49
@pditommaso done: nextflow-io/nextflow#774 , thanks :-)
micans
@micans
Jun 28 2018 12:34
This is probably not in keeping with the Nextflow philosophy, but I have a project where I want to redo the last stage with updated software (featureCounts). It would be great if it were possible to specify nextflow clean -p process_name, right now I see no other way than to redo the whole run.
Or maybe it is in keeping with the Nextflow philosophy!
Edgar
@edgano
Jun 28 2018 12:42
@micans if you only change the last stage, but the others are equal, when you run with the -resume it will cached the first process and it will run just the last part
An easy question, there is any "dummy" pipeline using bioconda? like rnatoy or similar??
thanks in advance
micans
@micans
Jun 28 2018 12:44
@edgano do you mean I change the process definition in the Nextflow file? In my scenario I would just have an updated external application. Nextflow does not have the necessary information to infer anything from that. Not sure I understand your comment!
Maxime Garcia
@MaxUlysse
Jun 28 2018 12:45
@edgano you can have a look at nf-core pipelines
We use conda environment to generate our docker container that we finally pull with singularity
So you can basically use pipelines with conda, docker or singularity
Edgar
@edgano
Jun 28 2018 12:47
thanks @MaxUlysse I knew nf_core :P I will check for a "easy" pipeline.
Alexander Peltzer
@apeltzer
Jun 28 2018 12:49
@edgano there are easy ones in there ;-)
e.g. the smaller ones in development, EAGER2, chipseq, methylseq
Edgar
@edgano
Jun 28 2018 12:50
@micans I thought you want to redo last stage with updated software ... you mean with a new container version...
No idea how to manage the external application issue... sorry
Thanks @apeltzer :smile:
micans
@micans
Jun 28 2018 12:51
No I don't mean new container version; I do mean updated software application (e.g. subread package). I don't understand what you propose I do ... what is it?
Edgar
@edgano
Jun 28 2018 12:53
If you have diff container for each process, when you modify the last stage's container. NF will redo that process. the first steps will be cached.
But it is not your case... forget it hahaha
micans
@micans
Jun 28 2018 12:53
Ah OK I get it. Cheers :smile:
@pditommaso is nextflow clean -p process_name a feasible feature request, or do the complexities of the process DAG prevent this?
or x, where x is something else entirely
Venkat Malladi
@vsmalladi
Jun 28 2018 14:48
@pditommaso is -no-requeue hardcoded in the slurm excutor and if so why?
Paolo Di Tommaso
@pditommaso
Jun 28 2018 14:58
to avoid slurm reschedule the execution of a failed job, it has to be managed by NF
@micans you can do nextflow log <run name> -F 'process =~ /foo.*/' | xargs rm -rf
a clean option would be nice, feel free to propose feature request and/or a pull request for that
micans
@micans
Jun 28 2018 15:01
That's pretty cool, thanks! This time I may feel sufficiently confident to propose a feature request unlike last time. Pull request .. I hope to get there.
Paolo Di Tommaso
@pditommaso
Jun 28 2018 15:02
:+1:
Venkat Malladi
@vsmalladi
Jun 28 2018 15:11
@pditommaso can we remove this option in our local copy, would we cause other issues or should we be fine?
Paolo Di Tommaso
@pditommaso
Jun 28 2018 15:11
our local copy
do you mean you maintain your own branch ?
Venkat Malladi
@vsmalladi
Jun 28 2018 15:13
yes
Paolo Di Tommaso
@pditommaso
Jun 28 2018 15:14
nextflow-io/nextflow#226
Venkat Malladi
@vsmalladi
Jun 28 2018 15:16
@pditommaso thanks
Paolo Di Tommaso
@pditommaso
Jun 28 2018 15:16
:+1: