These are chat archives for nextflow-io/nextflow

29th
Aug 2016
Mokok
@Mokok
Aug 29 2016 08:50
Hi @pditommaso , i was wondering.... if i launch the same job several times and erroS occur. Will the -resume option resume the last one that's been launch ? the last one that ended in error ? Or is it just a bad practice to launch the same job several time. (then it'd be useful to be able to know if there is an instance of NextFlow running, an what's the concerned job)
Big question for a Monday, sry ^^
Paolo Di Tommaso
@pditommaso
Aug 29 2016 08:51
each job it's unique as long as it receive a different input
thus, there's not first or last concept
but success or failed jobs
bare in mind that nextflow processes are (supposed to be) stateless
Mokok
@Mokok
Aug 29 2016 08:59
not sure we understood each other. by job i don't mean a process, i mean a complete worflow, a *.nf file ran with "nextflow run ..." cmd line.
was it ok ?
Paolo Di Tommaso
@pditommaso
Aug 29 2016 08:59
ok, let's call it a pipeline execution :)
Mokok
@Mokok
Aug 29 2016 09:00
^^ there are so many words
Paolo Di Tommaso
@pditommaso
Aug 29 2016 09:00
ok, launching the same pipeline multiple times it's perfectly fine
if you resume it, no tasks should be computed, thus why it is supposed to fail?
Mokok
@Mokok
Aug 29 2016 09:09
mh...since you resume a pipeline by adding -resume to the initial command line. For example, if i run the pipeline "A" 3 times ; I'll write the cmd line "nextflow run A" 3 times. But if the first and third run of A fail and that i want to resume both...how can i resume both ? (or the only one wanted)
Paolo Di Tommaso
@pditommaso
Aug 29 2016 09:09
are you running these in parallel or one after another ?
Mokok
@Mokok
Aug 29 2016 09:10
parallel
Paolo Di Tommaso
@pditommaso
Aug 29 2016 09:11
if you run them in parallel, you must launch them in different folders or specify a different -work-dir for each of them
Mokok
@Mokok
Aug 29 2016 09:12
mh ok, and then nextflow won't be lost as it has one 'A' pipeline in each -work-dir..thk
is there a way to know which pipeline is run by a nextflow thread ? his -work-dir ? (I'm thinking about how i could monitor nextflow executions)
Paolo Di Tommaso
@pditommaso
Aug 29 2016 09:16
what do you mean for thread ? POSIX process ID ?
Mokok
@Mokok
Aug 29 2016 09:18
if POSIX is the name of an Unix OS process yes
Johan Viklund
@viklund
Aug 29 2016 09:18
@Mokok are you running the pipeline on different inputs each time?
Paolo Di Tommaso
@pditommaso
Aug 29 2016 09:20
@Mokok if you launch nextflow with the -bg (background) flag it creates a .nextflow.id that contains the its process ID
Mokok
@Mokok
Aug 29 2016 09:21
@viklund not necessarily, the sources could be an output 'found' by the first process of the pipeline. then the command line wouldn't change whereas the result WOULD change.
@pditommaso with this process id i can check if it's still running. Anything more ?
Paolo Di Tommaso
@pditommaso
Aug 29 2016 09:22
yes
:)
(kidding)
Mokok
@Mokok
Aug 29 2016 09:31
np sometime i deserve it ^^
by anything more i meant more about nextflow. But indeed, after few research i found that i can get the executed cmd, and then some of the params that i'm interested in.
thank you
Paolo Di Tommaso
@pditommaso
Aug 29 2016 09:33
welcome, let me know if u have further doubts