These are chat archives for nextflow-io/nextflow

23rd
Mar 2016
Alexander Junge
@JungeAlexander
Mar 23 2016 16:45

Hi there, Just getting started with Nextflow and enjoying it very much so far. Great work! I am currently looking for a way to do implemented a nested for loop trying out all combinations of two parameter sets. For instance, this could look like this:

#!/usr/bin/env nextflow

a = [1, 2, 3]
b = [4, 5, 6]

process foo {
    input:
    val a
    val b

    """
    echo $a $b
    """
}

Where the foo process should echo 1 4, 1 5, 1 6, 2 4, ..., 3 6. Is there a convenient way build into Nextflow to implement this behavior?

Paolo Di Tommaso
@pditommaso
Mar 23 2016 17:05
There are two way, as long you need to iterate over a finite list of elements you can just use the each statement, for example:
a = [1, 2, 3]
b = [4, 5, 6]

process foo {
    input:
    each a
    each b

    """
    echo $a $b
    """
}
Alexander Junge
@JungeAlexander
Mar 23 2016 17:07
@pditommaso awesome! thank you very much for the swift reply and sorry that I overlooked this in the documentation.
Paolo Di Tommaso
@pditommaso
Mar 23 2016 17:08
Instead if you need to combine the values emitted by a channel you can use the spread operator
you are welcome
Support the project starring it on github if you like it ! :)
Alexander Junge
@JungeAlexander
Mar 23 2016 17:14
of course! just did :)
Paolo Di Tommaso
@pditommaso
Mar 23 2016 17:14
:clap:
Jason Byars
@jbyars
Mar 23 2016 18:28
Hi, debugging question of the morning. When you're debugging a pipeline, is there a simple way to specify process only the first N samples. For example if I have a Channel of 50 input files. Is there an quick way to say process only the first 3 and everything else in the workflow runs normally.
Paolo Di Tommaso
@pditommaso
Mar 23 2016 18:30
Quick ansewer no..
Boarding!
Jason Byars
@jbyars
Mar 23 2016 18:31
NP. I'll work something out. Thanks!
Michael L Heuer
@heuermh
Mar 23 2016 18:42
You could use the take operator (http://www.nextflow.io/docs/latest/operator.html#take) with an input parameter
Jason Byars
@jbyars
Mar 23 2016 18:43
Thank you, that's what I was missing.
Paolo Di Tommaso
@pditommaso
Mar 23 2016 20:39
Right, there's not an automatic feature but some work around can be easily implemented
The take operator with -1 as default is a good idea
Jason Byars
@jbyars
Mar 23 2016 20:41
I just wanted to do better than the classic printf debug or creating overly specific file name filters. This will work.
The other debugging mechanic I'm interested in would be --start-at-process similar to Ansible's --start-at-task. There is the resume after error mechanic, but often I find I want to back up a couple steps or skip a problem step. Unless, I'm misunderstanding the modify and resume mechanic
Paolo Di Tommaso
@pditommaso
Mar 23 2016 20:48
The resuming mechansim will alwasy try to reexecute the entire flow including all tasks, unless you provide some specific logic in the when statement
However there's no such start-from feature
Jason Byars
@jbyars
Mar 23 2016 20:51
Good to know. I'm not sure I would have interpreted the explanation from http://www.nextflow.io/docs/latest/getstarted.html?highlight=resume that way.
You might want to mention the when mechanic there.
Paolo Di Tommaso
@pditommaso
Mar 23 2016 21:44
@jbyars yes you are right, the when statement has been added recently
I should add an explanation about that
Jason Byars
@jbyars
Mar 23 2016 21:46
oh, I didn't realize it was a recent addition.