These are chat archives for nextflow-io/nextflow

29th
Mar 2018
Paolo Di Tommaso
@pditommaso
Mar 29 2018 07:24
@BEFH just to note that each is the simplest way to combine multiple values as shown below:
process run_bcf {

    input:
    each coh from (['a','b','c'])
    each chr from  (1..22)  
    each other from (['a','b','c'])

    output:
    file("${coh}_${chr}_${other}.vcf.gz") into outch

    script:
    """
    touch ${coh}_${chr}_${other}.vcf.gz
    """
}
Grafitos
@grafitos
Mar 29 2018 11:34
Hi, when executor are not local, where java code are executed when process exec is java code ? Local or using the "executor" ?
Paolo Di Tommaso
@pditommaso
Mar 29 2018 12:09
@grafitos java code is always executed locally, except when using exec block with the Ignite executor
in that case java task is executed in a distributed manner
Grafitos
@grafitos
Mar 29 2018 15:05
Thank you Paolo
Paolo Di Tommaso
@pditommaso
Mar 29 2018 15:06
:+1:
Shawn Rynearson
@srynobio
Mar 29 2018 17:53

If I have a tuple like the following: [sample_name, [file.1, file.2, file.3, ...n]] which is a collection of file associated to a single sample name., and I want to run a process per file, but not uncouple the sample name from the list of files, what is the best method?

I can't use each as it's not valid with set, and because they are coming through the same channel I can't write the input as two seperate lines.

Shawn Rynearson
@srynobio
Mar 29 2018 17:55
So that would change [sample_name, [file.1, file.2, file.3, ...n]] to something like [sample_name, file.1, file.2, file.3, ...n]
Paolo Di Tommaso
@pditommaso
Mar 29 2018 17:56
nope
input: set sample, file(x) from foo.transpose()
Shawn Rynearson
@srynobio
Mar 29 2018 17:57
but would this still run the same as using each?
so I could run the process for each file.
Paolo Di Tommaso
@pditommaso
Mar 29 2018 17:57
not understanding each
why do you want to use each ?
Shawn Rynearson
@srynobio
Mar 29 2018 17:58
Paolo Di Tommaso
@pditommaso
Mar 29 2018 17:58
I know that, but I'm not understanding in this context
Mike Smoot
@mes5k
Mar 29 2018 18:00
@srynobio each applies to normal groovy collections, not channels, so each wouldn't apply to your channel.
Paolo Di Tommaso
@pditommaso
Mar 29 2018 18:00
actually he is referring to NF each not the groovy one
Shawn Rynearson
@srynobio
Mar 29 2018 18:00

for example:

The sample_name would be a single individual.
and the files are a collections of one or more read groups file that bwa would need to run. And I need to keep the association to the original sample.

Mike Smoot
@mes5k
Mar 29 2018 18:01
right, but doesn't NF each only apply to groovy collections defined as variables somewhere in the script?
Paolo Di Tommaso
@pditommaso
Mar 29 2018 18:02
ah, I think if you apply to a channel, the channel is reduced to a list
but again, transpose is the way to go for this use case
Mike Smoot
@mes5k
Mar 29 2018 18:06
Yes, transpose is right here. I just find NF each very confusing. I find it much easier to reason about channels and manipulate the data in channels than to combine collections and channels and iterate and combine things in the process declaration. Different strokes for different folks, I suppose.
Shawn Rynearson
@srynobio
Mar 29 2018 18:07

So I was running something like:

process BWA {
    tag { sample_id }

    input:
    val sample_id from bwa_sample_info_ch
    each (readGroupFiles) from bwa_in_ch

    '"""
    bwa 
    """

so for each (readGroupFile) bwa was being ran, but sample_id confusion was being created. Will transpose work in the same fashion?

Shawn Rynearson
@srynobio
Mar 29 2018 18:32
looks like transpose will do the trick! Thanks @pditommaso & @mes5k