These are chat archives for nextflow-io/nextflow

22nd
Jun 2017
tonywang0613
@tonywang0613
Jun 22 2017 18:22
Hi, a general question, I thought one channel cannot use by two process, have to first duplicate it.
But I have download some reference file, into one channel, then using the same channel several times, it works
Can anyone tell me how it works?
Paolo Di Tommaso
@pditommaso
Jun 22 2017 18:23
Can you share a code example?
tonywang0613
@tonywang0613
Jun 22 2017 18:26
`process star {
tag {"star process " }
echo true
publishDir "/mnt/",mode:'copy'

input:
    set file(fastqfile1), file(fastqfile2) from readslist
    file(index) from genomeindex
    file(gtf) from stargtf
output:
    file("*genome.bam") into genomebam
    file("*transcriptome.bam") into transcriptomebam
    file("*bam") into star_up`
I generate genomebam here
then I used the genomebam twice
`
process markdup {
tag {"process markduplication"}
publishDir "/mnt",mode:'copy'
echo true
input:
    file(bam) from genomebam
and another process
It works
Paolo Di Tommaso
@pditommaso
Jun 22 2017 18:29
How is declared readslist
?
tonywang0613
@tonywang0613
Jun 22 2017 18:29
what you mean readslist?
Paolo Di Tommaso
@pditommaso
Jun 22 2017 18:30
In the star process you have from readslist
tonywang0613
@tonywang0613
Jun 22 2017 18:32
'

'process downsamples {
tag {"download sample: "+ input}

input:
    val(input) from inputfiles
    val(status) from READY

output:
    file '*' into samplefiles

script:
if (params.profile=='gmi-prod')
"""
aws s3 cp $input ./ --profile $profile --sse-kms-key-id  arn:aws:kms:eu-west-1:030819299652:key/ee74d7b7-5505-498b-88e9-3e76a643b1f4 --sse aws:kms --only-show-errors
"""
else
"""
aws s3 cp $input ./ --profile $profile --only-show-errors
"""

}

readslist=samplefiles.toList()
download into a channel, create a list out of the channel samplefiles
but the problem is not the readlist, it's the genomebam channel, that I use twice in later process
Paolo Di Tommaso
@pditommaso
Jun 22 2017 18:34
Ok, when you apply the toList operator you will get a channel emitting a single value is a singleton channel
When all inputs of a process are singleton channels it will executed only once, hence also all the process outputs are singleton channels
tonywang0613
@tonywang0613
Jun 22 2017 18:36
I also have code create genome.fa file channel, directly using that channel 3,4 times without duplicate it
Paolo Di Tommaso
@pditommaso
Jun 22 2017 18:36
And singleton channels can be used multiple times by different processes and operators
tonywang0613
@tonywang0613
Jun 22 2017 18:38
got it, singleton channel means that all values in that channel are generated by one single process
Paolo Di Tommaso
@pditommaso
Jun 22 2017 18:40
No, it means the channel is bound to a (single) vue you can read multiple times
*value
tonywang0613
@tonywang0613
Jun 22 2017 18:41
ok, in my case, I use star generate the genome bam file, pipe into a channel, that channel is singleton channel
I only process it once
if the inputs has several samples, the star process run several times, generate several bam file, then that channel is not singleton channel
Paolo Di Tommaso
@pditommaso
Jun 22 2017 18:45
Not sure, it looks the star will run once in any case as u definited the inputs
tonywang0613
@tonywang0613
Jun 22 2017 19:03
thanks very much
found this link
nextflow-io/nextflow#301
its exactly the answer I am looking for
Paolo Di Tommaso
@pditommaso
Jun 22 2017 19:04
:+1: