These are chat archives for nextflow-io/nextflow

4th
Sep 2017
Simone Baffelli
@baffelli
Sep 04 2017 12:46
Hello; is there any quick trick to use the each input repeater with HashMaps?
It does not work as is, it always passes the entire HashMap to the process
nevermind, entrySet solved it
Piotr Kaleta
@pkaleta
Sep 04 2017 15:10
Is there a way for two processes to consume the same input channel?
Basically I want to process the same input file using two callers. How do I do that in nextflow?
Luca Cozzuto
@lucacozzuto
Sep 04 2017 15:11
you can "store" from the same channel in two different objects
Simone Baffelli
@baffelli
Sep 04 2017 15:11
you can just pass it to several channel using .into{chan1;chan2}
Flow-based programming does not allow us to consume the same values multiple times, but you can copy them to a new channel
Piotr Kaleta
@pkaleta
Sep 04 2017 15:12
Nice, I'll try that. Thanks!
Simone Baffelli
@baffelli
Sep 04 2017 15:12

Otherwise, you could try something like:

process xyz{
input:
 file a from somechannel
 each caller from ['foo', 'bar', 'baz']
output:
 set file(b), val(caller) into processed
shell:
'''
!{caller} !{a} > b
'''
}

that would repeat the process for each caller in the list, assuming the callers are either available from the system or are placed in your bin directory and assuming you want to capture the processes standard output into a file called b. You may want to adapt it to your case, but I hope I was able to convey the general logic.

Luca Cozzuto
@lucacozzuto
Sep 04 2017 15:14
Channel
    .fromPath( params.reads )                                             
    .ifEmpty { error "Cannot find any reads matching: ${params.reads}" }
    .into { read_files_for_ribo;  read_files_for_size}
Piotr Kaleta
@pkaleta
Sep 04 2017 15:19
So is it true that .into() doesn't actually copy the files, but creates symlinks?

Also, assuming that I have output files produced by those callers in directories like

/flowcell1/sample1/caller1.vcf
/flowcell1/sample1/caller2.vcf
/flowcell1/sample2/caller1.vcf
/flowcell1/sample2/caller2.vcf
/flowcell2/sample1/caller1.vcf
/flowcell2/sample1/caller2.vcf
...

How do I create a "grouping" process where it processes caller1.vcf and caller2.vcf in a single invocation, to create a so called "consensus" output?

Simone Baffelli
@baffelli
Sep 04 2017 15:24
@pkaleta The first question: depends on the staging mode: https://www.nextflow.io/docs/latest/process.html#stageinmode.
Second question: why would you worry about the directories? If you follow the example given above, you could just process them separately and finally group them by sample and caller using groupBy
I should start a nextflow consultancy service :grin:
Luca Cozzuto
@lucacozzuto
Sep 04 2017 15:28
Dear all, is someone using nextfow with singularity and has problem with location of perl modules?
Paolo Di Tommaso
@pditommaso
Sep 04 2017 15:50
I'm not aware of any specific Perl problem
Luca Cozzuto
@lucacozzuto
Sep 04 2017 15:53
likely is a problem related to environment... let me investigate more :(
I should tattoo on myself "#! /usr/bin/env perl" (and not #! /usr/bin/perl")
Paolo Di Tommaso
@pditommaso
Sep 04 2017 15:58
:)
that's a good idea ;)
Simone Baffelli
@baffelli
Sep 04 2017 19:17
Or just stop using perl😀
Francesco Strozzi
@fstrozzi
Sep 04 2017 19:26
That's even better
😀
Paolo Di Tommaso
@pditommaso
Sep 04 2017 20:18
shhh, don't say that @cjfields would cry :)
Luca Cozzuto
@lucacozzuto
Sep 04 2017 20:31
Well, considering it! :+1:
Simone Baffelli
@baffelli
Sep 04 2017 21:26
Well, I don't use perl, on the other hand I sometime use akw :smile: not sure whats worse
Paolo Di Tommaso
@pditommaso
Sep 04 2017 21:27
the problem of perl are the libraries, at least awk doesn't have modules ;)
at least I hope ..:D
Simone Baffelli
@baffelli
Sep 04 2017 21:28
I'm not such an advanced user, I mostly use it to cobble together strings :smile:
The worst language I encountered so far is IDL :fearful:
Paolo Di Tommaso
@pditommaso
Sep 04 2017 21:30
Interface Def Lang ? it's not a lang .. :)
Simone Baffelli
@baffelli
Sep 04 2017 21:30
Nono, interactive data language
It used to be quite popular for satellite image analysts and astronomy
Paolo Di Tommaso
@pditommaso
Sep 04 2017 21:31
ummm, god saved me, at least for now ..
Simone Baffelli
@baffelli
Sep 04 2017 21:32
A matlab look-alike without all the syntactic sugar
Paolo Di Tommaso
@pditommaso
Sep 04 2017 21:32
*without*
Simone Baffelli
@baffelli
Sep 04 2017 21:33
Can't type today😀