These are chat archives for nextflow-io/nextflow

30th
Aug 2016
Johan Viklund
@viklund
Aug 30 2016 08:59
is there anyway to use the same channel as input in several processes without using tap or into before?
I'm thinking something like
input: file 'input' from file_channel.copy()
and that way I can use that construct in several processes
Paolo Di Tommaso
@pditommaso
Aug 30 2016 09:02
the only alternative is the one described here
Johan Viklund
@viklund
Aug 30 2016 09:04
into seperately from the process definition then
Paolo Di Tommaso
@pditommaso
Aug 30 2016 09:04
yes
Johan Viklund
@viklund
Aug 30 2016 09:04
My workflow is getting a lt of temporary channels...
Paolo Di Tommaso
@pditommaso
Aug 30 2016 09:04
it's very big?
Johan Viklund
@viklund
Aug 30 2016 09:05
it's 3 different variant callers, and some pre and post processing
7 processes
(I don't know if that's big)
I also have the callers optional and a postprocessing step that is only run if 2 callers are run (that will have to change now that I have 3 instead of 2 callers)
maybe I'll just put it in a sh wrapper
Paolo Di Tommaso
@pditommaso
Aug 30 2016 09:07
I think the champion continues to remain the guy from Sanger :)
Johan Viklund
@viklund
Aug 30 2016 09:08
the ability to just say input: f from channel.copy several times would make the code a lot easier to read
Paolo Di Tommaso
@pditommaso
Aug 30 2016 09:08
I think the multiple into in the process def can help
Johan Viklund
@viklund
Aug 30 2016 09:08
sure, it just feels backwards to have the producer having to know about all consumers
Paolo Di Tommaso
@pditommaso
Aug 30 2016 09:09
unfortunately that's not possible for some tricky aspects of the dataflow library used by nextflow
Johan Viklund
@viklund
Aug 30 2016 09:10
yes, I can imagine that, it would require the channels to contain more logic somehow
which would make that part messy
Paolo Di Tommaso
@pditommaso
Aug 30 2016 09:11
the problem is that when you are copying it with a channel.copy() some content could be already consumed ..
Johan Viklund
@viklund
Aug 30 2016 09:11
well, nextflow warns about that
but I see your point
you woudn't be able to warn about it in that case
do you think that is in general better to put it in the output definition of processes or have the split copying in separate plumbing code
(the problem I have is that if the bamfile index already exists, I want to use that and not re-index)
Paolo Di Tommaso
@pditommaso
Aug 30 2016 09:14
(the problem I have is that if the bamfile index already exists, I want to use that and not re-index)
for this you can use storeDir
Johan Viklund
@viklund
Aug 30 2016 09:15
ahh
but what if it has been generated by someone else already?
should I just move/link the file into storeDir at the start of my workflow?
Paolo Di Tommaso
@pditommaso
Aug 30 2016 09:17
yes, storeDir can be any path
it will only checks that the files declared as output exists in that folder
Johan Viklund
@viklund
Aug 30 2016 09:26
ah, ok
Mokok
@Mokok
Aug 30 2016 09:33
(a sneaky reader has appreciated the reading)
Paolo Di Tommaso
@pditommaso
Aug 30 2016 09:35
:)