These are chat archives for nextflow-io/nextflow

26th
May 2016
Mike Smoot
@mes5k
May 26 2016 21:48
I've got two channels that I'd like to combine. I've been using the mix operator for this, but the problem is that one of my channels may be empty. When this happens, nextflow hangs waiting for something to appear on the channel. Any ideas on how to combine two channels when one might be empty?
Paolo Di Tommaso
@pditommaso
May 26 2016 21:52
that should not happen
why you end up with an empty channel ?
channel factory always implicitly close the channel
Mike Smoot
@mes5k
May 26 2016 22:08
It begins by using the choice operator to branch into two separate channels based on the presence or absences of data. If all of the data is missing, then the "absence" channel will be full, but the "presence" channel will be empty. The opposite can happen, or some combination.
Paolo Di Tommaso
@pditommaso
May 26 2016 22:16
Um, looks weird. the choice operator closes all the channel when it has terminated
Mike Smoot
@mes5k
May 26 2016 22:16

Here's what I'm doing

sa_use_existing = Channel.create()
sa_reassemble   = Channel.create()
sample__assembly.choice( sa_use_existing, sa_reassemble ) { it -> it[1] == "none" ? 1 : 0 }

where sample__assembly is a tuple of id and filename

Paolo Di Tommaso
@pditommaso
May 26 2016 22:19
it looks fine
how is created sample__assembly ?
Mike Smoot
@mes5k
May 26 2016 22:19
Yeah, looks like the test. Here's an example that demonstrates what I'm seeing
ch1 = Channel.from(1, 2, 3)
ch2 = Channel.create()

ch1.mix(ch2).println()
Paolo Di Tommaso
@pditommaso
May 26 2016 22:20
this is the problem
ch1.mix(ch2) is meaning less
because ch2 emits nothing
what is meant for ch2 ?
Mike Smoot
@mes5k
May 26 2016 22:23

Maybe that was a bad example. Here's what I'm doing in my code:

sample__reassembly =  Channel.create().mix( new_assemblies, existing_assemblies )

where new_assemblies and existing_assemblies are channels that eventually lead back to sa_use_existing and sa_reassemble

Paolo Di Tommaso
@pditommaso
May 26 2016 22:24
the other way around, but the same issue Channel.create().mix( .. )
you should do new_assemblies.mix( existing_assemblies )
Mike Smoot
@mes5k
May 26 2016 22:25
And what if one or the other is empty?
Paolo Di Tommaso
@pditommaso
May 26 2016 22:26
you need to thing the a channel needs to emits a special item that is the stop signal
every channel need at some point to produce it otherwise the network i.e. the pipeline will hang
the channel create by Channel.create() never emits that by definition
so the mix hangs because it waits a for a stop signal for each of the input channels
Mike Smoot
@mes5k
May 26 2016 22:29
That makes sense. Can I use isEmpty(Channel.STOP) to get around this?
I guess I'm still not sure how to handle my case where it's valid for a channel to be empty.
Paolo Di Tommaso
@pditommaso
May 26 2016 22:30
I think you need
sample__reassembly = new_assemblies.mix( existing_assemblies )
makes sense?
Mike Smoot
@mes5k
May 26 2016 22:31
Isn't that just like the ch1/ch2 example above if existing_assemblies is empty?
Paolo Di Tommaso
@pditommaso
May 26 2016 22:34
no, unless you have existing_assemblies = Channel.create()
let put in this way, all operators implicitly close the channel but Channel.create()
Mike Smoot
@mes5k
May 26 2016 22:36
Ok, that makes sense.
Mike Smoot
@mes5k
May 26 2016 22:46
And I've got it working! Thanks @pditommaso
Paolo Di Tommaso
@pditommaso
May 26 2016 22:46
good!
none of you is coming to Codefest or BOSC 2016?
Jason Byars
@jbyars
May 26 2016 22:49
when is Codefest?
Paolo Di Tommaso
@pditommaso
May 26 2016 22:49
6-7 July
Jason Byars
@jbyars
May 26 2016 22:50
thanks.
Michael L Heuer
@heuermh
May 26 2016 22:56
<plug> Please come to the Codefest, if you can. Two days of all the right people, all in the same room, with laptops & wifi & beer.
Paolo Di Tommaso
@pditommaso
May 26 2016 22:56
I should be there at 80%
Mike Smoot
@mes5k
May 26 2016 23:06
Unfortunately can't make it this year.
Jason Byars
@jbyars
May 26 2016 23:08
I doubt I can make it, but I will inquire