These are chat archives for nextflow-io/nextflow

9th
Feb 2019
Anthony Ferrari
@af8
Feb 09 09:17

Dear all,

I have an output channel of this form set key, file(file_chunk) into out_ch.
Basically it emits entries like this :

 key_a, chunk_01
 key_a, chunk_02
 key_b, chunk_01
 key_a, chunk_03
 key_b, chunk_02
 key_b, chunk_03
 key_a, chunk_04
...

I want to gather the file chunks into a new channel big_file emitting [key, file(big_file)].

key_a, full_file_a
key_b, full_file_b
...

I need the key to merge later on with another channel which is the output of a parallel process.

My first idea was to use first groupTuple and/or collectFile operators but I am struggling to keep the key alongside the file.

I have a solution which would be to use groupTuple and then create a process to merge manually but actually I would love to do this directly with collectFile in one efficient line ;-)

Any ideas ?
Thanks!