These are chat archives for nextflow-io/nextflow

27th
Apr 2016
Szilveszter Juhos
@szilvajuhos
Apr 27 2016 08:51
Morning, we are going to use NF to process tumor/normal cancer samples at SciLifeLab.se, and we want to make it as modular as possible
for example to include new variant callers, or QC software
could anybody please show an example how to have modules in a way and avoid large monolithic workflows?
tried to go through documentation, likely I have missed something
Paolo Di Tommaso
@pditommaso
Apr 27 2016 08:54
Hi, for this kind of problem we usually use templates and profiles
basically you externalise the tools into external scripts that can be chosen at runtime using a config parameter
there's a simple example here
need to leave now, I will back in one hour
Szilveszter Juhos
@szilvajuhos
Apr 27 2016 08:59
cheers, will read through and try to understand
Paolo Di Tommaso
@pditommaso
Apr 27 2016 10:25
@szilvajuhos let me know if you have any doubt
Szilveszter Juhos
@szilvajuhos
Apr 27 2016 11:27
seems the other option is simply call nextflow from nextflow, am I right?
Paolo Di Tommaso
@pditommaso
Apr 27 2016 11:41
yes, but for your use case I would say that templates fit better
Evan Floden
@evanfloden
Apr 27 2016 16:24

I have a channel emitting sets containing val(id), file(file_location).

I wish to concatenate the contents of the files in file_location based on the id. So the contents of all files paired with id_01 are in 01.txt and all the contents of all the files paired with id_02 are in 01.txt.

I imagine I can solve this with a combination of groupTuple and collectFile ie:

someChannel.from( [01,'some.txt'], [01,'more.txt'], [02,'even.txt'], [02,'more.txt'] )

someChannel
    .groupTuple()
    .collectFile()
    .set {concatChannel }

The problem is that groupTuple returns key -> array where array is the list of files and I get the error:

ERROR ~ Not a valid file collector argument [nextflow.util.ArrayBag]:
Paolo Di Tommaso
@pditommaso
Apr 27 2016 16:28
You don't need the groupTuple
The collectFile does the groupping as well
Evan Floden
@evanfloden
Apr 27 2016 17:51
Cheers Paolo, I was way to far down the rabbit hole to see the light!
*too
Paolo Di Tommaso
@pditommaso
Apr 27 2016 19:08
The light!