These are chat archives for nextflow-io/nextflow

20th
Jan 2017
Fredrik Boulund
@boulund
Jan 20 2017 12:02
Thanks @pditommaso ! Just what I needed.
Trevor Tanner
@tantrev
Jan 20 2017 15:42
Is there any way to group jobs for submission to SLURM? I have to use STAR for aligning 1000s of files & since it uses so much memory (~30GB mem), it'd be nice to to have only load the genome-index once for a chunk of files.
"it'd be nice to have to only load"*
Paolo Di Tommaso
@pditommaso
Jan 20 2017 16:38
not by NF itself, you can try to use some specific SLURM option (if available)
Trevor Tanner
@tantrev
Jan 20 2017 16:41
I'll keep my fingers crossed for NF sub-workflow functionality :)
Paolo Di Tommaso
@pditommaso
Jan 20 2017 16:42
me too :)
Mike Smoot
@mes5k
Jan 20 2017 17:39
@pditommaso have you started designing sub-workflows?
Paolo Di Tommaso
@pditommaso
Jan 20 2017 17:39
not yet :/
Mike Smoot
@mes5k
Jan 20 2017 17:40
If you'd like any ideas I've given it a bit of thought...
Paolo Di Tommaso
@pditommaso
Jan 20 2017 17:42
yes, our course, though gitter is not a good place to discuss about that
Mike Smoot
@mes5k
Jan 20 2017 17:42
Sure. I'll write something up and send it to you, maybe later today.
Paolo Di Tommaso
@pditommaso
Jan 20 2017 17:43
you may want to comment here
nextflow-io/nextflow#238
Mike Smoot
@mes5k
Jan 20 2017 17:43
Ah yes, even better. Will do, thanks!
Paolo Di Tommaso
@pditommaso
Jan 20 2017 17:43
you are welcome
Trevor Tanner
@tantrev
Jan 20 2017 18:03
So this is probably a stupid question but I cannot figure how to get (certain) processes to execute in a sequential manner. I know it's possible - for example, this workflow https://github.com/ewels/NGI-RNAseq/blob/master/main.nf does it with its preprocessing .
Paolo Di Tommaso
@pditommaso
Jan 20 2017 18:04
processes are executed sequentially as long as there's a in/out dependency between them
Trevor Tanner
@tantrev
Jan 20 2017 18:23
gotcha. sorry, I guess what I'm really asking is when multiple input channels are being used, may only channels of size 1 or the same size be used? For example, when I use two input channels of different size (say one is of length 5, the other of length 2), the process only executes 2 times.
Paolo Di Tommaso
@pditommaso
Jan 20 2017 18:24
exactly
the shortest stop the execution
the logic is that at each run an item from each channel is picked
thus when there's an empty channel a process can't be executed any more
Trevor Tanner
@tantrev
Jan 20 2017 18:28
ah, that makes perfect sense. thank you, am slowly learning this dataflow thinking :smile:
Paolo Di Tommaso
@pditommaso
Jan 20 2017 18:29
that's a bit tricky, but once you got it, it's fun ;)