These are chat archives for nextflow-io/nextflow

18th
Apr 2018
Paolo Di Tommaso
@pditommaso
Apr 18 2018 05:16
@tbugfinder use a custom job definition, see point 5
tbugfinder
@tbugfinder
Apr 18 2018 08:31
@pditommaso I really had to read that multiple times but I was able to use it. Thx
tbugfinder
@tbugfinder
Apr 18 2018 15:20
workdir
tbugfinder
@tbugfinder
Apr 18 2018 17:04
Any idea how to create a single result zip file after processing all parallel pipelines?
Edgar
@edgano
Apr 18 2018 17:05
a bash script on the onComplete with the zip zip -r data *
Mike Smoot
@mes5k
Apr 18 2018 17:13
Why not make a final process downstream of all your other processes and do the zip there? That way you'd get retries and error handling that onComplete wouldn't.
Edgar
@edgano
Apr 18 2018 17:14
true ...
tbugfinder
@tbugfinder
Apr 18 2018 17:15
How to make a process the final one?
Mike Smoot
@mes5k
Apr 18 2018 17:26
Well, whichever processes are generating the data you want to zip, have those processes put the data into an output channel, say out_chan. Then you'll want to collect all of the data in the channel using the collect operator. Something like: out_chan.collect().set{ data_list }. The thing to understand about the collect operator is that consumes all of the data in the channel to produce a single list, so it can't run until the last process populating the channel has completed. Your final_zip process will then take the data_list channel as input and you'll run zip on the data from data_list.
jncvee
@jncvee
Apr 18 2018 19:15
is this a nextflow error or a system error ERROR ~ .nextflow/history.lock (No space left on device)
Mike Smoot
@mes5k
Apr 18 2018 19:24
looks like the disk where you're running nextflow has run out of space. df -h . in that dir should tell you what's going on.