These are chat archives for nextflow-io/nextflow

11th
Sep 2017
spaceturtle
@spaceturtle
Sep 11 2017 07:56
Hi, Could you explain a little how "scratch = true" works? I set it in my config file, but all output files still appear in "work" directory of my central storage.
Paolo Di Tommaso
@pditommaso
Sep 11 2017 08:20
yes, when setting scratch = true the process is executed in the node local storage, then output files are copied in the shared pipeline work dir, to make those files available to downstream tasks
Paolo Di Tommaso
@pditommaso
Sep 11 2017 08:27
@anfederico nope #264
Rickard Hammarén
@Hammarn
Sep 11 2017 08:31
thanks @pditommaso it was a missing . in the time request in the config. 10h instead of the required 10.h
Paolo Di Tommaso
@pditommaso
Sep 11 2017 08:31
oops
spaceturtle
@spaceturtle
Sep 11 2017 09:02
@pditommaso Thanks for your prompt response. Then how can I check if the process really generate the file on local storage? We are very sensitive to the performance of central storage, so we want to assure that .
Paolo Di Tommaso
@pditommaso
Sep 11 2017 09:24
you can change to a task work directory and run bash -x .command.run
it will dump the execution and you will see that it uses a local temp directory the outputs
Simone Baffelli
@baffelli
Sep 11 2017 12:37
Hello. I don't understand what I'm doing wrong, but somehow task.workDir returuns "null" as a path.
Has anybody experience something similar?
Ok, it works with exec but not with script
Paolo Di Tommaso
@pditommaso
Sep 11 2017 13:03
it only works for exec statement
something we need to improve #378
Marcel Martin
@marcelm
Sep 11 2017 16:25
Hi, is it possible to publish only some output files of a process when using publishDir? I have a process that outputs a huge BAM and a small report file. I don’t want to publish the BAM.
Phil Ewels
@ewels
Sep 11 2017 18:00
Yes :)
Marcel Martin
@marcelm
Sep 11 2017 18:14
Thanks! It’s not documented, so I hope this is not undefined behavior.
Phil Ewels
@ewels
Sep 11 2017 18:19
@pditommaso suggested this method to us I think..? It was a while ago now
It's in my slides for Thursday as a suggested "neat trick" already ;)