These are chat archives for nextflow-io/nextflow
Channelwith the large files just hold the s3urls and the first step in the script section is to grab a copy of the files. Thank you for pointing out cadvisor.
scratchfolder automatically clean up on the workers at the end of the each job?
scratch = true, is there a way to get the scratch directory before the script section runs? My idea was to use
beforeScriptto fetch big files directly to worker scratch directories before running the script section. Unfortunately,
beforeScriptappears to run in the normal shared work directory, not the scratch dir. This is really only an issue because I'm using an executable container and need to fetch before the container runs.