These are chat archives for nextflow-io/nextflow

12th
Dec 2017
Raoul J.P. Bonnal
@helios
Dec 12 2017 21:46
hi all, newbie here. How can I transfer file from a process to S3 ?
Paolo Di Tommaso
@pditommaso
Dec 12 2017 21:47
this is usually done with a publishDir directive and specifying a S3 path
Raoul J.P. Bonnal
@helios
Dec 12 2017 22:18
@pditommaso thanks ok worked, I tested it with small files. Is it even safe/reliable with big (>50GB) file ? Is it possible to get the average upload speed ?
Paolo Di Tommaso
@pditommaso
Dec 12 2017 22:22
big files are chunked and uploaded using S3 multipart upload protocol, so it should be fine
you can eventually tune the upload setting, see here
Is it possible to get the average upload speed ?
mm, no
Mike Smoot
@mes5k
Dec 12 2017 23:20
You hate clojure? Or love it? Or you're impressed by the scale of their operation?