These are chat archives for nextflow-io/nextflow

Aug 2018
Shawn Rynearson
Aug 05 2018 18:37

@tbugfinder you're correct that containers running on an instance will cut down on relaunch of process, but if your process creates any intermediate files they are left on the image which in the case of WGS/WES can be quite large, and given how the aws-batch agent only clean up non-referenced task-definitions, in my case they are all remaining. @tbugfinder I could handle it as you have but I try not to allocate extremely large EBS volumes so all process have enough space given that often I switch processing between WES and WGS.

Another question for both of you: How do you include your EBS volumes. I currently just create a large, high IOPS /root drive as opposed to creating an additional EBS volume with a defined task definition to mount.

Paolo Di Tommaso
Aug 05 2018 18:40
I'm very confused by this problem. Could you please open an issue on GutHub providing all details
Shawn Rynearson
Aug 05 2018 22:00
sure thing @pditommaso