These are chat archives for nextflow-io/nextflow
@tbugfinder you're correct that containers running on an instance will cut down on relaunch of process, but if your
process creates any intermediate files they are left on the image which in the case of WGS/WES can be quite large, and given how the aws-batch agent only clean up non-referenced task-definitions, in my case they are all remaining. @tbugfinder I could handle it as you have but I try not to allocate extremely large EBS volumes so all
process have enough space given that often I switch processing between WES and WGS.
Another question for both of you: How do you include your EBS volumes. I currently just create a large, high IOPS
/root drive as opposed to creating an additional EBS volume with a defined task definition to mount.