These are chat archives for nextflow-io/nextflow

30th
Apr 2018
Rohan Shah
@rohanshah
Apr 30 2018 18:19
is it possible to use k8s and batch together in a workflow? when I do I get an error in my k8s processes like /bin/bash: .command.run: No such file or directory. I'm unsure if this is an issue with using an S3 working directory (required by the batch executor) or if this has to do with something that changes internally when awsbatch is specified as one of the executors for a process
Paolo Di Tommaso
@pditommaso
Apr 30 2018 18:20
uh, no
they use very different storage systems
Rohan Shah
@rohanshah
Apr 30 2018 18:21
ah ok, i am able to do local + k8s execution in the same workflow though
Paolo Di Tommaso
@pditommaso
Apr 30 2018 18:23
yes, because k8s exposes a plain posix file system
instead Batch requires a S3 bucket
Rohan Shah
@rohanshah
Apr 30 2018 18:24
ah i see that makes sense, thanks for explanation
Paolo Di Tommaso
@pditommaso
Apr 30 2018 18:24
welcome
Rohan Shah
@rohanshah
Apr 30 2018 18:30
i suppose this would be another good reason for supporting ecs as an executor for light/quick processes since it presumably could also use S3 storage mechanism and work together with the Batch executor
tbugfinder
@tbugfinder
Apr 30 2018 18:56
Which advantages do you see using ECS compared to AWS Batch?