These are chat archives for nextflow-io/nextflow

Nov 2018
Anthony Underwood
Nov 10 2018 10:44

AWS batch is working beautifully on my root account. 50 genomes assembled in parallel in around 30 minutes for about $0.50!!! Value for money.

However if I set up a sub account and go through exactly the same process, the queue is set up and ready to go but when I run nextflow it hangs at this step and no jobs are submitted

[warm up] executor > awsbatch

Tail of the log shows

Nov-10 10:31:07.715 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'awsbatch'
Nov-10 10:31:07.715 [main] DEBUG nextflow.executor.Executor - Initializing executor: awsbatch
Nov-10 10:31:07.717 [main] DEBUG nextflow.Session - >>> barrier register (process: overall_quality)
Nov-10 10:31:07.717 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > overall_quality -- maxForks: 2
Nov-10 10:31:07.771 [main] DEBUG nextflow.script.ScriptRunner - > Await termination
Nov-10 10:31:07.771 [main] DEBUG nextflow.Session - Session await

What is the Session waiting for? It appears it's not even submitting to AWS batch since if I change the queue to a garbled name it still hangs at the same stage. On my working config on the root account a garbled queue name results in an error.

I'm sure it's a permission thing even though it's the same as the root account but if I knew what the session was waiting for that would help debugging

Anthony Underwood
Nov 10 2018 15:52
I've done my best to understand the underlying code but my knowledge of Groovy lets me down :(