These are chat archives for nextflow-io/nextflow

2nd
Jul 2018
Tobias Neumann
@t-neumann
Jul 02 2018 08:46
Is there a way to UNSET memory constraints for a given profile? like memory = '' ?
Paolo Di Tommaso
@pditommaso
Jul 02 2018 08:46
memory = null should do the trick
Tobias Neumann
@t-neumann
Jul 02 2018 08:48
indeed it does. thanks once again @pditommaso
Paolo Di Tommaso
@pditommaso
Jul 02 2018 08:48
:+1:
vkaimal
@vkaimal
Jul 02 2018 12:36
image.png
image.png
@fstrozzi Thanks for the tip! This is what I was originally trying to accomplish. Sorry about the delayed response -- I've been trying to make that work for the last two days, but with no success. :-( Above is what my job definition looks like.
My docker_scratch is 1000GB
tbugfinder
@tbugfinder
Jul 02 2018 14:34
@vkaimal which Ami are you using?
What's the source for docker_scratch?
I have a shared EFS source which is mounted within a custom AMI built using packer.
tbugfinder
@tbugfinder
Jul 02 2018 14:45
Could you login to the instance and container and check the setup?
Dave Istanto
@DaveIstanto
Jul 02 2018 16:33

@pditommaso sorry, late reply, I think I was not very clear with my question, what I meant was that when I run this simple script:


#!/usr/bin/env nextflow

number = 123

process tryingScript {

        output:
        val a into aChannel

        exec:
        println number
        a = 90
}    

aChannel.subscribe{println it}

nextflow successfully prints '90', but when I place the exec commands into separate script:


#!/usr/bin/env nextflow

println ${number}
a = 90

(the script above is called "nfTemplate.nf" and located in a proper template folder)
and call it as a template in the process :


#!/usr/bin/env nextflow

number = 123

process tryingScript {
        output:
        val a into aChannel

        script:
        template 'nfTemplate.nf'
}

aChannel.subscribe{println it}

nextflow outputs this line:
Missing value declared as output parameter: a

I was wondering if this would be possible to be done in nextflow

vkaimal
@vkaimal
Jul 02 2018 16:57
@tbugfinder I'm using an ECS optimized AMI (ami-5253c32d) to start with, where I install AWSCLI, mount docker_scratch and create my own custom AMI for my compute environment. docker_scratch is a 1000GB EBS volume.
tbugfinder
@tbugfinder
Jul 02 2018 17:18
@vkaimal
@vkaimal permissions are correct?
tbugfinder
@tbugfinder
Jul 02 2018 19:11
@vkaimal ist it setup within a job definition used by NF?
Paolo Di Tommaso
@pditommaso
Jul 02 2018 19:39
@ShawnConecone not sure to understand what you are trying to do, in the last snipped a is not defined, therefore you cannot output that value
vkaimal
@vkaimal
Jul 02 2018 19:40
@tbugfinder Yes. Appears that my latest attempt at creating the AMI has worked. Now some of the jobs complete fine, but others fail with OutOfMemoryError: Container killed due to memory usage. Fix one problem, another pops up!
tbugfinder
@tbugfinder
Jul 02 2018 19:43
@vkaimal how much memory did you assign within the job definition?
You might increase that.
vkaimal
@vkaimal
Jul 02 2018 23:49
@tbugfinder It's strange - the job definition has vCPUs= 8 and Memory = 32768 MiB.