These are chat archives for nextflow-io/nextflow

19th
Oct 2017
Simone Baffelli
@baffelli
Oct 19 2017 07:28
What is up with that c*ap article on nextflow you commented on twitter?
Paolo Di Tommaso
@pditommaso
Oct 19 2017 07:28
I'm abstaining for further comments for now
Simone Baffelli
@baffelli
Oct 19 2017 07:29
I see
Lukas Jelonek
@lukasjelonek
Oct 19 2017 07:39

Hey, I've seen that the drmaa support will become deprecated soon. I am still using it, as on our cluster or configuration the sge executor had problems in the past, whereas the drmaa executor did not have them. I checked it again today and the sge variant still does not finish. I looked into the work dir of one job and found the following log:

    Warning: no access to tty (Bad file descriptor).
    Thus no job control in this shell.
    set: Variable name must begin with a letter.

Can you guess whether this is a problem with the sge executor or with our installation of the gridengine?

Paolo Di Tommaso
@pditommaso
Oct 19 2017 07:41
I see, happy to know that somebody is using it
the above message is reported when using the sge executor ?
Lukas Jelonek
@lukasjelonek
Oct 19 2017 07:42
yes
Paolo Di Tommaso
@pditommaso
Oct 19 2017 07:43
it could be that the SHELL variable is not defined in your env
Lukas Jelonek
@lukasjelonek
Oct 19 2017 07:43
how can I check that? Starting jobs from the commandline with qsub works fine
Paolo Di Tommaso
@pditommaso
Oct 19 2017 07:44
let me check one thing
Lukas Jelonek
@lukasjelonek
Oct 19 2017 07:46
okay
Paolo Di Tommaso
@pditommaso
Oct 19 2017 07:46
what if you try to execute one of the failed job as shown below
qsub -v SHELL=/bin/bash .command.run
Lukas Jelonek
@lukasjelonek
Oct 19 2017 07:49
still the same problem
Paolo Di Tommaso
@pditommaso
Oct 19 2017 07:49
I see, you need to investigate with your sysadmins what's the cause
Lukas Jelonek
@lukasjelonek
Oct 19 2017 07:50
okay, then I will do that
Paolo Di Tommaso
@pditommaso
Oct 19 2017 07:50
I will suggest to focus on a single task as shown above
Lukas Jelonek
@lukasjelonek
Oct 19 2017 07:51
hopefully we will find a solution, otherwise I would be pleased to have drmaa support for a while
thanks for the quick help
Lukas Jelonek
@lukasjelonek
Oct 19 2017 09:07
So, I found the problem. On our system the commands are executed with /bin/csh by default. I fixed it with qsub -S /bin/bash .command.run. This starts the job with a bash. Can this be included into nextflow? On our side it will take some tedious discussions with our administrators to change the default execution shell on our cluster.
Paolo Di Tommaso
@pditommaso
Oct 19 2017 09:10
I think if you add that in the config file as process.clusterOptions = '-S /bin/bash' it should work
Lukas Jelonek
@lukasjelonek
Oct 19 2017 09:10
okay, I will try that
Now it works :+1: So I won't need drmaa anymore
Paolo Di Tommaso
@pditommaso
Oct 19 2017 09:14
nice
Lukas Jelonek
@lukasjelonek
Oct 19 2017 09:16
Thanks for your help. I should have asked a lot earlier
Paolo Di Tommaso
@pditommaso
Oct 19 2017 09:16
welcome
Alexander Peltzer
@apeltzer
Oct 19 2017 14:25
Trying to submit something to our PBS cluster right now, unfortunately the modules are not loaded correctly... and if I check the '.command.env' I have this there nxf_module_load qbic singularity 2.4
(with missing / signs...)
Félix C. Morency
@fmorency
Oct 19 2017 14:36
@pditommaso I don't know if you got an email about it, but I updated my PR
Paolo Di Tommaso
@pditommaso
Oct 19 2017 16:17
@apeltzer how is your module definition ?
@fmorency yes, thanks
Alexander Peltzer
@apeltzer
Oct 19 2017 17:29
@pditommaso so typically I load it that way on our cluster: modules load qbic/singularity/2.4
Paolo Di Tommaso
@pditommaso
Oct 19 2017 17:30
how have you specified it in the NF config ?
Alexander Peltzer
@apeltzer
Oct 19 2017 17:30
@pditommaso and I have this in my cluster.config process { executor = 'pbs' module = 'qbic/singularity/2.4' container = '/share/test/testcontainer.simg' cpus = 8 memory = 32.GB time = 24.h queue = 'cfc' }
NF is most current version
Paolo Di Tommaso
@pditommaso
Oct 19 2017 17:33
the point is that NF expects a module name scheme as name/version
can you open please an issue for that, we continue the discussion there
Alexander Peltzer
@apeltzer
Oct 19 2017 17:45
Ok
Here you go and thanks already nextflow-io/nextflow#489
Paolo Di Tommaso
@pditommaso
Oct 19 2017 17:49
thanks, I will ask for some info tomorrow
Alexander Peltzer
@apeltzer
Oct 19 2017 17:50
Okay!
Venkat Malladi
@vsmalladi
Oct 19 2017 22:14
I have a nextflow.config file, but the .command.env file is not being generated
anyone else have this issue?
Bo
@wenbostar
Oct 19 2017 23:08

Does anyone know how to solve this problem? N E X T F L O W ~ version 0.25.7
Launching pg.nf [naughty_franklin] - revision: 79c10d5da1
[warm up] executor > local
[68/8901e5] Submitted process > doMapping (1)
[1e/77f072] Submitted process > doMapping (2)
ERROR ~ Error executing process > 'doMapping (2)'

Caused by:
Process doMapping (2) terminated with an error exit status (1)

Command executed:

bwa mem -t 10 -M genome 32_DN.R1.fq.gz 32_DN.R2.fq.gz

Command exit status:
1

Command output:
(empty)

Command error:
/bin/bash: .command.sh: Permission denied

Venkat Malladi
@vsmalladi
Oct 19 2017 23:30
where are you running your workflow?
Bo
@wenbostar
Oct 19 2017 23:39
@vsmalladi , I ran my workflow in an AWS server.