These are chat archives for nextflow-io/nextflow

22nd
Dec 2016
Mike Smoot
@mes5k
Dec 22 2016 21:14

Hi Paolo @pditommaso, I'm getting the following output on after submitting a job to SGE (created with cfncluster)

/var/spool/sge/ip-172-19-0-217/job_scripts/10: line 25: syntax error near unexpected token `<'

I see this output in the .command.log file and I'm assuming it's generated by the qsub of .command.run. It looks like the worker node is using bash version 4.2. Have you seen anything like this?

Paolo Di Tommaso
@pditommaso
Dec 22 2016 21:16
Um, make sure jobs use BASH as default shell
I think it even exist an option for that on the qsub CLI
Mike Smoot
@mes5k
Dec 22 2016 21:18
Yeah, I think that's it. It looks like SGE is defaulting to /bin/sh I'll see what I can do to force that to become bash.
Mike Smoot
@mes5k
Dec 22 2016 21:42
Looks like it's working with -process.clusterOptions="-S /bin/bash"!
Paolo Di Tommaso
@pditommaso
Dec 22 2016 21:45
yes, but it I'm not wrong there should be a way to define bash as the default shell in the SGE config file
Brian Naughton
@hgbrian
Dec 22 2016 21:45
Hi! I am exploring nextflow as a way to push jobs to the cloud, hopefully without too much management (kind of like domino datalab, preferably). I am trying to test out nextflow + kubernetes on GCE (not working yet!) and nextflow cloud on AWS. I followed the nextflow cloud instructions pretty carefully, but when I run the (provided) ssh command it cannot connect. On the AWS console when I try to connect it says that there is no key pair associated with the instance. I have tried the default username/rsa key and a new one matching the IAM username. Any ideas? (Apologies if this is the wrong place for these kinds of questions.)
Paolo Di Tommaso
@pditommaso
Dec 22 2016 21:45
@mes5k You may also set the var
SHELL=/bin/bash
hi, are u using AWS provided key pairs ?
Mike Smoot
@mes5k
Dec 22 2016 21:48
I believe you're right Paolo - I just need to find that file. :) cfncluster does not make this obvious.
Brian Naughton
@hgbrian
Dec 22 2016 21:48
Hi Paolo, ah.... That is probably my mistake. Thanks! I'll look into that.
Paolo Di Tommaso
@pditommaso
Dec 22 2016 21:48
SGE config is a mess ..
@hgbrian well, the easier way is just not config user nor keypair
NF will just use your public ssh key and user name
Brian Naughton
@hgbrian
Dec 22 2016 21:53
@pditommaso hmm, i tried that and it didn't work for me (just imageId, instanceType, subnetId under cloud {})
Paolo Di Tommaso
@pditommaso
Dec 22 2016 21:53
weird, what base image are u using ?
Brian Naughton
@hgbrian
Dec 22 2016 21:54
So under Launch Configuration it says: - keyFile: /Users/briann/.ssh/id_rsa.pub
  • userName: 'briann'
Paolo Di Tommaso
@pditommaso
Dec 22 2016 21:55
yep
Brian Naughton
@hgbrian
Dec 22 2016 21:55
ami-43f49030 --- i tried to match the docs exactly
Paolo Di Tommaso
@pditommaso
Dec 22 2016 21:55
ok, our image
what problem are u getting ?
Brian Naughton
@hgbrian
Dec 22 2016 21:56
In case it matters, on the console under subnet I had to "Modify auto-assign IP settings" to turn on publicly accessible ips (previously it returned xxx.internal IPs and also did not work)
"Operation timed out"
Paolo Di Tommaso
@pditommaso
Dec 22 2016 21:57
um, make sure the ssh port open in your security group
Brian Naughton
@hgbrian
Dec 22 2016 21:57
will do
Paolo Di Tommaso
@pditommaso
Dec 22 2016 21:58
this is a good guide
Brian Naughton
@hgbrian
Dec 22 2016 21:59
ok i'll spend some time there before coming back, thanks!
Paolo Di Tommaso
@pditommaso
Dec 22 2016 22:00
:ok_hand: