These are chat archives for nextflow-io/nextflow

23rd
Nov 2015
Robert Syme
@robsyme
Nov 23 2015 13:20
To follow-up on the tput: unknown terminal error (and associated strangeness). It turns out that on our cluster, the TERMINFO variable is not set. Everything is fixed with a quick export TERMINFO=/usr/share/terminfo
Paolo Di Tommaso
@pditommaso
Nov 23 2015 13:21
ah
nice to know that
I should create a troubleshooting page
Robert Syme
@robsyme
Nov 23 2015 13:23
Actually, sorry. It just clears the tput messages. It's still looking for main.nf in $HOME/bin. I'll get to the bottom of this...
Paolo Di Tommaso
@pditommaso
Nov 23 2015 13:25
um, something related to the $PWD variable, maybe?
Robert Syme
@robsyme
Nov 23 2015 13:28
No, it looks ok (at least from interactive sessions).
Paolo Di Tommaso
@pditommaso
Nov 23 2015 13:29
I would suggest to run this small nextflow snippet
System.env.sort().each { k, v -> println "$k = $v" }
and verify that your env is fine
Robert Syme
@robsyme
Nov 23 2015 13:31
Looks ok:
...
PROFILEREAD = true
PWD = /home/rsyme/foobar
PYTHONSTARTUP = /etc/pythonstart
...
Paolo Di Tommaso
@pditommaso
Nov 23 2015 13:32
and I guess you are running it in the /home/rsyme/foobar path ..
Robert Syme
@robsyme
Nov 23 2015 13:33
Yup. Made a foobar dir to test. Note that this was run with nextflow run $PWD as nextflow run . looks for main.nf in ~/bin.
Don't worry about it too much, Paolo. I'll report back if I get to the bottom of it.
Paolo Di Tommaso
@pditommaso
Nov 23 2015 13:36
just one more try, could you save this to a file named mf.nf and run it?
file = new File('me.nf')
println "exist: ${file.exists()}"
ops there was a typo
Robert Syme
@robsyme
Nov 23 2015 13:40
rsyme@magnus-1:~/foobar2> nextflow run /home/rsyme/foobar2/mf.nf 
N E X T F L O W  ~  version 0.16.2
Launching /home/rsyme/foobar2/mf.nf
exist: true
Paolo Di Tommaso
@pditommaso
Nov 23 2015 13:41
that's fine, I'm not understanding why it is not finding the main script ..
Robert Syme
@robsyme
Nov 23 2015 13:43
I moved the nextflow executable to ~/nf-here and added it to $PATH to check if nextflow was looking for main.nf in the same folder as it, but it still looks in ~/bin. So strange. Anyway, I'll report back.
Paolo Di Tommaso
@pditommaso
Nov 23 2015 13:43
ok
at your convenience you could try to run nextflow -trace nextflow run <your project>and send me the log
have fun ;)
Robert Syme
@robsyme
Nov 23 2015 13:49
Fixed. I think the problem was that I had included in ~/.nextflow/config the line:
process.shell = ['/bin/bash', '--login']
Paolo Di Tommaso
@pditommaso
Nov 23 2015 13:50
!
the --login I suppose ..
Robert Syme
@robsyme
Nov 23 2015 13:51
Hmm... no, that's not it. I copied out the config and then rm -r ~/.nextflow, restored the config and everything works.
Something in ~/.nextflow (but not config).
Paolo Di Tommaso
@pditommaso
Nov 23 2015 13:51
ok, the important thing is that it's fine :)
Robert Syme
@robsyme
Nov 23 2015 13:51
...anyway. Everything seems to be working. I really have no idea how computers work ;)
Thanks Paolo, and sorry to have bothered you (again).
Paolo Di Tommaso
@pditommaso
Nov 23 2015 13:58
@robsyme you are very welcome
Robert Syme
@robsyme
Nov 23 2015 13:59

Aha. Getting closer:

rsyme@magnus-1:~> rm -rf ~/.nextflow/tmp/
rsyme@magnus-1:~> cd foobar1/
rsyme@magnus-1:~/foobar1> nextflow run .
N E X T F L O W  ~  version 0.16.2
Launching ./main.nf
exist: true
rsyme@magnus-1:~/foobar1> cd ../foobar2/
rsyme@magnus-1:~/foobar2> rm -r ../foobar1
rsyme@magnus-1:~/foobar2> nextflow run .
N E X T F L O W  ~  version 0.16.2
Launching ./main.nf
ERROR ~ Not such file: /home/rsyme/foobar1/main.nf

 -- Check '.nextflow.log' file for details
rsyme@magnus-1:~/foobar2>

It's looking for main.nf in the last good location

Paolo Di Tommaso
@pditommaso
Nov 23 2015 13:59
!
could you add -trace nextflow and send me the log?
Robert Syme
@robsyme
Nov 23 2015 14:00

In `/home/rsyme/.nextflow/tmp/launcher/nextflow-one_0.16.2/classpath-a77aacaa0375250dddee5656a26c91dc:

"-Duser.dir=/home/rsyme/foobar1"

Paolo Di Tommaso
@pditommaso
Nov 23 2015 14:00
ah
got it
This message was deleted
there should not be such property
ok, I will investigate why there's that "-Duser.dir=/home/rsyme/foobar1"
Robert Syme
@robsyme
Nov 23 2015 14:04
Thanks Paolo!
Paolo Di Tommaso
@pditommaso
Nov 23 2015 14:05
actually it's not produced by nextflow code, but by the Capsule package manager
one more thing: are you in contact with the Pawsey guys ?
Robert Syme
@robsyme
Nov 23 2015 14:07
Yup
Paolo Di Tommaso
@pditommaso
Nov 23 2015 14:08
ah, nice. Because @mariach presented nextflow some months ago at ISC in Frankfurt
and turns out that there was some people from Pawsey that were interested to nextflow
Robert Syme
@robsyme
Nov 23 2015 14:09
Well you can say that their users are already using it. One of the great things about nf is that you don't need the cluster admin's permission.
Paolo Di Tommaso
@pditommaso
Nov 23 2015 14:10
Nice.
Robert Syme
@robsyme
Nov 23 2015 14:10
I'm in Dijon at the moment, but when I get back to Perth, I'll suggest to more of the WA bioinformatics community that they should consider nf.
Paolo Di Tommaso
@pditommaso
Nov 23 2015 14:11
Yes, admin's permission is one of the worst developers' enemies ;)
Thanks for that.
Robert Syme
@robsyme
Nov 23 2015 14:13
Can you ping me here or on the github commit if you change the Capsule requirements?
Paolo Di Tommaso
@pditommaso
Nov 23 2015 14:13
sure
Robert Syme
@robsyme
Nov 23 2015 14:13
Cheers
Paolo Di Tommaso
@pditommaso
Nov 23 2015 14:14
Ciao
Paolo Di Tommaso
@pditommaso
Nov 23 2015 14:51
@robsyme it looks that is somehow related to the IBM JDK ..
is not an option to use the Open/Oracle JDK for you?
Robert Syme
@robsyme
Nov 23 2015 15:01

Default version is IBM:

~/foobar1> java -version
java version "1.7.0"
Java(TM) SE Runtime Environment (build pxa6470sr6fp1-20140108_01(SR6 FP1))
IBM J9 VM (build 2.6, JRE 1.7.0 Linux amd64-64 Compressed References 20140106_181350 (JIT enabled, AOT enabled)
J9VM - R26_Java726_SR6_20140106_1601_B181350
JIT  - r11.b05_20131003_47443.02
GC   - R26_Java726_SR6_20140106_1601_B181350_CMPRSS
J9CL - 20140106_181350)
JCL - 20140103_01 based on Oracle 7u51-b11

Switch to what I /think/ is Oracle:

~/foobar1> module load java
~/foobar1> java -version
java version "1.7.0_07"
Java(TM) SE Runtime Environment (build 1.7.0_07-b10)
Java HotSpot(TM) 64-Bit Server VM (build 23.3-b01, mixed mode)

Delete tmp and try again:

~/foobar1> rm -r ~/.nextflow/tmp
~/foobar1> nextflow run .
N E X T F L O W  ~  version 0.16.2
Launching ./main.nf
exist: true
~/foobar1> cd ../foobar2/
~/foobar2> rm -rf ../foobar1
~/foobar2> nextflow run .
N E X T F L O W  ~  version 0.16.2
Launching ./main.nf
ERROR ~ Not such file: /home/rsyme/foobar1/main.nf

 -- Check '.nextflow.log' file for details

The foobar1 string is still recorded in tmp:

~/foobar2> grep -rc foobar1 ~/.nextflow/tmp/
/home/rsyme/.nextflow/tmp/launcher/nextflow-one_0.16.2/classpath-a77aacaa0375250dddee5656a26c91dc:1
Paolo Di Tommaso
@pditommaso
Nov 23 2015 15:02
So, it's not a matter of JVM ...
I will let you know .. :)
Robert Syme
@robsyme
Nov 23 2015 15:03
No, but I hadn't even considered that I wasn't running Oracle's jdk.
Thanks :)
No rush - just hope that it helps somebody that comes after me.
Paolo Di Tommaso
@pditommaso
Nov 23 2015 15:04
yep, particularly nasty
:/
Paolo Di Tommaso
@pditommaso
Nov 23 2015 16:06
@robsyme This launcher script should solve that problem
save it somewhere and use it in place of the nextflow in your PATH
let me know if it works.
Robert Syme
@robsyme
Nov 23 2015 16:14
Perfect!
Paolo Di Tommaso
@pditommaso
Nov 23 2015 16:15
;)
great team work!
:)
Robert Syme
@robsyme
Nov 23 2015 16:25
Ha! Somewhat lopsided contributions here, but glad I could "help" :smile: