These are chat archives for nextflow-io/nextflow

7th
Jun 2017
Karin Lagesen
@karinlag
Jun 07 2017 08:39
hi!
I seem to remember a bin directory where I could put sw that I wanted to be able to run inside the script portion of a process, but I can't seem to find it in the docs
Paolo Di Tommaso
@pditommaso
Jun 07 2017 08:42
yes, in the PROJECT_HOME/bin
Karin Lagesen
@karinlag
Jun 07 2017 08:42
ok
Karin Lagesen
@karinlag
Jun 07 2017 08:44
thanks@
Paolo Di Tommaso
@pditommaso
Jun 07 2017 08:44
welcome
Karin Lagesen
@karinlag
Jun 07 2017 08:45
btw, I'm trying to log the git commitId, but this only gives null: ${workflow.commitId}
Paolo Di Tommaso
@pditommaso
Jun 07 2017 08:47
um, is a project downloaded from GiHub ?
Karin Lagesen
@karinlag
Jun 07 2017 08:49
no?
eh, well, yeah in a way, but not the way I think you mean :)
I assumed that all workflow things were available whenever?
Paolo Di Tommaso
@pditommaso
Jun 07 2017 08:50
that is supposed to be the Git commit when you run it directly from GH
Karin Lagesen
@karinlag
Jun 07 2017 08:51
ah, then I understand :)
Paolo Di Tommaso
@pditommaso
Jun 07 2017 08:51
have a look nextflow-io/nextflow#329
Karin Lagesen
@karinlag
Jun 07 2017 08:55
thanks!
Paolo Di Tommaso
@pditommaso
Jun 07 2017 08:55
:+1:
Karin Lagesen
@karinlag
Jun 07 2017 08:57
btw re the bin ri
directory
is it then supposed to copy the executable to the run directory?
Paolo Di Tommaso
@pditommaso
Jun 07 2017 08:58
nope, is supposed to be a shared file system folder
(assuming I've understood your question)
Karin Lagesen
@karinlag
Jun 07 2017 09:01
hm
got it to run now, thanks!
Paolo Di Tommaso
@pditommaso
Jun 07 2017 09:03
well done
Maxime Garcia
@MaxUlysse
Jun 07 2017 09:36
@skptic I caught up, and saw that you were planning to use CAW, if you ran into any problem, I'd be glad to take them ;-)
Evan Floden
@evanfloden
Jun 07 2017 09:38
Thanks @MaxUlysse. So far just just seeing how to incorperate the amazing MultiQC into NF pipelines but looking your work I think you guys should write the book on NF best practices. Very nice work and lots of ideas to take on!
Maxime Garcia
@MaxUlysse
Jun 07 2017 09:41
We got inspiration from a lot of others NF pipelines at the beginning too (and by that I mean a lot of copy/paste) ;-)
So I'm glad that it worked out so well
Karin Lagesen
@karinlag
Jun 07 2017 09:42
copy/paste is an undervalued programming technique
Maxime Garcia
@MaxUlysse
Jun 07 2017 09:43
@karinlag For the workflowId I'm using def rev = workflow.revision ?: workflow.commitId ?: workflow.scriptId.substring(0,10)
Loool, thanks a lot @pditommaso That sounds a lot like something I could copy/paste into a book
Karin Lagesen
@karinlag
Jun 07 2017 09:44
@MaxUlysse thanks! I'll incorporate that
Paolo Di Tommaso
@pditommaso
Jun 07 2017 09:45
better now :D
Shellfishgene
@Shellfishgene
Jun 07 2017 11:25
Is there a way to have a command run before the main in one in all the process sections? I mean automatically, without actually copying it into all process sections.
Shellfishgene
@Shellfishgene
Jun 07 2017 11:55
Like a beforeScript that automatically runs in all processes.
Phil Ewels
@ewels
Jun 07 2017 12:31
Not that I know of... You could assign the command to a variable, and then just put that variable name into all processes?
Shellfishgene
@Shellfishgene
Jun 07 2017 12:37
Hmm, I still have to edit all processes. This still for your RNASeq pipeline, I need to add export $PATH=$PATH:/foo/bar as a beforeScript. I didn't want to edit all the processes, to be lazy and also to be able to update. But maybe I'll just have to.
Rickard Hammarén
@Hammarn
Jun 07 2017 12:38
I think adding it to the bin folder and then calling it in the beforeScript clause of all processes is the easiest. That would be an extra line per process. Seems manageable to me
Phil Ewels
@ewels
Jun 07 2017 12:39
Shellfishgene
@Shellfishgene
Jun 07 2017 12:39
Yes, it's not really a problem. Just annoying, the whole point of nextflow is that workflowas are portable, no?
Phil Ewels
@ewels
Jun 07 2017 12:40
env.PATH = "$PATH:/foo/bar"
Maxime Garcia
@MaxUlysse
Jun 07 2017 12:40
The env thingy sounds like a smart idea to do it
Shellfishgene
@Shellfishgene
Jun 07 2017 12:40
Yes, the env scope seems like the correct way. How did I not see that before...
Phil Ewels
@ewels
Jun 07 2017 12:41
Would be interested to know if it works ;) :+1:
Shellfishgene
@Shellfishgene
Jun 07 2017 12:42
Our scheduler theoretically has a -v option to change variables, like qsub -v PATH=/foo/bar. It works great for all kinds of variables, exept for PATH...
Paolo Di Tommaso
@pditommaso
Jun 07 2017 12:43
what something about this
Phil Ewels
@ewels
Jun 07 2017 12:45
Very tidy :+1: Still have to add something to each process though.. Not loads better than just putting the command into a variable?
Shellfishgene
@Shellfishgene
Jun 07 2017 12:46
That would work, too! I'll try the env option first, if that works I won't have to touch the actual workflow at all
This scheduler has so many small details that cause very annoying problems...
Phil Ewels
@ewels
Jun 07 2017 12:50
@pditommaso - is there a way to know whether the workflow name has been specified on the command line?
We're using it in a few places now and it works really nicely, but it would be good to know whether it's a NF default or specified on the command line
eg. Don't use it as report titles if it's random :)
Paolo Di Tommaso
@pditommaso
Jun 07 2017 12:51
nope, but you can use a custom pattern to recognise it
Phil Ewels
@ewels
Jun 07 2017 12:51
ok yeah, that could certainly work..
are there specific rules that govern the defaults? /[a-zA-Z]+_[a-zA-Z]+/ I guess..
Paolo Di Tommaso
@pditommaso
Jun 07 2017 12:52
yes
Phil Ewels
@ewels
Jun 07 2017 12:54
ok awesome - all lower case too, that helps :+1:
did you compile the list of names yourself? ;)
Paolo Di Tommaso
@pditommaso
Jun 07 2017 12:55
;)
Phil Ewels
@ewels
Jun 07 2017 12:59
yay for open-source ;)
Shellfishgene
@Shellfishgene
Jun 07 2017 13:09
env.PATH="$PATH:/foo/bar" works as expected :)
Rickard Hammarén
@Hammarn
Jun 07 2017 13:10
@pditommaso You haven't thought about sneaking in Tommaso on that list? :wink:
Shellfishgene
@Shellfishgene
Jun 07 2017 13:12
I vote for adding Sanger also, at least it has Darwin...
Paolo Di Tommaso
@pditommaso
Jun 07 2017 13:20
let open a pull request for that !
Paolo Di Tommaso
@pditommaso
Jun 07 2017 14:06
@Hammarn it would be too narcissistic !
Phil Ewels
@ewels
Jun 07 2017 14:08
This sounds like a good opportunity for a twitter thread - favourite scientist names ;)
Shellfishgene
@Shellfishgene
Jun 07 2017 14:23
(rna_seq_py2.7) bash-4.1$ ~/temp/nextflow run ./NGI-RNAseq/main.nf --reads 'raw_data/*_R0{1,2}.fastq.gz' --genome fish --unstranded --outdir ./results -name fish_rnaseq_test
Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/path/to/tmp
Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/path/to/tmp
ERROR ~ Unexpected error [ExceptionInInitializerError]
Any idea what's wrong here?
Caused by: java.nio.file.AccessDeniedException: /path
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
        at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384)
        at java.nio.file.Files.createDirectory(Files.java:674)
        at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781)
        at java.nio.file.Files.createDirectories(Files.java:767)
        at nextflow.file.FileHelper.<clinit>(FileHelper.groovy:80)
        ... 12 common frames omitted
Shellfishgene
@Shellfishgene
Jun 07 2017 14:31
Sounds like a permission issue? Not sure what directory it's trying to create here.
Félix C. Morency
@fmorency
Jun 07 2017 14:32
It looks like it's trying to access /path
Shellfishgene
@Shellfishgene
Jun 07 2017 14:33
Ah, thougth that was an error message placeholder. Maybe I forgot to change a setting.
Indeed, blindly pasted java vars for conda and didn't change temp dir path
Félix C. Morency
@fmorency
Jun 07 2017 14:38
:)