These are chat archives for nextflow-io/nextflow

12th
Apr 2019
Rad Suchecki
@rsuchecki
Apr 12 00:46
Thanks @tobsecret my thoughts/experience exactly, but a friendly sysadmin shows interest in NF and into making it available as an env module. Not really necessary, but there is a small chance that this might tempt more users to try NF so I think it may be worth a shot.
@olavurmortensen yes, in this case e.g.
output:
  file outfile into channel1, channel2
Ólavur Mortensen
@olavurmortensen
Apr 12 08:33
@rsuchecki cool, that's more clean than the syntax I was using :D
Thank you
sureshhewa
@sureshhewabi
Apr 12 09:59
what is the syntax to get files into a variable without considering the case of the fie extension?
Eg: file '*.fasta' into myfasta
file '*.FASTA' into myfasta
file '*.fAsTa' into myfasta
I gave FASTA as an example, it is not what I am using. I just need to mention the extension in none case sensitive way
micans
@micans
Apr 12 10:04
First thing that comes to my minds is '*.[fF][aA][sS][tT][aA]' which is not great (and not tested whether it will work). Case insensitive suffixes are pretty un-Unixy.
sureshhewa
@sureshhewabi
Apr 12 10:10
Thanks @micans , I can have a go. Any better suggestions are welcome
Ólavur Mortensen
@olavurmortensen
Apr 12 10:27
@sureshhewabi @micans What about *.{fasta,FASTA}? Same thing, just different syntax.
micans
@micans
Apr 12 10:27
that does not catch fAsTa ..
there are 32 possibilities.
Maxime Garcia
@MaxUlysse
Apr 12 10:41
What about *.{f,F}*{a,A}?
'm guessing there's not that many output that would match what you actually want
micans
@micans
Apr 12 10:42
well, it would catch other stuff as well. Pros and cons.
Maxime Garcia
@MaxUlysse
Apr 12 10:42
Are people/tools really writing .fAsTA ?
micans
@micans
Apr 12 10:43
well it allows the use of 5 more bits ... lots you can encode with 5 bits.
sureshhewa
@sureshhewabi
Apr 12 11:05
@olavurmortensen For the time being I can use the format for the time being although it does not cover all the possibilities. It would be great if I can apply a function like UPPERCASE() to avoid this issue
@MaxUlysse As I said, I took FASTA as an example. but there are other file formats that has this issue
Jonas T Björkman
@tintin42
Apr 12 11:27
@rsuchecki Mmm, so in the outlined example you mean changing the executor to local for the final step only? That would help in that specifc scenario, but would of course not be a good solution if the tasks would consume a more substantial amount of resources.
So I take it that in NF there is really no way to avoid delays in a pipeline (with dependent jobs) if other jobs (other NF pipeline, other tool, manual submissions, etc.) are submitted to a queue with similar priority while the first pipeline is in progress of being executed.
krlgit
@krlgit
Apr 12 13:44

Hi there,
i have two beginner-questions:
(1) IntelliJ Idea, when set to Groovy, gives me basic hit-and-miss syntax highlighting of .nf files and complains about a lot of 'errors'
--> is there a better setup/editor to work with nextflow's DSL?

(2) Is there a convenient way to write (Unit-)Tests for processes?

Thanks!

Alexander Peltzer
@apeltzer
Apr 12 13:47
(has a Nextflow syntax support via extension)
https://atom.io/ works very well too
Regarding (2): Not really, but you may want to have a look here https://www.youtube.com/watch?v=V_IOX5VsaNs
Daniel E Cook
@danielecook
Apr 12 14:17
Does anyone know if a bash script is run in the background whether nextflow will wait for it to complete?
e.g.
long_command input1.txt & # run in bg
long_command input2.txt # run in fg
krlgit
@krlgit
Apr 12 14:21
Thanks @apeltzer !
micans
@micans
Apr 12 14:26
@danielecook ...
.command.run runs .command.stub, and that runs .command.sh. The stub file ends like this:
start_millis=$(nxf_date)
(
/bin/bash -euo pipefail /some/path/work/13ad4a462ec8c88f195101876938d2/.command.sh
) &
pid=$!
nxf_trace "$pid" .command.trace &
mon=$!
wait $pid || ret=$?
end_millis=$(nxf_date)
wait $mon
echo $((end_millis-start_millis)) >> .command.trace
Daniel E Cook
@danielecook
Apr 12 14:26
ah ok good to know
Thanks!
micans
@micans
Apr 12 14:26
:+1:
Tobias "Tobi" Schraink
@tobsecret
Apr 12 14:33
@rsuchecki Fair enough - our cluster admins did the same thing, but stopped updating the modules pretty quickly.
krlgit
@krlgit
Apr 12 16:59

I'm trying to specify a relative path in my script like
'../sideload/somefile' -- which does not work.

Is there a variable that is dynamically set to the directory of the .nf file, so i could do something like
../$NFHOME/sideload/somefile ?

micans
@micans
Apr 12 16:59
$baseDir
krlgit
@krlgit
Apr 12 17:00
Oh my... thanks! :)
Was too stupid to find it in the docs.
micans
@micans
Apr 12 17:01
np :)
Michael L Heuer
@heuermh
Apr 12 17:04
How should one add libraries to the classpath for the exec: process keyword?
Michael L Heuer
@heuermh
Apr 12 17:36
def libraries = ConfigHelper.resolveClassPaths( session.getLibDir() )
https://github.com/nextflow-io/nextflow/blob/master/modules/nextflow/src/main/groovy/nextflow/script/ScriptRunner.groovy#L301
Getting somewhere...
Must be NXF_CLASSPATH environment variable; I see NXF_LIB, config.libDir refered to in the source code, but those aren't surfaced in the docs