These are chat archives for nextflow-io/nextflow

14th
Oct 2016
amacbride
@amacbride
Oct 14 2016 04:11

@pditommaso Is there any way to access the name of the current process from within the script execution? (I didn't see anything in the workflow introspection docs.) I'd find it useful both within the script and also as part of the definition block -- so for example, using the convention that the template script is derived from the process name.

For example:

process myName {

    […]

    script:
        template “${workflow.processName}.sh”
}



myName.sh:
    echo “starting ${workflow.processName}
Evan Floden
@evanfloden
Oct 14 2016 06:40
@amacbride try this, I think it solves the script part anyway.
process myName {

    script:
    """
        echo ${task.process}.sh
    """
}
amacbride
@amacbride
Oct 14 2016 14:40
Awesome! I will try it out.
(I'm using external template scripts instead of inline, but if that variable is passed in, it should work)
Paolo Di Tommaso
@pditommaso
Oct 14 2016 14:45
yes
Félix C. Morency
@fmorency
Oct 14 2016 15:08
mmm tried publishDir = "./results_${params.id}/${task.process}" and it doesn't work
Paolo Di Tommaso
@pditommaso
Oct 14 2016 15:09
always DiD ?
Félix C. Morency
@fmorency
Oct 14 2016 15:09
no, local executor on bare metal
Paolo Di Tommaso
@pditommaso
Oct 14 2016 15:10
does your process declare any output file?
Félix C. Morency
@fmorency
Oct 14 2016 15:10
yes
Paolo Di Tommaso
@pditommaso
Oct 14 2016 15:11
it must work, what's the problem ?
Félix C. Morency
@fmorency
Oct 14 2016 15:11
outputs are located in results_XYZ/[:]
Paolo Di Tommaso
@pditommaso
Oct 14 2016 15:12
um, let me try
Félix C. Morency
@fmorency
Oct 14 2016 15:13
process { publishDir = "./results_${params.id}/${task.process}" }
is located in my nextflow.config
Paolo Di Tommaso
@pditommaso
Oct 14 2016 15:14
ahh, the point is that task variable is not accessible in that context
you can use it only in the process script
no wait
it's a bit more tricky
if you put that in the config file
it tries to resolve task.process when parsing the config
Paolo Di Tommaso
@pditommaso
Oct 14 2016 15:20
resulting in that empty value
Félix C. Morency
@fmorency
Oct 14 2016 15:20
is there a generic way of doing ^
Paolo Di Tommaso
@pditommaso
Oct 14 2016 15:20
Yes, is this way
process {
    publishDir = {"./results_${params.id}/${task.process}"}

}
you have to enclose in a { .. } all the string so it's lazy evaluated
you an simplify a bit, like this
process {
    publishDir = {"results_$params.id/$task.process"}
}
Félix C. Morency
@fmorency
Oct 14 2016 15:23
it works
cool!
Paolo Di Tommaso
@pditommaso
Oct 14 2016 15:24
:+1:
Félix C. Morency
@fmorency
Oct 14 2016 15:48
how do you handle python venv in nextflow?
Paolo Di Tommaso
@pditommaso
Oct 14 2016 15:55
good question, actually I'm not a big python user
what env vars set venv ?
Félix C. Morency
@fmorency
Oct 14 2016 15:56
good question. i know that the python executable will be located at a different place depending on the venv
rapidly, i see that it sets VIRTUAL_ENV and PATH
Paolo Di Tommaso
@pditommaso
Oct 14 2016 15:59
env {
  PATH="..." 
  VIRTUAL_ENV="..."
}
in the nextflow.config file
I think python also uses PYHTONPATH
Félix C. Morency
@fmorency
Oct 14 2016 16:01
indeed
Félix C. Morency
@fmorency
Oct 14 2016 17:49
can we access to process inputs from perl/python scripting?
Paolo Di Tommaso
@pditommaso
Oct 14 2016 17:50
it's an inline perl/python script?
Félix C. Morency
@fmorency
Oct 14 2016 17:50
yes
Paolo Di Tommaso
@pditommaso
Oct 14 2016 17:51
you will need to use the proper perl/python syntax to handle that
the process input is just a file path string
Félix C. Morency
@fmorency
Oct 14 2016 17:52
basically "${myfile}" should work?
Paolo Di Tommaso
@pditommaso
Oct 14 2016 17:52
yep
Félix C. Morency
@fmorency
Oct 14 2016 17:52
impressive
Paolo Di Tommaso
@pditommaso
Oct 14 2016 17:53
but you would need to escape perl $
or use the alternate var syntax with shell
Félix C. Morency
@fmorency
Oct 14 2016 18:02
ok if the venv is activated in my shell nextflow gets it
no need to env {} and whatnot
is there a way to run something if the task is skipped? like create dummy files
Paolo Di Tommaso
@pditommaso
Oct 14 2016 18:13
is there a way to run something if the task is skipped?
how exactly is skipped the task ?
Félix C. Morency
@fmorency
Oct 14 2016 18:13
using when:
Paolo Di Tommaso
@pditommaso
Oct 14 2016 18:14
nope
if you want the output in any case, you can use a conditional script eg.
Félix C. Morency
@fmorency
Oct 14 2016 18:15
i see
Paolo Di Tommaso
@pditommaso
Oct 14 2016 18:15
process foo {

script:
if( <cond> ) 
"""
do this
"""
else 
"""
do that
"""
}
Félix C. Morency
@fmorency
Oct 14 2016 18:50
can you use scoped variable and params like params { beta { skip = false } } and pass them on the command line? --beta.skip true doesn't work
Paolo Di Tommaso
@pditommaso
Oct 14 2016 18:52
it should work
Félix C. Morency
@fmorency
Oct 14 2016 18:53
mmm I pass --beta.skip true and log.info "$params.beta.skip" and it prints false
Paolo Di Tommaso
@pditommaso
Oct 14 2016 18:53
wait
nested params are not supposed to work
replace params { beta { skip = false } } with params { beta_skip = false }
Félix C. Morency
@fmorency
Oct 14 2016 18:55
yeah ok, that works
Félix C. Morency
@fmorency
Oct 14 2016 20:26
i have the first part of our pipeline implemented using nextflow
\o/
Paolo Di Tommaso
@pditommaso
Oct 14 2016 20:46
Cool!
amacbride
@amacbride
Oct 14 2016 21:32
Sorry if this is a dumb question, but where did the .nextflow.log file end up? I no longer see it after upgrading to 0.22.3 from 0.19.4 (I thought it might be in the .nextflow dir in the working dir, but I didn't see it.)
Paolo Di Tommaso
@pditommaso
Oct 14 2016 21:55
@amacbride .nextflow.log is always in the launch directory, .nextflow.history was moved to .nextflow/history
amacbride
@amacbride
Oct 14 2016 23:24
Ah, figured it out -- I had run a script as root a few months back, so there were .nextflow.log.* files owned by root -- the current script ran (as myself), but didn't print anything to stdout saying that it couldn't write the log file.