These are chat archives for nextflow-io/nextflow

6th
May 2019
Alaa Badredine
@AlaaBadredine_twitter
May 06 07:57
Hello, I have a question concerning the new update of the Nextflow 19.04.1. Before the update, it used to show the tags when the processes are launched between parenthesis. Since the updates, these tags are not showing anymore. Is there a new option that I haven't seen ?
Paolo Di Tommaso
@pditommaso
May 06 09:46
Good point, you can revert to the previous logging as described here.
@AlaaBadredine_twitter You may also want to report an request for enhancement on GitHub
Alaa Badredine
@AlaaBadredine_twitter
May 06 09:49
@pditommaso thank you, I will submit a request for the tagging feature.
Alaa Badredine
@AlaaBadredine_twitter
May 06 11:38
Stephen Kelly
@stevekm
May 06 14:54
@lebernstein I was thinking about doing something like that with the afterScript, writing some information to a file from inside the process script section then using a script called from afterScript to look for said file and load the contents into a SQLite db or something like that. The problem is that what I really want is the path to the files in the publishDir, and other aspects of the task object itself during Nextflow task execution, I think some of these things are not available when the script section executes. I think what I would really need is a task.onComplete directive that I could fill with my custom code and scripts.
Laurence E. Bernstein
@lebernstein
May 06 17:21

@stevekm I use a combination of a few things like so:

afterScript "source ${scriptsPath}/statusUpdate.sh ${sampleName} ${workingDir} process_name"

errorStrategy 'ignore'

publishDir "${workingDir}"

I use 'ignore' so I can run all samples regardless of any failures and then use the afterScript to record the last process that ran.

Ólavur Mortensen
@olavurmortensen
May 06 18:52

Does anyone by any chance have a short-hand version of this line:

set file('sample.vcf'), file('sample.vcf.idx') into variants_ch

I found a case where a tool somehow wasn't aware of the index of a file just using the symbolic link, so I have to output the index file as well. Repeating the statement above, both for output and input, is cumbersome though (both to read and to write).

Ólavur Mortensen
@olavurmortensen
May 06 19:32
Or, in other words, does someone have a more concise way of achieving the same result?
Stephen Kelly
@stevekm
May 06 19:45
not sure what alternative you had in mind, I do the same thing in my pipelines,
if for some reason the symlink to the .idx is not working then you could try using stageInMode: copy
but this will also copy the .vcf
if that is bad for you then you can use rsync inside your script block to convert the symlink to a copy of the original file
sourcepath="\$(python -c "import os; print(os.path.realpath('\${item}'))")"
            rsync -va "\${sourcepath}" "\${item}"
where item is the name of your .idx file