These are chat archives for nextflow-io/nextflow

15th
Oct 2018
Rad Suchecki
@rsuchecki
Oct 15 2018 02:20 UTC
@tobsecret - glad you found a workaround. I believe what happens is that if you only have one file, it is just an object and if it is 2+ you have a list holding these objects. I'd go with something like this: len = bamfiles instanceof List ? bamfiles.size() : 1
anonymous
@naveenmeena584
Oct 15 2018 06:47 UTC
there are paper for bencmark there u compare result of native vs docker i want to know what is difference between these term :Tasks Mean task time Mean execution time Execution time std. deviation
i need simplification for his
itz would be very helpful
Paolo Di Tommaso
@pditommaso
Oct 15 2018 07:05 UTC
@rsuchecki @tobsecret yes, but List won't work, use len = bamfiles instanceof Path ? 1 : bamfiles.size() instead
@toniher can do that, use a collect with a dummy output to trigger the execution when the upstream task is complete, to have an idea https://github.com/nextflow-io/rnaseq-nf/blob/master/main.nf#L115
@oschwengers great! thank you!
Paolo Di Tommaso
@pditommaso
Oct 15 2018 07:15 UTC
@FirasSadiyah nextflow is a mere (parallel) executor, it creates the task wrappers and launch them with cluster, you should investigate why that package is failing when using multicores
Firas
@FirasSadiyah
Oct 15 2018 08:55 UTC
Many thanks @pditommaso, any hint where to start my investigation. SGE configuration?
Paolo Di Tommaso
@pditommaso
Oct 15 2018 09:14 UTC
I would start trying to understand better why SGE is killing it? memory limit or time limit? maybe qstat provide more information?
Firas
@FirasSadiyah
Oct 15 2018 09:55 UTC
👍
Rahul Pisupati
@rbpisupati
Oct 15 2018 14:02 UTC
Hi everyone, I am getting null error for getParent() and toRealPath() file attributes? I am calling these from process. Is there any workaround?
Paolo Di Tommaso
@pditommaso
Oct 15 2018 14:35 UTC
please include a code snippet and the error message
Rahul Pisupati
@rbpisupati
Oct 15 2018 14:39 UTC

here is the error for the code

Oct-15 14:36:48.571 [Actor Thread 4] ERROR nextflow.processor.TaskProcessor - Error executing process > 'parse_inputfiles (null)'

Caused by:
  22516_merge.snpvcf.bed

Source block:
  input_folder = input_file.toRealPath().getParent()
  prefix = input_file.getBaseName()
  """
  snpmatch parser -v -i $input_file
  """

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`
Oct-15 14:36:48.574 [Actor Thread 4] DEBUG nextflow.Session - Session aborted -- Cause: Process `null` script contains error(s)
Oct-15 14:36:48.603 [Actor Thread 4] DEBUG nextflow.Session - The following nodes are still active:

But the script works if I change input to "*bed".

Paolo Di Tommaso
@pditommaso
Oct 15 2018 14:43 UTC
you are not supposed to be able to access the file in that way
Rahul Pisupati
@rbpisupati
Oct 15 2018 14:45 UTC
@pditommaso I am not accessing the file, but I am changing storeDir where input file is present.
Paolo Di Tommaso
@pditommaso
Oct 15 2018 14:48 UTC
let's put in another way, NF is designed to avoid tasks to work in the input source path
Maxime Garcia
@MaxUlysse
Oct 15 2018 14:50 UTC
Hi @pditommaso is it possible to rename a file when remapping a channel between two processes ?
Paolo Di Tommaso
@pditommaso
Oct 15 2018 14:51 UTC
input:
file 'hola.txt' from your_channel
that's it, you don't need to rename it
Maxime Garcia
@MaxUlysse
Oct 15 2018 14:51 UTC
even easier
perfect
Maxime Garcia
@MaxUlysse
Oct 15 2018 14:58 UTC
OK, so it's not working, because, I want my file to be name after a variable that is in my input set...
set idPatient, status, idSample, file "${idSample}.bam" from mergedBam
something like that
Rahul Pisupati
@rbpisupati
Oct 15 2018 14:59 UTC
@pditommaso how do you suggest to add files to input source path?
Paolo Di Tommaso
@pditommaso
Oct 15 2018 15:15 UTC
@MaxUlysse yes
@rbpisupati if you want those file copied there and then used by a downstream task, you are doing wrong
Maxime Garcia
@MaxUlysse
Oct 15 2018 15:17 UTC
yes, but it is not working
Paolo Di Tommaso
@pditommaso
Oct 15 2018 15:17 UTC
@rbpisupati if instead you just want to copy there use publishDir
Maxime Garcia
@MaxUlysse
Oct 15 2018 15:17 UTC
I have "No such variable: idSample"
OK, just need to add ()
Paolo Di Tommaso
@pditommaso
Oct 15 2018 15:19 UTC
@MaxUlysse very good ;)
Maxime Garcia
@MaxUlysse
Oct 15 2018 15:20 UTC
one small problem, I add that before:
set idPatient, status, idSample, file(bam) from mergedBam
Rahul Pisupati
@rbpisupati
Oct 15 2018 15:20 UTC
@pditommaso okay thanks! yeah i know it is bit messy. i will change the code :))
Maxime Garcia
@MaxUlysse
Oct 15 2018 15:21 UTC
So I would want this to be accessible from bam in fact
Paolo Di Tommaso
@pditommaso
Oct 15 2018 15:21 UTC
not sure to understand
Maxime Garcia
@MaxUlysse
Oct 15 2018 15:22 UTC
long story short, this bam file does not have a consistent file name, so I want it to be named ${idSample}.bam and be accessible within bam
Paolo Di Tommaso
@pditommaso
Oct 15 2018 15:23 UTC
you should replace in the command $bam with ${idSample}.bam
won't that be enough ?
or declare
script:
def bam = "${idSample}.bam"
Maxime Garcia
@MaxUlysse
Oct 15 2018 15:25 UTC
Nothing easier than that?
Can't do:
set idPatient, status, idSample, file("${idSample}.bam" = bam) from mergedBam
Paolo Di Tommaso
@pditommaso
Oct 15 2018 15:25 UTC
LOL
syntax of the day
file(bam: "${idSample}.bam" )
should do the trick
tho, I don't like it and I think at some point I will deprecate it
Maxime Garcia
@MaxUlysse
Oct 15 2018 15:27 UTC

ok, so better use:

script:
def bam = "${idSample}.bam"

already

But yeah, that was exactly what I was looking for
thanks a lot
Caspar
@caspargross
Oct 15 2018 16:21 UTC

is there a way to access workflow variables in the .config files? I would like to create a profile where I assign one parameter depending on the projectDir. Like this:

params { sample = "${workflow.projectDir}/data/test_samples.tsv"}

is there a way to get this to work?

Paolo Di Tommaso
@pditommaso
Oct 15 2018 16:26 UTC
in the config file you can use baseDir, it should do what you are looking for
Caspar
@caspargross
Oct 15 2018 16:26 UTC
thanks!
Brad Langhorst
@bwlang
Oct 15 2018 20:28 UTC
is there any way to copy a directory from the working directory to the published location? publishDir "${outputDir}/bams_for_import/", mode: 'copyNoFollow', pattern: 'bams_for_import/*.{md.bam}’ does not seem to do it. (this is a directory of symlinks like ../foo.bam)
Tobias "Tobi" Schraink
@tobsecret
Oct 15 2018 20:29 UTC
:pray: Thanks folks!