These are chat archives for nextflow-io/nextflow

15th
Oct 2018
Rad Suchecki
@rsuchecki
Oct 15 2018 02:20
@tobsecret - glad you found a workaround. I believe what happens is that if you only have one file, it is just an object and if it is 2+ you have a list holding these objects. I'd go with something like this: len = bamfiles instanceof List ? bamfiles.size() : 1
anonymous
@naveenmeena584
Oct 15 2018 06:47
there are paper for bencmark there u compare result of native vs docker i want to know what is difference between these term :Tasks Mean task time Mean execution time Execution time std. deviation
i need simplification for his
itz would be very helpful
Paolo Di Tommaso
@pditommaso
Oct 15 2018 07:05
@rsuchecki @tobsecret yes, but List won't work, use len = bamfiles instanceof Path ? 1 : bamfiles.size() instead
@toniher can do that, use a collect with a dummy output to trigger the execution when the upstream task is complete, to have an idea https://github.com/nextflow-io/rnaseq-nf/blob/master/main.nf#L115
@oschwengers great! thank you!
Paolo Di Tommaso
@pditommaso
Oct 15 2018 07:15
@FirasSadiyah nextflow is a mere (parallel) executor, it creates the task wrappers and launch them with cluster, you should investigate why that package is failing when using multicores
Firas
@FirasSadiyah
Oct 15 2018 08:55
Many thanks @pditommaso, any hint where to start my investigation. SGE configuration?
Paolo Di Tommaso
@pditommaso
Oct 15 2018 09:14
I would start trying to understand better why SGE is killing it? memory limit or time limit? maybe qstat provide more information?
Firas
@FirasSadiyah
Oct 15 2018 09:55
👍
Rahul Pisupati
@rbpisupati
Oct 15 2018 14:02
Hi everyone, I am getting null error for getParent() and toRealPath() file attributes? I am calling these from process. Is there any workaround?
Paolo Di Tommaso
@pditommaso
Oct 15 2018 14:35
please include a code snippet and the error message
Rahul Pisupati
@rbpisupati
Oct 15 2018 14:39

here is the error for the code

Oct-15 14:36:48.571 [Actor Thread 4] ERROR nextflow.processor.TaskProcessor - Error executing process > 'parse_inputfiles (null)'

Caused by:
  22516_merge.snpvcf.bed

Source block:
  input_folder = input_file.toRealPath().getParent()
  prefix = input_file.getBaseName()
  """
  snpmatch parser -v -i $input_file
  """

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`
Oct-15 14:36:48.574 [Actor Thread 4] DEBUG nextflow.Session - Session aborted -- Cause: Process `null` script contains error(s)
Oct-15 14:36:48.603 [Actor Thread 4] DEBUG nextflow.Session - The following nodes are still active:

But the script works if I change input to "*bed".

Paolo Di Tommaso
@pditommaso
Oct 15 2018 14:43
you are not supposed to be able to access the file in that way
Rahul Pisupati
@rbpisupati
Oct 15 2018 14:45
@pditommaso I am not accessing the file, but I am changing storeDir where input file is present.
Paolo Di Tommaso
@pditommaso
Oct 15 2018 14:48
let's put in another way, NF is designed to avoid tasks to work in the input source path
Maxime Garcia
@MaxUlysse
Oct 15 2018 14:50
Hi @pditommaso is it possible to rename a file when remapping a channel between two processes ?
Paolo Di Tommaso
@pditommaso
Oct 15 2018 14:51
input:
file 'hola.txt' from your_channel
that's it, you don't need to rename it
Maxime Garcia
@MaxUlysse
Oct 15 2018 14:51
even easier
perfect
Maxime Garcia
@MaxUlysse
Oct 15 2018 14:58
OK, so it's not working, because, I want my file to be name after a variable that is in my input set...
set idPatient, status, idSample, file "${idSample}.bam" from mergedBam
something like that
Rahul Pisupati
@rbpisupati
Oct 15 2018 14:59
@pditommaso how do you suggest to add files to input source path?
Paolo Di Tommaso
@pditommaso
Oct 15 2018 15:15
@MaxUlysse yes
@rbpisupati if you want those file copied there and then used by a downstream task, you are doing wrong
Maxime Garcia
@MaxUlysse
Oct 15 2018 15:17
yes, but it is not working
Paolo Di Tommaso
@pditommaso
Oct 15 2018 15:17
@rbpisupati if instead you just want to copy there use publishDir
Maxime Garcia
@MaxUlysse
Oct 15 2018 15:17
I have "No such variable: idSample"
OK, just need to add ()
Paolo Di Tommaso
@pditommaso
Oct 15 2018 15:19
@MaxUlysse very good ;)
Maxime Garcia
@MaxUlysse
Oct 15 2018 15:20
one small problem, I add that before:
set idPatient, status, idSample, file(bam) from mergedBam
Rahul Pisupati
@rbpisupati
Oct 15 2018 15:20
@pditommaso okay thanks! yeah i know it is bit messy. i will change the code :))
Maxime Garcia
@MaxUlysse
Oct 15 2018 15:21
So I would want this to be accessible from bam in fact
Paolo Di Tommaso
@pditommaso
Oct 15 2018 15:21
not sure to understand
Maxime Garcia
@MaxUlysse
Oct 15 2018 15:22
long story short, this bam file does not have a consistent file name, so I want it to be named ${idSample}.bam and be accessible within bam
Paolo Di Tommaso
@pditommaso
Oct 15 2018 15:23
you should replace in the command $bam with ${idSample}.bam
won't that be enough ?
or declare
script:
def bam = "${idSample}.bam"
Maxime Garcia
@MaxUlysse
Oct 15 2018 15:25
Nothing easier than that?
Can't do:
set idPatient, status, idSample, file("${idSample}.bam" = bam) from mergedBam
Paolo Di Tommaso
@pditommaso
Oct 15 2018 15:25
LOL
syntax of the day
file(bam: "${idSample}.bam" )
should do the trick
tho, I don't like it and I think at some point I will deprecate it
Maxime Garcia
@MaxUlysse
Oct 15 2018 15:27

ok, so better use:

script:
def bam = "${idSample}.bam"

already

But yeah, that was exactly what I was looking for
thanks a lot
Caspar
@caspargross
Oct 15 2018 16:21

is there a way to access workflow variables in the .config files? I would like to create a profile where I assign one parameter depending on the projectDir. Like this:

params { sample = "${workflow.projectDir}/data/test_samples.tsv"}

is there a way to get this to work?

Paolo Di Tommaso
@pditommaso
Oct 15 2018 16:26
in the config file you can use baseDir, it should do what you are looking for
Caspar
@caspargross
Oct 15 2018 16:26
thanks!
Brad Langhorst
@bwlang
Oct 15 2018 20:28
is there any way to copy a directory from the working directory to the published location? publishDir "${outputDir}/bams_for_import/", mode: 'copyNoFollow', pattern: 'bams_for_import/*.{md.bam}’ does not seem to do it. (this is a directory of symlinks like ../foo.bam)
Tobias "Tobi" Schraink
@tobsecret
Oct 15 2018 20:29
:pray: Thanks folks!