These are chat archives for nextflow-io/nextflow

5th
Nov 2015
Michael L Heuer
@heuermh
Nov 05 2015 17:03
FYI version 0.16.0 has been released and is available from homebrew/science, thanks!
Paolo Di Tommaso
@pditommaso
Nov 05 2015 17:03
I've seen that, thanks!
any progress on nxf + spark ?
Michael L Heuer
@heuermh
Nov 05 2015 17:06

A question about the new publishDir directive:

This works

params.outputDir = "${baseDir}/${params.sample}"
params.publishMode = "symlink" // or copy, link

outputDir = "${params.outputDir}"
publishMode = "${params.publishMode}"

process downloadSample {
  publishDir outputDir, mode: publishMode

but this does not

bamFiles = "${params.dir}/**.bam"
bams = Channel.fromPath(bamFiles).map { path -> tuple(path.baseName, path.toRealPath().getParent(), path) }

process transform {
  publishDir parent, mode: 'copy'

  input:
    set sample, parent, file(bam) from bams

Can I pull an input variable like parent and put it in the publishDir directive, or do I need to find another way to get it there? The use case here is to publish to the same directory as the input BAM file

Paolo Di Tommaso
@pditommaso
Nov 05 2015 17:09
I'm missing something
what's the file that you want to be copied
Michael L Heuer
@heuermh
Nov 05 2015 17:12

sorry, here's the rest

  output:
    set sample, parent, file("${sample}.adam") into reads

  """
  adam-submit transform -force_load_bam $bam ${sample}.adam
  """
}

so I'd like to publish via copy ${sample}.adam to the same directory that $bam is in

Paolo Di Tommaso
@pditommaso
Nov 05 2015 17:13
it should, what is happening ?
Michael L Heuer
@heuermh
Nov 05 2015 17:14
ERROR ~ Not such variable: parent
Paolo Di Tommaso
@pditommaso
Nov 05 2015 17:14
ahhh
your are trying to use a variable in the context of the process
not supported (yet)
unless .. try the following
publishDir "$parent", mode: 'copy'
Michael L Heuer
@heuermh
Nov 05 2015 17:17
if I use publishDir "${parent}" it doesn't complain but writes to a null directory
yours results in ERROR ~ Not such variable: $parent
Paolo Di Tommaso
@pditommaso
Nov 05 2015 17:18
I think I've understood the problem
If you write only publishDir "$parent" it should work
Michael L Heuer
@heuermh
Nov 05 2015 17:18
ok good because I probably only half understand it :)
Paolo Di Tommaso
@pditommaso
Nov 05 2015 17:19
:)
anyway would you like to open an issue for that?
Michael L Heuer
@heuermh
Nov 05 2015 17:21

publishDir "$parent" also writes to a directory called null. Maybe the path.toRealPath().getParent() doesn't work where it is now?

The previous version took some fine tuning to get right, https://github.com/heuermh/bdg-nextflow/blob/master/transform.nf#L24

Paolo Di Tommaso
@pditommaso
Nov 05 2015 17:22
no I think it's a problem that does not resolve variables in the process scope
Michael L Heuer
@heuermh
Nov 05 2015 17:24
ok, maybe related, in the first example, I get
Cannot get property 'outputDir' on null object
if instead of publishDir outputDir I use publishDir "${params.outputDir}"
Paolo Di Tommaso
@pditommaso
Nov 05 2015 17:28
I'm not able to replicate
I've just run this:
params.outDir = 'x'

process foo {
  publishDir "$params.outDir" 

  output: 
  file 'hello'

  '''
  echo Hello > hello
  '''
}
running it -with-docker heuermh/adam should work, then modify the publishDir lines to replicate
Paolo Di Tommaso
@pditommaso
Nov 05 2015 17:36
ok, I will debug it a bit more
Michael L Heuer
@heuermh
Nov 05 2015 17:37
thank you for looking into it! I'm always impressed by your willingness to help.
Paolo Di Tommaso
@pditommaso
Nov 05 2015 17:37
LOL
me too :)