These are chat archives for nextflow-io/nextflow

29th
Jul 2015
Michael L Heuer
@heuermh
Jul 29 2015 16:58
I think I might be missing a simple trick; I would like to write an output file to the same directory as the input file when searching recursively.
bams = Channel.fromPath("${baseDir}/**.bam")
process transform {
  input:
    file(bam) from bams

  """
  do-something -i ${bam} -o ${bam with different file extension}
  """
}
Paolo Di Tommaso
@pditommaso
Jul 29 2015 19:13
@heuermh um, that's really discouraged, however I think you should be able to do that with
do-something -i ${bam} -o \$(readlink -f ${bam with different file extension})
Michael L Heuer
@heuermh
Jul 29 2015 19:41

Interesting, will readlink work across executors & docker?

Here's a more complete example

#!/usr/bin/env nextflow

bamFiles = "${baseDir}/**bam"
bams = Channel.fromPath(bamFiles)

process transform {
  input:
    set file(bam) from bams
  output:
    set bam, file("${bam}.filename") into filenames

  """
  echo ${bam} > ${bam}.filename
  """
}

filenames.subscribe { bam, filename -> copy(bam, filename) }

def copy (bam, filename) {
  log.info "Copying ${filename} alongside ${bam}"
  filename.copyTo(bam.toAbsolutePath().getParent().toAbsolutePath())
}

Given

foo.bam
bar/bar.bam

it results in

bar.bam.filename
foo.bam
foo.bam.filename
bar/bar.bam

Maybe I need a toRealPath in there, which I guess is the JDK equivalent of readlink.

Paolo Di Tommaso
@pditommaso
Jul 29 2015 19:44
ok, with a method it's much easier
def copy (bam, filename) {
  log.info "Copying ${filename} alongside ${bam}"
  filename.copyTo( bam.toRealPath().resolveSibling(bam.name) )
}
I haven't tried but it should work
have a look to the Path API
no readlink it's really an hack I think it won't work with Docker, much better doing that with an external method
Michael L Heuer
@heuermh
Jul 29 2015 19:48
Copying /Users/xxx/working/bdg-nextflow/work/78/57dde986b5177214adfe7dcfc3af9b/bar.bam.filename alongside bar.bam
ERROR ~ Not such file: bar.bam
Paolo Di Tommaso
@pditommaso
Jul 29 2015 19:49
sorry
filename.toRealPath().resolveSibling(bam.name)
it's filename the source, not bam
Michael L Heuer
@heuermh
Jul 29 2015 19:58
hmm, I'm trying that and a few other things, they don't resolve out of the work directory. From what I can tell after bar/bar.bam is captured the path is only bar.bam, which is convenient for process isolation, but breaks this kind of wanting to work with legacy tools stuff.
Paolo Di Tommaso
@pditommaso
Jul 29 2015 20:05
let me try
Michael L Heuer
@heuermh
Jul 29 2015 20:07
maybe I need to use a regex instead of glob pattern to capture the intermediate directories?
Paolo Di Tommaso
@pditommaso
Jul 29 2015 20:08
no, the real problem is that set bam, file("${bam}.filename") into ..
the bam is considered a value not a file
Michael L Heuer
@heuermh
Jul 29 2015 20:10

oh, well if I do this

bams = Channel.fromPath(bamFiles).map { path -> tuple(path, path.toRealPath().name) }

the real path is bar.bam not bar/bar.bam

Paolo Di Tommaso
@pditommaso
Jul 29 2015 20:17
I've manage do that in this way
process foo {

  input: 
  file bam from file("$HOME/sample.bam")

  output: 
  set 'source', '*.bai' into filenames 

  """
  cp $bam ${bam.baseName}.bai 
  ln -s \$(readlink $bam) source
  """
}

filenames.subscribe {
  println ([ it[0].toRealPath(), it[1] ])
}
but it's pretty ugly
the problem is that set bam, file("${bam}.filename") into filenames
the bam is resolve as a value not a file
ideally it should be possible to file set file(bam), file("${bam}.filename") into filenames
but does not work
I will give a try to see if can manage to improve it