These are chat archives for nextflow-io/nextflow

22nd
Nov 2014
Paolo Di Tommaso
@pditommaso
Nov 22 2014 00:00
With nextflow you don't need to do that because you can emit that information in a tuple, for example
Michael L Heuer
@heuermh
Nov 22 2014 00:02
Yeah, let's see, the pipeline should really just take sample (or samples) as its only parameter
Paolo Di Tommaso
@pditommaso
Nov 22 2014 00:02
process doThis {

  output: 
  set x, file "ssake.fa.gz" into ssakeFasta 

  script: 
  x = 'your value here'  
  """
  echo ${params.sample} 
  """ 
}
Michael L Heuer
@heuermh
Nov 22 2014 00:02
then I could create a Channel that outputs the _R1/_R1 fastq files and then go from there
er, _R1/_R2 I mean
Paolo Di Tommaso
@pditommaso
Nov 22 2014 00:03
Yes, have a look to this
at the end of your pipeline you can rename the final result files, with a "semantic" id/name
Michael L Heuer
@heuermh
Nov 22 2014 00:04
Yeah I like that, feels more "reactive", which the nextflow api is (curious, do you use Rx under the hood?)
Paolo Di Tommaso
@pditommaso
Nov 22 2014 00:04
Yes!
it was really inspired to RxJava but it is build on top of Gpars dataflow
that is the "magic" component
Michael L Heuer
@heuermh
Nov 22 2014 00:06
Then for some tools that output a ton of files (and not necessarily always the same number of files) in order to pass that along to the next process might it be best to gzip an output directory?
Paolo Di Tommaso
@pditommaso
Nov 22 2014 00:07
I'm supposing that you are working in a shared file system, right?
Michael L Heuer
@heuermh
Nov 22 2014 00:08
Not sure exactly how we're going to be deploying this yet; we set up a small cluster on SLURM but I've only been working locally so far
Paolo Di Tommaso
@pditommaso
Nov 22 2014 00:09
OK, well in any case you do not need to compress the output directory
the files are passed as symlinks to the next stage
Michael L Heuer
@heuermh
Nov 22 2014 00:10
I see, so if the working directory is shared across the cluster, then everything should be ok
Paolo Di Tommaso
@pditommaso
Nov 22 2014 00:10
yep
Michael L Heuer
@heuermh
Nov 22 2014 00:11
Cool! I'm impressed that I got something working in only an hour or two this afternoon, and should be able to clean it up nicely
Paolo Di Tommaso
@pditommaso
Nov 22 2014 00:12
Thanks
it's a piece of software design for bioinformaticians
Michael L Heuer
@heuermh
Nov 22 2014 00:13
The only thing that's not so great is the mixed escaping due to groovy vs. bash, but it's not too difficult to figure out
Paolo Di Tommaso
@pditommaso
Nov 22 2014 00:16
Well, the idea is that the script should not to be too complex
if you have a big bash piece of code you may consider to put in a external script and invoke from the process
Michael L Heuer
@heuermh
Nov 22 2014 00:17

Agreed.

The workaround from above appears not to work

process fastqToSsake {
  input:
  file r1 from fastq
  file r2 from fastq

  output:
  file "${s}.ssake.fa.gz" into ssakeFasta

  script:
  def s = params.sample
  """                                                                                                                                       
  ngs-fastq-to-ssake -1 ${r1} -2 ${r2} -o ${s}.ssake.fa.gz --insert-size 500                                                                
  """
}
$ ./nextflow run assemble.nf
N E X T F L O W  ~  version 0.11.2
[warm up] executor > local
ERROR ~ Not such variable: s
Paolo Di Tommaso
@pditommaso
Nov 22 2014 00:18
ah, sorry
Michael L Heuer
@heuermh
Nov 22 2014 00:18
not a big deal though, I'll rewrite the sample vs. file handling stuff as discussed above
Paolo Di Tommaso
@pditommaso
Nov 22 2014 00:18
do not use del s
but simply
s = params.simple
Michael L Heuer
@heuermh
Nov 22 2014 00:19
Ah yes, that works
Paolo Di Tommaso
@pditommaso
Nov 22 2014 00:19
god
Michael L Heuer
@heuermh
Nov 22 2014 00:19
Thanks! And have a nice weeked
Paolo Di Tommaso
@pditommaso
Nov 22 2014 00:19
oops .. I mean good :)
Michael L Heuer
@heuermh
Nov 22 2014 00:19
er weekend
Paolo Di Tommaso
@pditommaso
Nov 22 2014 00:19
the same to you
Michael L Heuer
@heuermh
Nov 22 2014 00:19
I'm not sure why, but I type a lot worse in this chat interface than in irc
Paolo Di Tommaso
@pditommaso
Nov 22 2014 00:21
yes, but at the end is not so bad gitter
ah, star the project if you like it ;)