These are chat archives for nextflow-io/nextflow

27th
Mar 2019
Joseph Guhlin
@jguhlin
Mar 27 01:23
Does groovy have na easy way to escape strings for use in filenames / scripts? For example, a list of filenames with spaces in them needing to be passed to the command line. (samtools merge out.cram $reads but $reads has filenames with spaces in them)
Tobias "Tobi" Schraink
@tobsecret
Mar 27 01:32
You can use "samtools merge out.cram ${reads.join(' --reads ')}" afaik
Not sure if that's the exact groovyism but there's something to that effect for sure
Joseph Guhlin
@jguhlin
Mar 27 01:36
Awesome, thanks. I'll give that a try
didn't work. :/ I'm using this in a script block """ so not sure if that would affect it, thanks though
Just cheating and using *.cram for now, seems to work thanks to nf's workdir magic
Tobias "Tobi" Schraink
@tobsecret
Mar 27 01:43
Oh, then you can leave out the "" that I added ofc.
${reads.join(" somestring ")} should concatenate [read1, read2, read3] to somestring read1 somestring read2 somestring read3. Again, I don't remember the exact groovyism
Joseph Guhlin
@jguhlin
Mar 27 01:46
ah yeah, but the issue is the spaces in the filenames borking the command line, was hoping there was an escape function so I could just map them against that and then pass them in
Tobias "Tobi" Schraink
@tobsecret
Mar 27 01:47
ooooooooh
yeah, then the *.cram solution is your best friend
Joseph Guhlin
@jguhlin
Mar 27 01:49
Yeah, it's just a merge step, so not too concerned with this script. Better for the non-bioinformaticians to see "Stella" or "Stumpy" rather than an archaic sequencing run name, haha. But unintended consequences of some having spaces in their names
micans
@micans
Mar 27 10:14
spaces in names .. I have some sympathy that it is a natural way of thinking, but it leads to sooooooo much pain. Perhaps in plan 9 it is all plain sailing with spaces.
Jonathan Manning
@pinin4fjords
Mar 27 10:30
Morning all! I have an initialisation process I need to happen before a downstream step, every time a workflow runs (so I'm disabling cache). But it doesn't impact on results of downstream processes, just how the processes are run, so I don't want to set and input/output link that would invalidate downstream caches. How do I make sure that init process runs before the other processes, without that explicit dependency?
Jonathan Manning
@pinin4fjords
Mar 27 11:16
YES. Thank you :-)
Carl Witt
@carlwitt
Mar 27 12:48
How can I access the task.attempt variable outside a directive? I'd like to either access the value of the memory directive within the script. If not possible, I'd like to recompute the value.
process metaspades {
    memory { 2.GB * 2 ** task.attempt } // this works
    int some_num = task.attempt * 3 // this doesn't
    """
    ${memory} // this also doesn't work
    ${some_num} // this would work if the definition didn't fail
    """
}
Maxime Garcia
@MaxUlysse
Mar 27 12:54
you should be able to access task.memory
Carl Witt
@carlwitt
Mar 27 12:54
Right! Thanks. Now I just have to transform it to an int. Thanks!
Maxime Garcia
@MaxUlysse
Mar 27 12:55
if you need you can use task.memory.toGiga() too
you can also access task.cpus if needed
Carl Witt
@carlwitt
Mar 27 12:58
What is the type of the result of the expression task.memory.toGiga()?
Maxime Garcia
@MaxUlysse
Mar 27 12:58
Good question
Carl Witt
@carlwitt
Mar 27 12:59
Currently, metaspades.py -m ${task.memory} is substituted to metaspades.py -m 16 GB but I need to loose the suffix.
Maxime Garcia
@MaxUlysse
Mar 27 12:59
So use toGiga() you'll loose the suffix
We're using that to specify memory to JAVA with:
gatk --java-options -Xmx${task.memory.toGiga()}g -> gatk --java-options -Xmx16g
Carl Witt
@carlwitt
Mar 27 13:00
Awesome. Thanks!
Maxime Garcia
@MaxUlysse
Mar 27 13:01
Np
Tobias "Tobi" Schraink
@tobsecret
Mar 27 15:21
I'm trying to run the nf-core pipeline test case with singularity, but I'm getting this error that something is wrong with the config file
Launching `nf-core/deepvariant` [furious_majorana] - revision: 2b5486356c [master]
ERROR ~ Unknown config attribute: genomes_base -- check config file: /home/schrat01/.nextflow/assets/nf-core/deepvariant/nextflow.config

null

 -- Check '.nextflow.log' file for details
Paolo Di Tommaso
@pditommaso
Mar 27 15:26
nf-core slack channel is best place for this
Tobias "Tobi" Schraink
@tobsecret
Mar 27 18:02
thank you!
Laurence E. Bernstein
@lebernstein
Mar 27 19:05

So I have this problem (again because I worked around it the first time).. I want to parse a file (lets call it File1) to get the name of File2 and use File2 as an input to my process so that it is staged properly in the work dir. How do I make this happen? I have no problem getting the file name out with:

input:
  val File2 from file1.splitCsv()

but now how can I use File2 in my input section?
This sort of thing doesn't work

input:
  val File2 from file1.splitCsv()
  file input_file from file(File2)
Jonathan Manning
@pinin4fjords
Mar 27 19:17
Hi- how do I specify executor parameters like submitRateLimit in the directive declarations block for a process?
Jonathan Manning
@pinin4fjords
Mar 27 19:27
Do I need to define a new executor?
Laurence E. Bernstein
@lebernstein
Mar 27 21:06
Not an expert, but you want to have process-specific specifications for submitRateLimit parameter? If so, I think YES you need to define multiple executors and choose which one to use on a per process basis. Something like this:
process {
  withName: 'process1' {
    executor = 'type1'
  }
  withName: 'process2' {
    executor = 'type2'
  }
}
Jonathan Manning
@pinin4fjords
Mar 27 21:06
Thought it'd be something like that- thanks
Rad Suchecki
@rsuchecki
Mar 27 23:52
@lebernstein does the filename contained in File1 contain the full path?