These are chat archives for nextflow-io/nextflow

17th
Jul 2018
Tobias Neumann
@t-neumann
Jul 17 2018 07:46
@pditommaso Don't know if you read it: is there a way to selectively say which files go to the publishDir and which ones not? Because right now the output of any process goes to the globally set publishDir, but actually I would like only the files from a couple of processes there and have bigger intermediate files not moved there in the first place.
Paolo Di Tommaso
@pditommaso
Jul 17 2018 07:56
@aunderwo you can use a custom function to parse that file
read_correction_output.map { file-> myParseFunction(file.text) }
Anthony Underwood
@aunderwo
Jul 17 2018 08:27
@pditommaso Thanks. When I map what happen what happens to the output? Does it form another channel? I guess I would have to combine this with another channel to bring the parsed variable (a float) together with files from another channel
Paolo Di Tommaso
@pditommaso
Jul 17 2018 08:28
it creates another channel, but that it's transparent, you just need to use it in place of the original read_correction_output
Anthony Underwood
@aunderwo
Jul 17 2018 09:10
When I do that nextflow complains that it has already been used
def find_average_depth(pair_id, lighter_output){
  m = lighter_output =~  /.+Average coverage is ([0-9]+\.[0-9]+)\s.+/
  return new Tuple(pair_id, m[0][1])
}

read_correction_output.map { pair_id, file -> find_average_depth(pair_id, file.text) }
read_correction_output.subscribe{ println it}
Paolo Di Tommaso
@pditommaso
Jul 17 2018 09:10
tiple `
new line
code
Lukas Jelonek
@lukasjelonek
Jul 17 2018 09:14

Hi, I observe some weird behaviour when using variables and can't explain why when using variables in closures before the actual script template:

Channel.from(1,2,3,4).into{ch_1_1; ch_1_2}
Channel.from(1,1,1,1,1,1).into{ch_2_1; ch_2_2}

process works {

    input:
    val x from ch_1_1.collect()
    val y from ch_2_1.collect()

    script:
    x.each{
        int z = it
        println (z)
        int a =  y[z]
        println a
    }
    """echo done"""
}

process works_not {

    input:
    val x from ch_1_2.collect()
    val y from ch_2_2.collect()

    script:
    x.each{ z ->
        println (z)
        int a =  y[z]
        println a
    }
    """echo done"""

}

Do you have an explanation?

Paolo Di Tommaso
@pditommaso
Jul 17 2018 09:15
@aunderwo you have to use the new result of map ie.
foo = read_correction_output.map { pair_id, file -> find_average_depth(pair_id, file.text) }
foo.subscribe{ println it}
or
read_correction_output.map { pair_id, file -> find_average_depth(pair_id, file.text) }.set { foo }
foo.subscribe{ println it}
or
read_correction_output
   .map { pair_id, file -> find_average_depth(pair_id, file.text) }
   .subscribe{ println it}
@lukasjelonek what's supposed to be the weird behaviour ?
Lukas Jelonek
@lukasjelonek
Jul 17 2018 09:17
N E X T F L O W  ~  version 0.30.2
Launching `./main.nf` [silly_austin] - revision: 34c2c42f4d
ERROR ~ Variable `z` already defined in the process scope @ line 30, column 20.
           int a =  y[z]
                      ^

1 error
When I remove the println(z) line it works, otherwise there seems to be a compile or interpreter error
I am not sure if it's a bug or just my lack of knowledge of some cornercases that have to be handled differently
Anthony Underwood
@aunderwo
Jul 17 2018 09:22
@pditommaso - perfect thanks
Paolo Di Tommaso
@pditommaso
Jul 17 2018 09:23
@lukasjelonek yes, it looks a bug
Lukas Jelonek
@lukasjelonek
Jul 17 2018 09:24
Then I will create an issue for that
Paolo Di Tommaso
@pditommaso
Jul 17 2018 09:24
thank you
Lukas Jelonek
@lukasjelonek
Jul 17 2018 09:34
Thank you for nextflow :thumbsup:
Paolo Di Tommaso
@pditommaso
Jul 17 2018 09:38
thank you for using it
and by the way hope to see you here
Lukas Jelonek
@lukasjelonek
Jul 17 2018 11:00
I'll check if I can make it
Paolo Di Tommaso
@pditommaso
Jul 17 2018 12:28
:+1:
Paolo Di Tommaso
@pditommaso
Jul 17 2018 12:42
if you are interested to NF on Azure put a thumb up at this feature request
Azure/azure-storage-java#305
Félix C. Morency
@fmorency
Jul 17 2018 14:27
Is it possible to have multiple blocks in the script:section? Something like
script:
"""
something_in_bash.sh
"""
"""
#!/usr/bin/env python
import gravity

something_in_python()
"""
"""
something_in_bash_again.sh
"""
My .command.sh only seems to contain the last block
Paolo Di Tommaso
@pditommaso
Jul 17 2018 15:20
put a + between them
aahhh
no
they must be the same lang
Félix C. Morency
@fmorency
Jul 17 2018 15:25
Oh ok. Thanks :)
Tim Dudgeon
@tdudgeon
Jul 17 2018 15:26
When using the Ignite executor is it possible to specify not to execute tasks on the head node from where the initial nextflow job is started?
Paolo Di Tommaso
@pditommaso
Jul 17 2018 15:29
-cluster.master.maxCpus 0
Tim Dudgeon
@tdudgeon
Jul 17 2018 15:32
thanks
Paolo Di Tommaso
@pditommaso
Jul 17 2018 15:33
:+1: