These are chat archives for nextflow-io/nextflow

14th
Apr 2017
Tim Diels
@timdiels
Apr 14 2017 00:12
Is there something like val x from channel.clone().map stuff to avoid having to name each fork of a channel? Should be equivalent to
channel = channel.tap { anonymous }
anonymous = anonymous.map stuff
process name {
    input:
    val x from anonymous
}
Tim Diels
@timdiels
Apr 14 2017 00:35
How do you combine each and set? Or what about each and file? E.g. something like each file fasta from fastas which injects a file, but also behaves like each in that non-each inputs are repeated for each each input. Or I could just use operators...
Tim Diels
@timdiels
Apr 14 2017 01:07
What's wrong with my when: block? https://pastebin.com/Les9PRPJ The script worked without it, but now it causes: https://pastebin.com/gzZGZVvd Thanks in advance
Paolo Di Tommaso
@pditommaso
Apr 14 2017 10:19
@amacbride Can you try to dump all the environment by using for example the env | sort command. Also are you running that task within a container ?
How do you combine each and set? Or what about each and file?
You can't. We are thinking to add the suppor for each file x from .. but surely not for set
To handle that you need to feed the process with a channel containing the combinations you need. See combine
What's wrong with my when: block?
put a script:before your command, that is mandatory when using when: eg.
when:
    fasta != databaseName

script:
    """
    echo $fasta $databaseName
    """
Tim Diels
@timdiels
Apr 14 2017 11:04
Thanks!
amacbride
@amacbride
Apr 14 2017 13:49
@pditommaso Sure, will do. Yes, it's being run via Singularity.
Paolo Di Tommaso
@pditommaso
Apr 14 2017 14:26
That's the problem
The extra
Oops
The external enclosure is not imported when using containers
*environment (fucking syntax checker..)
Tim Diels
@timdiels
Apr 14 2017 16:00
I'm trying to enumerate (like python) a channel, what am I doing wrong here? Error is line 12: https://pastebin.com/jhuA36DB
Mike Smoot
@mes5k
Apr 14 2017 16:04
@timdiels nextflow doesn't have a "map with index" operator, so you'll need to roll your own
Tim Diels
@timdiels
Apr 14 2017 18:07
And how do you add a new operator? I tried DataflowQueue.metaClass.enumerate = { ... } but apart from enumerating I'm not sure what to put in the closure
Paolo Di Tommaso
@pditommaso
Apr 14 2017 18:09
I've never tried the DataflowQueue.metaClass extension method for operators, not sure it will work
what is supposed to do enumerate ?
Tim Diels
@timdiels
Apr 14 2017 18:49
@pditommaso For example, Channel.from(5,8,3).enumerate {i, x -> [i, x]} .toList().view() should print [[0,5], [1,8], [2,3]]. I.e. it provides an index
Paolo Di Tommaso
@pditommaso
Apr 14 2017 18:50
you can do easily with a map
int index=0
Channel.from(5,8,3).map { it -> [index++, it] } .println()
Tim Diels
@timdiels
Apr 14 2017 18:52

What happens if this is repeated?

int index=0
Channel.from(5,8,3).map { it -> [index++, it] } .println()
int index=0
Channel.from(5,8,3).map { it -> [index++, it] } .println()

Don't both closures refer to the same index that way?

And thus interfere with each other
Paolo Di Tommaso
@pditommaso
Apr 14 2017 18:53
it could, you should two different index variable
or you can write an helper method eg.
function enumerate( channel ) {
  int index=0
  channel.map { it -> [index++, it] }
}
Tim Diels
@timdiels
Apr 14 2017 18:55
Could you tell me what I did wrong in my attempt at the same: https://pastebin.com/jhuA36DB ?
Paolo Di Tommaso
@pditommaso
Apr 14 2017 18:56
let me see
Paolo Di Tommaso
@pditommaso
Apr 14 2017 19:01
frankly I don't, I understand what your are trying to do but for some reason doesn't seem a valid syntax
Tim Diels
@timdiels
Apr 14 2017 19:02
Okay, I'll use your enumerate then, thanks
Paolo Di Tommaso
@pditommaso
Apr 14 2017 19:03
Im checking if it's possible to use the metaClass trick
yes!
groovyx.gpars.dataflow.DataflowQueue.metaClass.foo = {
  int index=0
  delegate.map { item -> [index++, item]  }
}

Channel.from(5,8,3).foo().println()
Tim Diels
@timdiels
Apr 14 2017 19:07
ah, perfect!
Paolo Di Tommaso
@pditommaso
Apr 14 2017 19:07
nice trick ;)
Tim Diels
@timdiels
Apr 14 2017 19:09
I came across it while reading the Groovy docs today :)
Paolo Di Tommaso
@pditommaso
Apr 14 2017 19:10
yes, it's allows to do many magic things
Tim Diels
@timdiels
Apr 14 2017 19:16

splitCsv doesn't like it though

groovyx.gpars.dataflow.DataflowQueue.metaClass.foo = {
  int index=0
  delegate.map { item -> [index++, item]  }
}
Channel
    .fromPath("$params.input/species.txt")
    .splitCsv(sep: "\t", header: true)
    .foo()

gives me

Launching `./main.nf` [high_roentgen] - revision: be313e16ba
ERROR ~ No signature of method: groovyx.gpars.dataflow.DataflowQueue.splitCsv() is applicable for argument types: () values: []
Possible solutions: split(java.util.List), split(groovy.lang.Closure), split(groovyx.gpars.dataflow.DataflowWriteChannel, groovyx.gpars.dataflow.DataflowWriteChannel), split(groovyx.gpars.group.PGroup, java.util.List), split(groovyx.gpars.scheduler.Pool, java.util.List), split(java.util.Map, java.util.List)
Paolo Di Tommaso
@pditommaso
Apr 14 2017 19:24
weird
Tim Diels
@timdiels
Apr 14 2017 19:58
I figured it out, I had to use groovyx.gpars.dataflow.DataflowReadChannel.metaClass.foo,
but how does splitCsv get tied into Channel? I don't find it in the code, all I see is a CsvSplitter class, does it find it through introspection or something?
Tim Diels
@timdiels
Apr 14 2017 20:15
I see, by meta class magic
Tim Diels
@timdiels
Apr 14 2017 20:25

Interesting, this

class A {
}
println A.metaClass
A.metaClass.a=1
println A.metaClass

outputs

org.codehaus.groovy.runtime.HandleMetaClass@5af3a0f[groovy.lang.MetaClassImpl@5af3a0f[class A]]
groovy.lang.ExpandoMetaClass@45c8d09f[class A]
A similar thing happens when setting foo on DataflowQueue.metaClass, NextflowDelegatingMetaClass gets replaced by ExpandoMetaClass and so, splitCsv is no longer found.