These are chat archives for nextflow-io/nextflow

14th
Mar 2018
Vladimir Kiselev
@wikiselev
Mar 14 2018 07:05
@mes5k , great, many thanks, I think this is what I was looking for. I want to add a unique prefix to every file in a channel.
Vladimir Kiselev
@wikiselev
Mar 14 2018 07:21
@mes5k where should I put your code above? Does it have to be outside of a process or inside? And also how do I access the ind? Shall I use set val(ind), file(cram) from your_channel? I’ve tried several things in the nextflow console but couldn’t figure this out...
Vladimir Kiselev
@wikiselev
Mar 14 2018 07:31
So, for example in this example the channel manipulation worked well but the process test did not work…
ch1 = Channel
     .fromPath('test/*.txt')
def ind = 0
ch2 = ch1.map{ [ind++, it] }
process test {
    input:
        set val(inds), file(f) from ch2
    """
    echo test > $inds.txt
    ""”
}
Paolo Di Tommaso
@pditommaso
Mar 14 2018 08:28
I guess the problem is $inds.txt
this is interpreted as attribute txt on object inds, use `${inds}.txt instead
Vladimir Kiselev
@wikiselev
Mar 14 2018 09:13
@pditommaso great, many thanks, all worked!
do you also reply to Gitter even when giving a talk? ;-)
Paolo Di Tommaso
@pditommaso
Mar 14 2018 09:21
there's always a hole to give a look at the gitter chat ;)
Tiffany Delhomme
@tdelhomme
Mar 14 2018 10:49
Hi all, does anyone know if there is a way to construct a channel from a text file which actually contains a list of input file paths?
Vladimir Kiselev
@wikiselev
Mar 14 2018 11:44

This is what I asked recently about. Here is what worked for me, assuming all your paths are on a separate line in the paths.txt:

paths_list = Channel.fromPath(‘paths.txt’)
process get_paths {
    input: 
        val path from paths_list.flatMap{ it.readLines() }
    “””
    echo ${path}
    “”"
}

hope it works

Paolo Di Tommaso
@pditommaso
Mar 14 2018 11:53
that's not enough, you need to map the string path to a file path
input: 
        file path from paths_list.flatMap{ it.readLines() }.map { file(it) }
something we need to improve at some point
Vladimir Kiselev
@wikiselev
Mar 14 2018 12:52
Oops, sorry, in my case it was a sample id, so I didn’t need to make a file path, thanks @pditommaso !
Alexander Peltzer
@apeltzer
Mar 14 2018 12:56
Hes just giving his talk right now ;-)
Maxime Garcia
@MaxUlysse
Mar 14 2018 13:04
I'm still surprised he didn't already answered
I was suspecting he was actually multiple people, I think it's proof that there's only one Paolo
Alexander Peltzer
@apeltzer
Mar 14 2018 13:05
Hes back in 2 secs I guess :-D
Edgar
@edgano
Mar 14 2018 13:11
@MaxUlysse we will never know... if there is just one paolo or multiple....
Paolo Di Tommaso
@pditommaso
Mar 14 2018 13:25
I've arrived too late to the party :joy:
Tiffany Delhomme
@tdelhomme
Mar 14 2018 13:47
@wikiselev @pditommaso thanks for the tips! will try this.
Vladimir Kiselev
@wikiselev
Mar 14 2018 13:48
@apeltzer @MaxUlysse @pditommaso Paolo, too bad, too bad...
Clément ZOTTI
@czotti
Mar 14 2018 15:25

Hello everyone, I have an issue regarding index generation.
I want to generate indexes for a channel of arbitrary size. For now I have something like that, and it didn't work.

Channel
    .from(1..100)
    .set{ counter }

counter
    .count()
    .set{ length }

length
    .map { it -> Channel.from(1..it)}
    .set{ indexes }

indexes
    .view()
    .toList()
    .view()

With this, the nexflow run never stop, it's idling forever after the toList. What did I do wrong or misunderstood ?

Mike Smoot
@mes5k
Mar 14 2018 15:31
Sooo, you're making a channel containing another channel and then trying to coerce the inner channel into a list? I'm really not understanding what your end goal is here.
Clément ZOTTI
@czotti
Mar 14 2018 15:36
My end gloal is to randomShuffle these indexes Channel.from(1..it).randomShuffle(it, 1337), counter is a list of directory and I want to use these indexes to distribute each directory into a specific channel for a specific processing
Mike Smoot
@mes5k
Mar 14 2018 15:41
Can you show an mock up of what you expect the final indexes.view() to look like?
Clément ZOTTI
@czotti
Mar 14 2018 15:42
indexes.view() with the randomShuffle, I expect something like this:
1
3
7
24
...
98
8
Mike Smoot
@mes5k
Mar 14 2018 15:50
Here's shuffling
def inds = [1,2,3,4,5]
Collections.shuffle(inds)
Channel.from( inds ).view()
Clément ZOTTI
@czotti
Mar 14 2018 15:51
Thanks I'll check it.
meredithslota
@meredithslota
Mar 14 2018 16:33
What's the status of the CWL --> Nextflow script conversion project?
We are looking at using Nextflow and that CWL support is critical to being able to share our workflows with others.
Luca Cozzuto
@lucacozzuto
Mar 14 2018 17:38
@meredithslota May I ask why without CWL you cannot share your workflow?
meredithslota
@meredithslota
Mar 14 2018 17:52
We are working to make workflows more portable, and the groups we are collaborating with are trying to align to a common standard.
The goal is to make them more interoperable.
So ideally we'd share them as CWL workflows but users working on Nextflow wouldn't be left out in the cold. :)
Paolo Di Tommaso
@pditommaso
Mar 14 2018 20:52
@meredithslota Nextflow users are never left in the cold :) kidding apart which is the target platform of your collaborators ?
Michael L Heuer
@heuermh
Mar 14 2018 20:57
Making "workflows more portable" by choosing lowest common denominator is not always the best way to go.
Paolo Di Tommaso
@pditommaso
Mar 14 2018 20:57
I tend to agree
Mike Smoot
@mes5k
Mar 14 2018 21:02
Am I right in thinking that there are basic incompatibilities between Nextflow and CWL and likewise basic incompatibilities between WDL and CWL?
Paolo Di Tommaso
@pditommaso
Mar 14 2018 21:08
in principle the conversion from CWL to nextflow should be possible, but we realised that's to complex to make it sustainable, so I don't think there would be any evolution on that side
more recently I was investigating the import of CWL task into NF scripts nextflow-io/nextflow#545
that could make sense in some context
Mike Smoot
@mes5k
Mar 14 2018 21:19
Would that be a single CWL task run as a process in nextflow?
Paolo Di Tommaso
@pditommaso
Mar 14 2018 21:19
yep
Mike Smoot
@mes5k
Mar 14 2018 21:20
interesting. I like the idea of cross compatibility.
Paolo Di Tommaso
@pditommaso
Mar 14 2018 21:21
that could be useful if a some point there's a consistent catalog of tools wrapper in CWL
Mike Smoot
@mes5k
Mar 14 2018 21:21
exactly what I was thinking. All linked together with elegant nextflow code. :)
Michael L Heuer
@heuermh
Mar 14 2018 21:26
kinda off topic, I know there are various tools for generating tool descriptions from other tool descriptions, which is the most upstream?
Paolo Di Tommaso
@pditommaso
Mar 14 2018 21:29
I start from the assumption that tool descriptors are just evil :satisfied:
Paolo Di Tommaso
@pditommaso
Mar 14 2018 22:29
@czotti have you tried randomSample ?