These are chat archives for nextflow-io/nextflow

11th
Jan 2018
Maxime Garcia
@MaxUlysse
Jan 11 2018 08:20 UTC
@heuermh I'm directly importing my docker images into singularity, I haven't looked into anything like that
sorry
Phil Ewels
@ewels
Jan 11 2018 12:18 UTC
@skptic - nice! That will simplify things..
@pditommaso / others - regarding multiple input files in channels (https://www.nextflow.io/docs/latest/process.html#multiple-input-files).. Is there a way to maintain original input files but add in a cardinality number earlier in the path?
eg. dir/??/* -> dir/01/foo.txt, dir/01/bar.txt, dir/02/foo.txt, dir/02/bar.txt
etc. Numbers themselves not important - I just want to avoid having duplicate filenames (and so overwriting stuff) whilst also keeping the original filenames (so that MultiQC recognises the files)
Can't have the MultiQC pattern in the output channel directly as the channel has a bunch of different file types with different filenames that match different patterns in MultiQC
Paolo Di Tommaso
@pditommaso
Jan 11 2018 12:22 UTC
umm, nice idea
Phil Ewels
@ewels
Jan 11 2018 12:22 UTC
If not - is there an alternative to .collect() that we can use to do this somehow? This is the line that I'm playing with:
file ('rseqc/*') from rseqc_results.collect()
Paolo Di Tommaso
@pditommaso
Jan 11 2018 12:22 UTC
It should not work, but it should be possible to implement
would you like to open a feature request for that?
Phil Ewels
@ewels
Jan 11 2018 12:23 UTC
can do :+1:
in mean time, is it possible to use .join() or something else instead?
Paolo Di Tommaso
@pditommaso
Jan 11 2018 12:23 UTC
I'm thinking at that ..
you should write a custom function that takes the list of files and resolves the conflicts
.collect().map { listOfFiles -> massage(listOfFiles) }
Phil Ewels
@ewels
Jan 11 2018 12:27 UTC
nextflow-io/nextflow#568
Paolo Di Tommaso
@pditommaso
Jan 11 2018 12:27 UTC
:+1:
Paolo Di Tommaso
@pditommaso
Jan 11 2018 13:59 UTC
hey @MaxUlysse, thanks for sharing your notes
are you planning to use CAW with Batch for production deployments ?
Maxime Garcia
@MaxUlysse
Jan 11 2018 14:09 UTC
I'm just exploring the possibilities ;-)
I need to try with a better test data, so that we can see how much it would cost, but we still need to wait for Amazon to have a data center in Sweden
Paolo Di Tommaso
@pditommaso
Jan 11 2018 14:15 UTC
I see, nice
Alexander Peltzer
@apeltzer
Jan 11 2018 14:16 UTC
@MaxUlysse Just out of curiousity: Is it the same for Sweden that you have to keep data inside the country for legal issues (thats the situation here in Germany at least)...?
Phil Ewels
@ewels
Jan 11 2018 14:20 UTC
Yup
Paolo Di Tommaso
@pditommaso
Jan 11 2018 14:34 UTC
@/all we are starting to think about Nextflow Hack/workshop 2018, just wondering if there's any US based organisation who may be interested to host the event ?
Alexander Peltzer
@apeltzer
Jan 11 2018 15:04 UTC
I'll ask around
Paolo Di Tommaso
@pditommaso
Jan 11 2018 15:04 UTC
:+1:
Alexander Peltzer
@apeltzer
Jan 11 2018 15:05 UTC
Is there a way btw to run nextflow pull user/repository without a main.nf file, e.g.: nextflow pull apeltzer/NGI-ExoSeq ?
the pull does complain about a missing main.nf or nextflow.config(the latter is present though)
Paolo Di Tommaso
@pditommaso
Jan 11 2018 15:06 UTC
there's should be at least main.nf or a nextflow.config file
ah
but in the config you need yo declared the mainScriptfile name
Alexander Peltzer
@apeltzer
Jan 11 2018 15:07 UTC
aaah! ok
Sorry, thats the point ;-) Still learning here ;-)
Paolo Di Tommaso
@pditommaso
Jan 11 2018 15:10 UTC
no problem
Félix C. Morency
@fmorency
Jan 11 2018 15:59 UTC
Not in the US, but close to the border :)
Paolo Di Tommaso
@pditommaso
Jan 11 2018 16:21 UTC
nice
Félix C. Morency
@fmorency
Jan 11 2018 16:36 UTC
Sherbrooke is a ~45min car ride to the US border
And a 1h45 car ride from Montreal
Paolo Di Tommaso
@pditommaso
Jan 11 2018 16:43 UTC
I see, that could be an option
Félix C. Morency
@fmorency
Jan 11 2018 16:43 UTC
Let me know if you're interested
Paolo Di Tommaso
@pditommaso
Jan 11 2018 16:44 UTC
do you have an auditorium/meeting room for ~ 30 people ?
Félix C. Morency
@fmorency
Jan 11 2018 16:44 UTC
Sure
Paolo Di Tommaso
@pditommaso
Jan 11 2018 16:44 UTC
nice
ok, still very hypothetical, we are starting to discuss these days
I will take in consideration as an option
Félix C. Morency
@fmorency
Jan 11 2018 16:45 UTC
Let me know if you need any info
Paolo Di Tommaso
@pditommaso
Jan 11 2018 16:46 UTC
ok, thanks for now
Félix C. Morency
@fmorency
Jan 11 2018 18:47 UTC

Is there a way to get the combinations on a channel? For example, in Python, one can do

print list(itertools.combinations([1, 2, 3], 2))
[(1, 2), (1, 3), (2, 3)]

combine() gives the cartesian product which is not exactly what I'm looking for

Paolo Di Tommaso
@pditommaso
Jan 11 2018 19:10 UTC
similary you can create a combinations of list items and then transform to a channel eg
GroovyCollections.combinations([[1,2,3], ['a','b']]).channel().println()
or
[[1,2,3], ['a','b']].combinations().channel().println()
Félix C. Morency
@fmorency
Jan 11 2018 19:12 UTC
Didn't know about the .channel() method
Is it mentioned somewhere in the doc?
Paolo Di Tommaso
@pditommaso
Jan 11 2018 19:16 UTC
actually no ..
but it's just shortcut for Channel.from
Félix C. Morency
@fmorency
Jan 11 2018 19:18 UTC
Right. Thanks
Paolo Di Tommaso
@pditommaso
Jan 11 2018 19:18 UTC
:+1: