These are chat archives for nextflow-io/nextflow

11th
Jan 2018
Maxime Garcia
@MaxUlysse
Jan 11 2018 08:20
@heuermh I'm directly importing my docker images into singularity, I haven't looked into anything like that
sorry
Phil Ewels
@ewels
Jan 11 2018 12:18
@skptic - nice! That will simplify things..
@pditommaso / others - regarding multiple input files in channels (https://www.nextflow.io/docs/latest/process.html#multiple-input-files).. Is there a way to maintain original input files but add in a cardinality number earlier in the path?
eg. dir/??/* -> dir/01/foo.txt, dir/01/bar.txt, dir/02/foo.txt, dir/02/bar.txt
etc. Numbers themselves not important - I just want to avoid having duplicate filenames (and so overwriting stuff) whilst also keeping the original filenames (so that MultiQC recognises the files)
Can't have the MultiQC pattern in the output channel directly as the channel has a bunch of different file types with different filenames that match different patterns in MultiQC
Paolo Di Tommaso
@pditommaso
Jan 11 2018 12:22
umm, nice idea
Phil Ewels
@ewels
Jan 11 2018 12:22
If not - is there an alternative to .collect() that we can use to do this somehow? This is the line that I'm playing with:
file ('rseqc/*') from rseqc_results.collect()
Paolo Di Tommaso
@pditommaso
Jan 11 2018 12:22
It should not work, but it should be possible to implement
would you like to open a feature request for that?
Phil Ewels
@ewels
Jan 11 2018 12:23
can do :+1:
in mean time, is it possible to use .join() or something else instead?
Paolo Di Tommaso
@pditommaso
Jan 11 2018 12:23
I'm thinking at that ..
you should write a custom function that takes the list of files and resolves the conflicts
.collect().map { listOfFiles -> massage(listOfFiles) }
Phil Ewels
@ewels
Jan 11 2018 12:27
nextflow-io/nextflow#568
Paolo Di Tommaso
@pditommaso
Jan 11 2018 12:27
:+1:
Paolo Di Tommaso
@pditommaso
Jan 11 2018 13:59
hey @MaxUlysse, thanks for sharing your notes
are you planning to use CAW with Batch for production deployments ?
Maxime Garcia
@MaxUlysse
Jan 11 2018 14:09
I'm just exploring the possibilities ;-)
I need to try with a better test data, so that we can see how much it would cost, but we still need to wait for Amazon to have a data center in Sweden
Paolo Di Tommaso
@pditommaso
Jan 11 2018 14:15
I see, nice
Alexander Peltzer
@apeltzer
Jan 11 2018 14:16
@MaxUlysse Just out of curiousity: Is it the same for Sweden that you have to keep data inside the country for legal issues (thats the situation here in Germany at least)...?
Phil Ewels
@ewels
Jan 11 2018 14:20
Yup
Paolo Di Tommaso
@pditommaso
Jan 11 2018 14:34
@/all we are starting to think about Nextflow Hack/workshop 2018, just wondering if there's any US based organisation who may be interested to host the event ?
Alexander Peltzer
@apeltzer
Jan 11 2018 15:04
I'll ask around
Paolo Di Tommaso
@pditommaso
Jan 11 2018 15:04
:+1:
Alexander Peltzer
@apeltzer
Jan 11 2018 15:05
Is there a way btw to run nextflow pull user/repository without a main.nf file, e.g.: nextflow pull apeltzer/NGI-ExoSeq ?
the pull does complain about a missing main.nf or nextflow.config(the latter is present though)
Paolo Di Tommaso
@pditommaso
Jan 11 2018 15:06
there's should be at least main.nf or a nextflow.config file
ah
but in the config you need yo declared the mainScriptfile name
Alexander Peltzer
@apeltzer
Jan 11 2018 15:07
aaah! ok
Sorry, thats the point ;-) Still learning here ;-)
Paolo Di Tommaso
@pditommaso
Jan 11 2018 15:10
no problem
Félix C. Morency
@fmorency
Jan 11 2018 15:59
Not in the US, but close to the border :)
Paolo Di Tommaso
@pditommaso
Jan 11 2018 16:21
nice
Félix C. Morency
@fmorency
Jan 11 2018 16:36
Sherbrooke is a ~45min car ride to the US border
And a 1h45 car ride from Montreal
Paolo Di Tommaso
@pditommaso
Jan 11 2018 16:43
I see, that could be an option
Félix C. Morency
@fmorency
Jan 11 2018 16:43
Let me know if you're interested
Paolo Di Tommaso
@pditommaso
Jan 11 2018 16:44
do you have an auditorium/meeting room for ~ 30 people ?
Félix C. Morency
@fmorency
Jan 11 2018 16:44
Sure
Paolo Di Tommaso
@pditommaso
Jan 11 2018 16:44
nice
ok, still very hypothetical, we are starting to discuss these days
I will take in consideration as an option
Félix C. Morency
@fmorency
Jan 11 2018 16:45
Let me know if you need any info
Paolo Di Tommaso
@pditommaso
Jan 11 2018 16:46
ok, thanks for now
Félix C. Morency
@fmorency
Jan 11 2018 18:47

Is there a way to get the combinations on a channel? For example, in Python, one can do

print list(itertools.combinations([1, 2, 3], 2))
[(1, 2), (1, 3), (2, 3)]

combine() gives the cartesian product which is not exactly what I'm looking for

Paolo Di Tommaso
@pditommaso
Jan 11 2018 19:10
similary you can create a combinations of list items and then transform to a channel eg
GroovyCollections.combinations([[1,2,3], ['a','b']]).channel().println()
or
[[1,2,3], ['a','b']].combinations().channel().println()
Félix C. Morency
@fmorency
Jan 11 2018 19:12
Didn't know about the .channel() method
Is it mentioned somewhere in the doc?
Paolo Di Tommaso
@pditommaso
Jan 11 2018 19:16
actually no ..
but it's just shortcut for Channel.from
Félix C. Morency
@fmorency
Jan 11 2018 19:18
Right. Thanks
Paolo Di Tommaso
@pditommaso
Jan 11 2018 19:18
:+1: