These are chat archives for nextflow-io/nextflow

12th
May 2016
Mike Smoot
@mes5k
May 12 2016 00:13 UTC
To answer my own question: adding .map{ f -> "'${f}'" } seems to do the trick.
Paolo Di Tommaso
@pditommaso
May 12 2016 12:44 UTC
@mes5k Right, alternatively you should be able to file_list.collect { f -> "'${f}'" }.join(' ') in the process script
Paolo Di Tommaso
@pditommaso
May 12 2016 12:55 UTC
@mfoll I've fixed the DAG issue with your pipeline. If you want to give a try update the snapshot with
CAPSULE_RESET=1 NXF_VER=0.19.0-SNAPSHOT nextflow info
then
NXF_VER=0.19.0-SNAPSHOT nextflow run ..
Matthieu Foll
@mfoll
May 12 2016 13:17 UTC
thanks, I’ll try
btw I was also interested in the question just above. I needed to create a command line from a list of files looking like -V file1 -V file2...
This is what I did:
files = Channel.fromPath( '*.log' ).toList()

process print_test {

   input:
   file files

   echo true

   shell:
   file_command = files.collect { f -> "'${f}'" }.join(' -V ')
   '''  
   echo "-V !{file_command}"
   '''
}
(that’s just a toy example)
Do you think it’s the right approach? (at least it’s working)
Paolo Di Tommaso
@pditommaso
May 12 2016 13:24 UTC
@mfoll but in this way you will lost the -V on the first file
it should be
Matthieu Foll
@mfoll
May 12 2016 13:24 UTC
yes but I put it after
Paolo Di Tommaso
@pditommaso
May 12 2016 13:24 UTC
file_command = files.collect { f -> "-V '${f}'" }.join(' ')
make sense?
Matthieu Foll
@mfoll
May 12 2016 13:25 UTC
ah yes
better
Paolo Di Tommaso
@pditommaso
May 12 2016 13:25 UTC
:+1:
Matthieu Foll
@mfoll
May 12 2016 13:59 UTC
test_dag.pdf
The DAG is working great!
It will go in the paper I am writing at the moment!
Paolo Di Tommaso
@pditommaso
May 12 2016 14:06 UTC
Great!
I hope you will mention you know what! :)
Matthieu Foll
@mfoll
May 12 2016 14:08 UTC
About that: for nextflow what is the rigth citation? The poster as written on github or your paper about docker and nextflow?
mariach
@mariach
May 12 2016 14:18 UTC
Hi Matthieu! We are now submitting the Nextflow paper. So the moment it gets accepted I will send you the right citation
When are you submitting your paper?
Matthieu Foll
@mfoll
May 12 2016 14:19 UTC
ok perfect
I hope by the end of the month
But I can add it during the review process
Tiffany Delhomme
@tdelhomme
May 12 2016 14:19 UTC
This message was deleted
mariach
@mariach
May 12 2016 14:19 UTC
Ok cool! Let's keep in touch :)
If it's absolutely needed we can also put the Nextflow paper in arXiv so that people can cite it
Matthieu Foll
@mfoll
May 12 2016 14:22 UTC
That would be good too actually
Paolo Di Tommaso
@pditommaso
May 12 2016 14:55 UTC
BTW dont forget you can even produce a SVG file and edit it to make pixel perfect
Jason Byars
@jbyars
May 12 2016 19:49 UTC
quick question about data fetching at the beginning of a flow. If I have a channel of files from a S3 bucket, on an sge cluster, is the fetch handled by the master node or distributed across the workers? Thanks!
Paolo Di Tommaso
@pditommaso
May 12 2016 19:50 UTC
No, it's handled on the master
this should be improved
Jason Byars
@jbyars
May 12 2016 19:51 UTC
ahh ok, then the behavior I'm seeing is rational
Mike Smoot
@mes5k
May 12 2016 20:43 UTC
Am I correct in thinking that channels are only "consume once"? Is there a way to declare a channel to be "consume many"? I know that .into effectively solves this, so I'm mostly just curious.
Paolo Di Tommaso
@pditommaso
May 12 2016 20:44 UTC
yes, channel links one producer to one consumer
with the exception of a value channel, which emits always the same value and can be consumed by different processes/operators
Mike Smoot
@mes5k
May 12 2016 20:48 UTC
Is that created with Channel.value()?
Paolo Di Tommaso
@pditommaso
May 12 2016 20:48 UTC
yep
Mike Smoot
@mes5k
May 12 2016 20:49 UTC
Is there a way for an output channel to be a value channel?
Paolo Di Tommaso
@pditommaso
May 12 2016 20:49 UTC
No at this time
Mike Smoot
@mes5k
May 12 2016 20:50 UTC
Ok, got it! It would be nice not to have to split everything with .into, but .into is unambiguous and makes for a pretty DAG. :smile:
Paolo Di Tommaso
@pditommaso
May 12 2016 20:51 UTC
yep :)
@mes5k is there a https URL for this script?
Mike Smoot
@mes5k
May 12 2016 21:25 UTC
Not sure, I'll ping the list
Paolo Di Tommaso
@pditommaso
May 12 2016 21:25 UTC
If it's in a GitHub repo it could be retrieved through rawgit
Mike Smoot
@mes5k
May 12 2016 21:27 UTC
This is the repo, but I'm not sure if the built, minified version is available: https://github.com/cytoscape/cytoscape.js
Paolo Di Tommaso
@pditommaso
May 12 2016 21:29 UTC
I see, ok let me know if you find something in the list
Paolo Di Tommaso
@pditommaso
May 12 2016 22:10 UTC
nope, redirect to http
it's not a big issue, but it would be better to have it as https otherwise when serving that graph page over https the browser will show an security warning
Mike Smoot
@mes5k
May 12 2016 22:12 UTC
I don't know how reliable the CDN urls are. Not really sure how reliable the Cytoscape urls are either, though.
Paolo Di Tommaso
@pditommaso
May 12 2016 22:14 UTC
:)
ok, they are identical
thanks
Jason Byars
@jbyars
May 12 2016 22:33 UTC
sge trivia? I'm getting Unable to run job: "job" denied: use parallel environments instead of requesting slots explicitly when I try to run something real. The only parameters I'm setting on the process are executor 'sge' and cpus 16. Any idea what causes this error?
Paolo Di Tommaso
@pditommaso
May 12 2016 22:34 UTC
SGE is a bit dumb regarding this
you need to specify the parallel environment, that is how you want to parallelise your task i.e. shared mem, mpi, etc
with nxf you will need to specify penv directive
Jason Byars
@jbyars
May 12 2016 22:39 UTC
Ahh, I didn't realize that wasn't optional. Let's see if smp works...
Paolo Di Tommaso
@pditommaso
May 12 2016 22:39 UTC
usually is that
Jason Byars
@jbyars
May 12 2016 22:49 UTC
looks like that was it... on to the next bug :smile:
now all we need is a command in the DSL to send you a pint
Jason Byars
@jbyars
May 12 2016 23:50 UTC
sge plugs in some silly small memory limit by default doesn't it?