These are chat archives for nextflow-io/nextflow

12th
May 2016
Mike Smoot
@mes5k
May 12 2016 00:13
To answer my own question: adding .map{ f -> "'${f}'" } seems to do the trick.
Paolo Di Tommaso
@pditommaso
May 12 2016 12:44
@mes5k Right, alternatively you should be able to file_list.collect { f -> "'${f}'" }.join(' ') in the process script
Paolo Di Tommaso
@pditommaso
May 12 2016 12:55
@mfoll I've fixed the DAG issue with your pipeline. If you want to give a try update the snapshot with
CAPSULE_RESET=1 NXF_VER=0.19.0-SNAPSHOT nextflow info
then
NXF_VER=0.19.0-SNAPSHOT nextflow run ..
Matthieu Foll
@mfoll
May 12 2016 13:17
thanks, I’ll try
btw I was also interested in the question just above. I needed to create a command line from a list of files looking like -V file1 -V file2...
This is what I did:
files = Channel.fromPath( '*.log' ).toList()

process print_test {

   input:
   file files

   echo true

   shell:
   file_command = files.collect { f -> "'${f}'" }.join(' -V ')
   '''  
   echo "-V !{file_command}"
   '''
}
(that’s just a toy example)
Do you think it’s the right approach? (at least it’s working)
Paolo Di Tommaso
@pditommaso
May 12 2016 13:24
@mfoll but in this way you will lost the -V on the first file
it should be
Matthieu Foll
@mfoll
May 12 2016 13:24
yes but I put it after
Paolo Di Tommaso
@pditommaso
May 12 2016 13:24
file_command = files.collect { f -> "-V '${f}'" }.join(' ')
make sense?
Matthieu Foll
@mfoll
May 12 2016 13:25
ah yes
better
Paolo Di Tommaso
@pditommaso
May 12 2016 13:25
:+1:
Matthieu Foll
@mfoll
May 12 2016 13:59
test_dag.pdf
The DAG is working great!
It will go in the paper I am writing at the moment!
Paolo Di Tommaso
@pditommaso
May 12 2016 14:06
Great!
I hope you will mention you know what! :)
Matthieu Foll
@mfoll
May 12 2016 14:08
About that: for nextflow what is the rigth citation? The poster as written on github or your paper about docker and nextflow?
mariach
@mariach
May 12 2016 14:18
Hi Matthieu! We are now submitting the Nextflow paper. So the moment it gets accepted I will send you the right citation
When are you submitting your paper?
Matthieu Foll
@mfoll
May 12 2016 14:19
ok perfect
I hope by the end of the month
But I can add it during the review process
Tiffany Delhomme
@tdelhomme
May 12 2016 14:19
This message was deleted
mariach
@mariach
May 12 2016 14:19
Ok cool! Let's keep in touch :)
If it's absolutely needed we can also put the Nextflow paper in arXiv so that people can cite it
Matthieu Foll
@mfoll
May 12 2016 14:22
That would be good too actually
Paolo Di Tommaso
@pditommaso
May 12 2016 14:55
BTW dont forget you can even produce a SVG file and edit it to make pixel perfect
Jason Byars
@jbyars
May 12 2016 19:49
quick question about data fetching at the beginning of a flow. If I have a channel of files from a S3 bucket, on an sge cluster, is the fetch handled by the master node or distributed across the workers? Thanks!
Paolo Di Tommaso
@pditommaso
May 12 2016 19:50
No, it's handled on the master
this should be improved
Jason Byars
@jbyars
May 12 2016 19:51
ahh ok, then the behavior I'm seeing is rational
Mike Smoot
@mes5k
May 12 2016 20:43
Am I correct in thinking that channels are only "consume once"? Is there a way to declare a channel to be "consume many"? I know that .into effectively solves this, so I'm mostly just curious.
Paolo Di Tommaso
@pditommaso
May 12 2016 20:44
yes, channel links one producer to one consumer
with the exception of a value channel, which emits always the same value and can be consumed by different processes/operators
Mike Smoot
@mes5k
May 12 2016 20:48
Is that created with Channel.value()?
Paolo Di Tommaso
@pditommaso
May 12 2016 20:48
yep
Mike Smoot
@mes5k
May 12 2016 20:49
Is there a way for an output channel to be a value channel?
Paolo Di Tommaso
@pditommaso
May 12 2016 20:49
No at this time
Mike Smoot
@mes5k
May 12 2016 20:50
Ok, got it! It would be nice not to have to split everything with .into, but .into is unambiguous and makes for a pretty DAG. :smile:
Paolo Di Tommaso
@pditommaso
May 12 2016 20:51
yep :)
@mes5k is there a https URL for this script?
Mike Smoot
@mes5k
May 12 2016 21:25
Not sure, I'll ping the list
Paolo Di Tommaso
@pditommaso
May 12 2016 21:25
If it's in a GitHub repo it could be retrieved through rawgit
Mike Smoot
@mes5k
May 12 2016 21:27
This is the repo, but I'm not sure if the built, minified version is available: https://github.com/cytoscape/cytoscape.js
Paolo Di Tommaso
@pditommaso
May 12 2016 21:29
I see, ok let me know if you find something in the list
Paolo Di Tommaso
@pditommaso
May 12 2016 22:10
nope, redirect to http
it's not a big issue, but it would be better to have it as https otherwise when serving that graph page over https the browser will show an security warning
Mike Smoot
@mes5k
May 12 2016 22:12
I don't know how reliable the CDN urls are. Not really sure how reliable the Cytoscape urls are either, though.
Paolo Di Tommaso
@pditommaso
May 12 2016 22:14
:)
ok, they are identical
thanks
Jason Byars
@jbyars
May 12 2016 22:33
sge trivia? I'm getting Unable to run job: "job" denied: use parallel environments instead of requesting slots explicitly when I try to run something real. The only parameters I'm setting on the process are executor 'sge' and cpus 16. Any idea what causes this error?
Paolo Di Tommaso
@pditommaso
May 12 2016 22:34
SGE is a bit dumb regarding this
you need to specify the parallel environment, that is how you want to parallelise your task i.e. shared mem, mpi, etc
with nxf you will need to specify penv directive
Jason Byars
@jbyars
May 12 2016 22:39
Ahh, I didn't realize that wasn't optional. Let's see if smp works...
Paolo Di Tommaso
@pditommaso
May 12 2016 22:39
usually is that
Jason Byars
@jbyars
May 12 2016 22:49
looks like that was it... on to the next bug :smile:
now all we need is a command in the DSL to send you a pint
Jason Byars
@jbyars
May 12 2016 23:50
sge plugs in some silly small memory limit by default doesn't it?