These are chat archives for nextflow-io/nextflow

7th
May 2015
Paolo Di Tommaso
@pditommaso
May 07 2015 09:20
@andrewcstewart Yes, I'm supposing that the ones left by failed jobs.
In our cluster we use a cronjob to cleanup old pending containers
Jose Espinosa-Carrasco
@JoseEspinosa
May 07 2015 13:41
It works smoothly!!! :clap: :clap: :clap:
Paolo Di Tommaso
@pditommaso
May 07 2015 13:41
good
thanks for having found the bug
Jose Espinosa-Carrasco
@JoseEspinosa
May 07 2015 13:43
No worries,
Just one more question then it should be
.map { it.join() }

\\ Or

.map { it.join([]) }
It works both ways, so I guess is only a question of style, isn't?
Paolo Di Tommaso
@pditommaso
May 07 2015 13:44
umh, I would use it.join('')
the argument of join is the separator character
so it should be a string
Jose Espinosa-Carrasco
@JoseEspinosa
May 07 2015 13:45
The guru has spoken
Paolo Di Tommaso
@pditommaso
May 07 2015 13:45
:)
Andrew Stewart
@andrewcstewart
May 07 2015 20:38
@pditommaso do you use the MapR component of cirrus ?
Paolo Di Tommaso
@pditommaso
May 07 2015 20:39
no
Andrew Stewart
@andrewcstewart
May 07 2015 20:39
you still have to go through that registration step though right?
Paolo Di Tommaso
@pditommaso
May 07 2015 20:40
um, what registration step?
create
Prompts you for how many workers you want in your cluster. It first launches the master node, asks you to perform the free M3 MapR license, and then launches the worker nodes.
Paolo Di Tommaso
@pditommaso
May 07 2015 20:41
nono
this doc is obsolete
ignore it
Andrew Stewart
@andrewcstewart
May 07 2015 20:43
oh
Paolo Di Tommaso
@pditommaso
May 07 2015 20:43
there's a dashboard where you configure queues specifying ami, instance type, max instances, storage, etc
than they provides some command line tools to submits jobs to these queues
Andrew Stewart
@andrewcstewart
May 07 2015 20:44

So is the basic procedure then just

cirrus_workstation_cli create
cirrus_workstation_cli connect
# (login)
cirrus_cluster_cli create ...
nextflow run …

?

Paolo Di Tommaso
@pditommaso
May 07 2015 20:44
think something similar to qsub
Andrew Stewart
@andrewcstewart
May 07 2015 20:44
right
I get the submission part
im trying to figure out their general setup approach
their workstation demo talks about running NX and stuff like that
(which I guess is a nice feature but seems overkill)
Paolo Di Tommaso
@pditommaso
May 07 2015 20:46
I use a ec2 instance as master node, where I launch the nextflow pipeline
the setup it's almost zero
Andrew Stewart
@andrewcstewart
May 07 2015 20:46
you set that up yourself? or you use their workstation thing?
is it just kicking off an ec2 instance using their AMI?
Paolo Di Tommaso
@pditommaso
May 07 2015 20:48
you won't need a their ami as with StarCluster
it works with stocks ami or your custom ami
I've prepared a linux ami with Docker
Andrew Stewart
@andrewcstewart
May 07 2015 20:48
are you referring to the actual cluster or the standalone ec2 instance used for kicking off the cluster?
let me ask this way: what is step #1?
Paolo Di Tommaso
@pditommaso
May 07 2015 20:49
configure the queue parameters with their UI/server
Andrew Stewart
@andrewcstewart
May 07 2015 20:49
so wait.. you get an account on their website ?
Paolo Di Tommaso
@pditommaso
May 07 2015 20:50
yep
Andrew Stewart
@andrewcstewart
May 07 2015 20:50
does this cost money/
Paolo Di Tommaso
@pditommaso
May 07 2015 20:50
they provide an evaluation period, after that pay per use similar to amazon
Andrew Stewart
@andrewcstewart
May 07 2015 20:51
hm
so its not really open source then
Paolo Di Tommaso
@pditommaso
May 07 2015 20:51
consider, that product will become a service of AWS ..
(I'm just supposing that)
Andrew Stewart
@andrewcstewart
May 07 2015 20:52
I can see that happening. “Elastic Cluster Service” or something.
Paolo Di Tommaso
@pditommaso
May 07 2015 20:52
likely
Andrew Stewart
@andrewcstewart
May 07 2015 21:00
Did you have to ‘request a demo’ to get access?
Paolo Di Tommaso
@pditommaso
May 07 2015 21:00
yes
Andrew Stewart
@andrewcstewart
May 07 2015 21:01
I dont want their crappy demo I just want a URL :D
Paolo Di Tommaso
@pditommaso
May 07 2015 21:03
In my case we had a conf call, showing me product, then they gave me an evaluation account
Andrew Stewart
@andrewcstewart
May 07 2015 21:06
gotha
thanks for the info btw
Paolo Di Tommaso
@pditommaso
May 07 2015 21:06
welcome
Andrew Stewart
@andrewcstewart
May 07 2015 21:06
I just emailed them saying I dont want a demo I just want the eval :)
Paolo Di Tommaso
@pditommaso
May 07 2015 21:06
ahah
Andrew Stewart
@andrewcstewart
May 07 2015 23:02
Awesome, I can reach nextflow.io again
:D
IT thinks it may have had something to do with the site being sered from a different data center than the one we use for Direct Conenct