These are chat archives for nextflow-io/nextflow

13th
Mar 2017
Mike Smoot
@mes5k
Mar 13 2017 16:31
Hi @pditommaso can you share what your plans for running Nextflow on Google infrastructure are? I'm also curious what your opinions of Cloud Dataflow and Apache Beam are? My sense is that Beam would probably be ok if you were living in a Java world and writing a bunch of new Java code, but it's unclear how well it would integrate heterogenous tools and environments.
Paolo Di Tommaso
@pditommaso
Mar 13 2017 16:34
Hi Mike, if I'm not right Beam is the open source core of Google Dataflow
we have no plan to support them, instead the idea is to implement a model similar to Amazon to support GCP
in my opinion Apache Spark eat everything in the area of data analytics platforms
Mike Smoot
@mes5k
Mar 13 2017 16:38
And I've been wondering how Nextflow and Spark might eventually play together.
Paolo Di Tommaso
@pditommaso
Mar 13 2017 16:41
that's on the radar, though very difficult to give you specific details
Mike Smoot
@mes5k
Mar 13 2017 16:56
I don't know enough about Spark to have much insight. I haven't pursued it for our work because so much of our toolchain exists outside of that environment. I'm going to make an effort to play with spark a bit...
Paolo Di Tommaso
@pditommaso
Mar 13 2017 16:57
it's surely an interesting platform
but with little adoption in genomics
the only tools that I know are ADAM and Hail
Mike Smoot
@mes5k
Mar 13 2017 17:03
Lots of opportunity for enterprising graduate students!
Paolo Di Tommaso
@pditommaso
Mar 13 2017 17:03
yes !
:)
Michael L Heuer
@heuermh
Mar 13 2017 17:37
@mes5k meet us on https://gitter.im/bigdatagenomics/adam to discuss further :)
Mike Smoot
@mes5k
Mar 13 2017 17:39
Thanks Michael, I'm going to experiment a bit and see which of our many nails this might be a good hammer for!
Michael L Heuer
@heuermh
Mar 13 2017 17:44
It's been a while since I tried, but this might do something
$ ./nextflow run https://github.com/heuermh/bdg-nextflow -with-docker heuermh/adam