These are chat archives for thunder-project/thunder

14th
Jul 2015
Krisztián Szűcs
@kszucs
Jul 14 2015 13:43
@freeman-lab Hey! Do You have any news about array abstraction over spark?
Jeremy Freeman
@freeman-lab
Jul 14 2015 18:27
@kszucs yes! we've just opened up the project, you can check out the repo (https://github.com/bolt-project/bolt) and the documentation page (http://bolt-project.org/)
and i'll add you to the gitter
Jeremy Freeman
@freeman-lab
Jul 14 2015 22:11
@naory re: the ipython question (sorry for the delay!), there are a couple options here, but the simplest is as follows
if you're already able to start Spark + IPython, you can just start Thunder from the existing SparkContext
from thunder import ThunderContext
tsc = ThunderContext(sc)
and then send its egg file across the cluster
from thunder.utils.launch import findThunderEgg
tsc.addPyFile(findThunderEgg())
that second step makes it unnecessary to pip install thunder on all the workers