These are chat archives for thunder-project/thunder

23rd
Jul 2015
Jeremy Freeman
@freeman-lab
Jul 23 2015 00:21
ah! there ya go
shariqiqbal2810
@shariqiqbal2810
Jul 23 2015 13:51
Hi! I'm working on getting Thunder up an running on an EC2 cluster and I've been running into some issues. Namely, the zope.cachedescriptors module loaded in 'thunder/extraction/source.py' seems to be unavailable on the slaves (I can import it on the master), so when I call certain things like TimeSeries.dims or try to run PCA, I get this error:
ImportError: No module named zope.cachedescriptors
Any help/suggestions would be appreciated :)
shariqiqbal2810
@shariqiqbal2810
Jul 23 2015 15:17
Nevermind I resolved the issue! Looks like my ec2.py script was not installing zope.cachedescriptors to the slaves. For some reason the version I installed with pip did not include that line.
Jeremy Freeman
@freeman-lab
Jul 23 2015 15:23
glad you worked it out @shariqiqbal2810 ! can you tell us what version of thunder you pip installed? and you launched by running the thunder-ec2 executable?
the version that gets deployed should be the one you installed, in which case that conflict shouldn’t arise, but there might be a bug
shariqiqbal2810
@shariqiqbal2810
Jul 23 2015 16:21
I pip installed without specifying a version, and it gave me 0.5.1. I launched the cluster using thunder-ec2 and the version of thunder on the cluster was 0.6.0.dev
Jeremy Freeman
@freeman-lab
Jul 23 2015 16:22
perfect, thanks for the info! there must be a bug in how it’s getting the version on ec2, because by default it should use the same version that you have locally (which in this case would have been 0.5.1), will look into it
shariqiqbal2810
@shariqiqbal2810
Jul 23 2015 16:23
No problem! Glad I could help
shariqiqbal2810
@shariqiqbal2810
Jul 23 2015 17:06
Hello again! After pulling the newest version of Thunder, I found that the ec2.py isn't working properly. Specifically, the install_thunder method is trying to copy from a python folder inside the thunder project that doesn't exist. This is the error I'm getting:
Traceback (most recent call last):
  File "/Users/shariqiqbal/anaconda/lib/python2.7/site-packages/thunder_python-0.6.0.dev0-py2.7.egg/thunder/utils/ec2.py", line 519, in <module>
    install_thunder(master, opts)
  File "/Users/shariqiqbal/anaconda/lib/python2.7/site-packages/thunder_python-0.6.0.dev0-py2.7.egg/thunder/utils/ec2.py", line 210, in install_thunder
    ssh(master, opts, "~/spark-ec2/copy-dir /root/thunder/python/thunder/utils/data/")
  File "/Users/shariqiqbal/anaconda/lib/python2.7/site-packages/thunder_python-0.6.0.dev0-py2.7.egg/thunder/utils/ec2.py", line 299, in ssh
    raise Exception(stdout)
Exception: File or directory /root/thunder/python/thunder/utils/data/ doesn't exist!
shariqiqbal2810
@shariqiqbal2810
Jul 23 2015 17:17
Once again, nevermind! For some reason the github website clone link didn't give me the latest version, so I didn't get the fix that you coded just this morning! I just pulled and should be all set!