These are chat archives for thunder-project/thunder

28th
Jul 2015
tomsains
@tomsains
Jul 28 2015 12:45
@freeman-lab I have just tried installing thunder on one of our machines which is running ubuntu.
I have launched a cluster on amazon EC2 and then I have tried to view the first image of the data set by calling image(images[0]) but this error is being returned:
Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.runJob.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 2.0 failed 4 times, most recent failure: Lost task 0.3 in stage 2.0 (TID 11, ip-172-31-32-65.eu-west-1.compute.internal): org.apache.spark.api.python.PythonException: Traceback (most recent call last):
  File "/root/spark/python/pyspark/worker.py", line 88, in main
    command = pickleSer._read_with_length(infile)
  File "/root/spark/python/pyspark/serializers.py", line 156, in _read_with_length
    return self.loads(obj)
  File "/root/spark/python/pyspark/serializers.py", line 405, in loads
    return cPickle.loads(obj)
  File "build/bdist.linux-x86_64/egg/thunder/__init__.py", line 15, in <module>
  File "build/bdist.linux-x86_64/egg/thunder/extraction/__init__.py", line 1, in <module>
  File "build/bdist.linux-x86_64/egg/thunder/extraction/cleaners.py", line 1, in <module>
  File "build/bdist.linux-x86_64/egg/thunder/extraction/source.py", line 10, in <module>
  File "build/bdist.linux-x86_64/egg/thunder/extraction/source.py", line 43, in Source
ImportError: No module named zope.cachedescriptors
i seem to remeber someone reporting a similar error recently but i couldn't find it in the issues section on git hub
tomsains
@tomsains
Jul 28 2015 13:22
sorry guys. False alarm - I just scrolled up the page and found the solution
Jeremy Freeman
@freeman-lab
Jul 28 2015 14:53
no worries! still working on a more general fix on this one, should have it today
Jeremy Freeman
@freeman-lab
Jul 28 2015 15:54
@tomsains @shariqiqbal2810 that issue should be properly fixed now, turns out to have been a problem with the git tag, so no upgrade / reinstallation neccessary
if you pip install, say, v0.5.1, you should get v0.5.1 on the cluster, and there shouldn’t be any dependency conflicts
Jeremy Freeman
@freeman-lab
Jul 28 2015 16:01
also @/all you may have noticed a reorgnization to the repo by which the top-level scala folder has been removed and what was the python folder is now top-level
this shouldn't have any user-facing consequences, but especially if you’ve been working against the master branch, be aware that things have moved around
the big win is that we eliminated all scala dependencies, which should prevent some deployment / installation errors, and is part of a road map torwards simplifying deployment more genreally