These are chat archives for thunder-project/thunder
Hello again guys, how are you? I am just a step away form setting thunder here at now it's the time for demos, but I stumbled into an obvious problem that I hadn't foreseen somehow…
In order to load images,
thunder uses the ThunderContext
tsc which is created when thunder starts using the SparkContext
sc. But when submitting jobs to the cluster using
thunder-submit, there’s no initialization step where
tsc is created and therefore no
tsc.LoadImagesAsSeries for instance.
Is there a ThunderContext hiding around in thunder-submit and I haven’t found him yet, or do I have to create mine in order to load the data when running thunder-submit?
seriesloading examples, can you confirm you are running thunder
0.5.1? if so, this is an issue that’s fixed on master, i just need to push a release to pypi
@freeman-lab Also another small and probably unrelated issue I'm coming across in the same environment.
When I use Colorize.image() to display the example images that load ('mouse-images'), the image does not automatically display. I need to "import matplotlib.pyplot as plt" and do a plt.show().
From what I read in the code for colorize.py (https://github.com/thunder-project/thunder/blob/master/thunder/viz/colorize.py), image() itself seems that it should all ready show my image, though that is not happening. Also the tutorials also seem structured to suggest that image() should automatically display the images.
@AlexandreLaborde I believe you need to create a ThunderContext at the beginning of your script that you submit. This is also true for Spark. When you run Spark or Thunder in interactive mode, it automatically creates a spark-context for you (sc or tsc respectively). However, whenever you submit a job to a cluster, this context needs to be created at the beginning of your script.
I am still newish to both Spark and Thunder, so someone can correct me if I'm wrong, but I am pretty certain this is the case.