These are chat archives for thunder-project/thunder

11th
Nov 2015
Kyle
@kr-hansen
Nov 11 2015 01:59

@jwittenbach Thanks for the pointer. That may have been the problem to some degree, as it solved some issues. However, I'm still not convinced everything is working. I also installed Thunder on my local Windows for local testing (which was tricky in and of itself, so there may be additional issues there). My Windows version is using Spark 1.4.1 for Hadoop 2.6 as well and I get similar errors. There seems to be an issue with loading binaries?

For example, working through this Tutorial (http://thunder-project.org/thunder/docs/tutorials/thunder_context.html) most things seem to work, but several steps fail as follows:

  1. image(data.values().first()) - The image is not automatically shown. I have to enter plt.show() to view the image. Perhaps that is built in to Colorize.image, but I didn't anticipate that was the case from the structure of the tutorial code. However, putting in the full paths as you suggest allows everything to load
  2. data = tsc.loadSeries(datapath + '/iris/iris.bin', inputFormat='binary') - Fails. However, I can load the .mat and .npy arrays in the following steps just fine to get the correct output.
  3. data = tsc.loadSeries('savelocation') - Gives an error similar to when I was using tsc.loadExample. The other conversion steps before this seem to work without errors.
Kyle
@kr-hansen
Nov 11 2015 02:13

@jwittenbach So after doing a little bit more on the tutorial I mentioned above, it is actually not an issue with tsc.loadExample. Things worked fine continuing through the makeExample portion, and I can correctly load and view:
data = tsc.loadExample('mouse-images')
img = data0
image(img)
plt.show()

This is also true for the 'fish-images.' However, I cannot correctly load the 'mouse-series' or 'fish-series'. Is it perhaps that the series files are saved as binaries?

Also, I can put together an iPython Notebook sample tomorrow if that would be helpful in figuring out what may be happening.