These are chat archives for thunder-project/thunder

Nov 2015
Nov 06 2015 21:23

I'm a new user to Thunder and am planning to get it up and going on my institution's cluster. Their cluster has Hadoop 2.6 installed with Spark 1.2-1.5 installed and optimized for Hadoop 2.6.

From what I understand, the Thunder from pip was built for Hadoop 1.x, which I believe explains why I can't load the example datasets. Will building Thunder from source allow it to work on this setup, or will I need to install redundant versions of Hadoop 1.x and Spark dependent on Hadoop 1.x on the cluster to get Thunder running in this environment?