These are chat archives for thunder-project/thunder

6th
Mar 2015
wolfbill
@wolfbill
Mar 06 2015 08:35
@freeman-lab I'd like to setup a thunder cluster so that it can dealwith big data
wolfbill
@wolfbill
Mar 06 2015 09:10
@npyoung Hi man,how did you setup you thunder cluster?
tomsains
@tomsains
Mar 06 2015 10:11

@wolfbill have you got thunder and it's dependencies installed on your local machine?
Running Spark and thunder on Amazons EC2 (which is a cloud computing service) is the easiest way to launch a large cluster.

The first thing that you need to do is set up an account and your security credentials with amazon web services (AWS) - detailed instructions on how to do this are provided here: http://thunder-project.org/thunder/install_ec2.html

Once you have followed these instructions and have:

  • Your 'Access' and 'Secret' keys added to your /.bash_profile
  • Your keypair stored in your home folder (on your local machine)
  • permissions for your key pair set by typing chmod 600 ~/mykey.pem into your terminal

you can then launch a cluster on EC2 with thunder and spark installed by typing:

thunder-ec2 -k mykey -i ~/mykey.pem -s <number-of-nodes> launch <cluster-name>

where mykey is the name of your keypair file
and you insert the number of nodes and you give your cluster a name

tomsains
@tomsains
Mar 06 2015 10:32
@freeman-lab when using the Fourier Transform function what are the units of the frequency - I'm assuming this can't be in Hz because you do not input your frame rate.
So is the frequency the distance between peaks in number of frames?