These are chat archives for thunder-project/thunder

Jul 2016
Jul 20 2016 09:15

Hi All,

I have a series of images (800x800 pixels) over time (2512 frames) and I'm trying to calculate the correlation between every pair of pixels (640000x640000 correlation matrix).
Everything seems to work until I'm trying to show the correlation matrix.
This is the code I'm using:

from thunder import images
%matplotlib inline
import matplotlib.pyplot as plt
data = images.fromtif('/homedtic/ainsabato/eranet/Bobby2_1',engine=sc)
series = data.toseries()
signal = series.flatten()
correlations = series.flatten().correlate(signal)

I'm very new to Spark and Thunder, so maybe it is just a simple problem.
Besides I would also like to calculate the p-value of correlations (I assume correlate calculates Pearson rho), how can I do it?
Thank you in advance.