These are chat archives for thunder-project/thunder
I have a series of images (800x800 pixels) over time (2512 frames) and I'm trying to calculate the correlation between every pair of pixels (640000x640000 correlation matrix).
Everything seems to work until I'm trying to show the correlation matrix.
This is the code I'm using:
from thunder import images %matplotlib inline import matplotlib.pyplot as plt data = images.fromtif('/homedtic/ainsabato/eranet/Bobby2_1',engine=sc) series = data.toseries() signal = series.flatten() correlations = series.flatten().correlate(signal) plt.imshow(correlations)
I'm very new to Spark and Thunder, so maybe it is just a simple problem.
Besides I would also like to calculate the p-value of correlations (I assume correlate calculates Pearson rho), how can I do it?
Thank you in advance.