These are chat archives for thunder-project/thunder
.fromrdd()as you now might not have the same keys do to the
filterpart. If you leave the comforts of Thunder you need to take care of the keys.
data.mean()on a dataset where a single volume is 1024x1024x300 int16. I am getting:
ValueError: can not serialize object larger than 2G, did you solve that?
ThunderContextis gone is a definite upgrade. You can now just start Python however you want (IPython, Jupyter Notebook, PySpark, etc) and then simply import Thunder like you would any other package (e.g.
import thunder as td)
Colorize, we wanted to make Thunder more modular, so it was removed as it’s a little tangential to image/time-series manipulation. I don’t think the code was ever moved anywhere else, but you can still find it in the old release: https://github.com/thunder-project/thunder/blob/v0.6.0/thunder/viz/colorize.py
map()and save back out but aggregation seem to fail.
thunderdoes averaging across the distributed axis, but you should be careful doing that with
int16data, you might get overflow problems
sc.images.fromarraywith a big array, and I think I just worked around it