These are chat archives for thunder-project/thunder

27th
Apr 2016
Henry
@hluetck
Apr 27 2016 07:08
@jwittenbach @freeman-lab
data = fromrdd(rdd.map(lambda kv: ((kv[0],), kv1)))
This worked for me. Thanks very much for the help!
Jeremy Freeman
@freeman-lab
Apr 27 2016 12:51
@hluetck great! would you mind opening an issue on GitHub describing this example and
how it was solved?
Kyle
@kr-hansen
Apr 27 2016 13:31
@freeman-lab It is possible to call .mean().toarray() so that is what I have been doing. It makes sense to me for .mean()etc to return the same type of object as a series and images of the same type.
So just to be certain, is images.squeeze()intended to only squeeze out spatial image dimensions, and not that same singleton dimension from collapsing across all images (time)? That is something that might be worth clarifying on the current docs (http://docs.thunder-project.org/). As it currently reads, I interpreted that "removing a single dimension axes from images" should also remove the single dimension time axes from all image collapsing, though it isn't written to work that way.
Also, if images.squeeze() is meant to only work on spatial dimensions, perhaps a note in the documentation directing someone to .toarray() to remove the time dimension might be helpful. As it stands, .toarray() doesn't show up anywhere in the documentation as a method that can be applied to images since it is in the base. There is a brief blurb in the overview mentioning .toarray() being applied to both series and images, but it isn't clear that converting it to an ndarray will also remove that additional dimension.
Sorry for being kind of annoying about it, but I'm gonna be showing Thunder to some people in my lab in the near future and am trying to foresee issues I'll run into once they start using it to try and save myself some of the confusion that may arise :) As it stands, I'll be fine explaining these things to them, but I'm just trying to consider issues people might run into who aren't as comfortable diving into the code on the github to figure out how things are working.
Jason Wittenbach
@jwittenbach
Apr 27 2016 15:55
@hluetck thanks for testing that out! This case is simple enough, that we should handle it in Thunder so the user doesn’t have to be bothered to remember this book-keeping. Just submitted a PR so that your original code should work now as well: thunder-project/thunder#292