These are chat archives for thunder-project/thunder

12th
Oct 2015
Davis Bennett
@d-v-b
Oct 12 2015 22:26
@freeman-lab how hard would it be to enforce that all the records in a series object are spatially contiguous
per worker
or was this the imageblocks idea
Jeremy Freeman
@freeman-lab
Oct 12 2015 22:42
@d-v-b that's more or less what you get if you convert images -> blocks
specifically, you get a collection of blocks each of which spans some range in x,y,z and all of t
Davis Bennett
@d-v-b
Oct 12 2015 22:46
...and is there a blockData.maxProject() method ?
Jeremy Freeman
@freeman-lab
Oct 12 2015 22:50
you mean max project over a spatial dimension?
Davis Bennett
@d-v-b
Oct 12 2015 22:50
ya
Jeremy Freeman
@freeman-lab
Oct 12 2015 22:51
hm, currently no, but it's possible in principle
the logic of reassembling the result into an image is missing
some of what we're doing in bolt might make it more straightforward
the goal here is just to do max projections at all time points?
Davis Bennett
@d-v-b
Oct 12 2015 22:55
the goal is to to a max projection after an operation like dff
without unnecessary images --> series conversions
Davis Bennett
@d-v-b
Oct 12 2015 23:02
but also I'm motivated by the intuition that there's no advantage to totally random distribution of timeseries records across workers
and that if the distribution of timeseries records across workers should have any structure, it should be spatial structure
Jeremy Freeman
@freeman-lab
Oct 12 2015 23:06
yeah that's definitely a good intuition, and is why many operations work well in the block domain
(e.g. spatiotemporal feature extraction)
Matt Valley
@mtv2101
Oct 12 2015 23:50
@freeman-lab Is it possible to pull data from an S3 bucket that lives in a different AWS region from the EC2 instance? I get a error message to the effect of "data has moved" when I try to do this. Perhaps a bucket policy could permit this?