These are chat archives for thunder-project/thunder

21st
Feb 2015
Davis Bennett
@d-v-b
Feb 21 2015 20:29
@freeman-lab any idea how to speed this up?
mxProj = imDat.maxProject(axis=2).astype('float16',casting='unsafe').pack(sorting=True)
imDat is not cached
Davis Bennett
@d-v-b
Feb 21 2015 21:01
ok this runs at least 10 times faster on 40 nodes instead of 16 nodes
Jeremy Freeman
@freeman-lab
Feb 21 2015 23:22
@tomsains you are correct, the planarcrosscorr method should be running each plane separately, so it only adjusts xy displacements on a plane by plane basis
the crosscorr method does full 3D cross correlation (a single displacement in xy and z), but depending on your data, that may or may not be what you want
if it's light-sheet data, for example, that will potentially shift everything by at least an entire plane!
which is probably not what you want
@d-v-b not at the moment, other than scaling out the cluster, which it sounds like you did successfully :clap:
Davis Bennett
@d-v-b
Feb 21 2015 23:24
haha adding nodes is EZ
Davis Bennett
@d-v-b
Feb 21 2015 23:33
@freeman-lab having trouble with maxProject
subDat = imDat.applyValues(lambda x: x[0])
mxProj = subDat.maxProject(axis=2).pack(sorting=True)
it can't sort if the data have changed length
but it can't return useful data unless they are sorted properly
specifically, I get this error from the above code:
ValueError: Length of new index (3000) must match length of original index (1)
the output without sorting is jumbled up, but its histogram matches that of a manually-computed max projection, suggesting that sorting is the culprit