These are chat archives for HdrHistogram/HdrHistogram
ArrayIndexOutOfBoundsExceptionsays that "Any shifting into the bottom-most half bucket would represents a loss of accuracy". I think I follow, but just in case... Let's say you have a sub bucket count of 2048 and min value 1 (unit magnitude 0), and you have a value of 2048 recorded (first entry in bucket index 1). Conceptually if you shift right twice, that's 512 which can be represented in the bottom bucket, but adjusting the counts index (2048) by -1024 * 2 would get you to bucket 0 which maps to value 0 which is wrong. However, value 4092 (index 2047+1024-1=3070) gets shifted to index 3070-2048 = 1022, which is correct. So, because the bottom half bucket of bucket 0 isn't on half the scale of the top half, the index shift by a multiple of subBucketHalfCount doesn't work right -- is this the "loss of accuracy" you were referring to, aka being wrong?
subBucketHalfCountMagnitude? Also, when calculating
unitMagnitude, it's (int) Math.floor(...) -- wouldn't the int cast already floor? Or is there some subtletey there I'm missing
final int index = Math.min(Math.max(0, countsArrayIndex(value)), (countsArrayLength - 1));