These are chat archives for HdrHistogram/HdrHistogram

22nd
Apr 2015
Stevo Slavić
@sslavic
Apr 22 2015 01:40
hello
first of all, please forgive my ignorance
I'm wondering if code like https://github.com/apache/kafka/blob/0.8.2/core/src/main/scala/kafka/tools/TestEndToEndLatency.scala could benefit from using HdrHistogram
Gil Tene
@giltene
Apr 22 2015 01:52
Yes. Assuming numMessages is significant enough (in the 1000s or higher), HdrHistogram will be smaller and faster, and will produce the same %'lies.
However, the code also has the classic Coordinated Omission issue exhibited by most in-code measurement that takes time before & after an operation. You may want to look at the discussion of this at http://latencyutils.org
Stevo Slavić
@sslavic
Apr 22 2015 01:56
thank you very much
Stevo Slavić
@sslavic
Apr 22 2015 02:04
I've started writing something similar to referenced test (with same measurement issue) with a difference that it's supposed to run continuously and stores only one measurement in the test itself - monitoring tool/lib is polling for values periodically, these raw collected data get stored, statistics can be calculated on that data and both raw recorded values and statistic can be visualized; some measurements can be lost or repeated depending on polling frequency
do you see any fundamental failures in this approach? and would HdrHistogram be applicable/useful in this case?