These are chat archives for HdrHistogram/HdrHistogram

Oct 2015
Gil Tene
Oct 03 2015 23:50 UTC
I can see using JSON for the metadata in a log format of stream record format, basically single records that are JSON encoded. In log files, they can be identified as such for single-line record (e.g. #MetaData: (...), or maybe #JSONMetaData:(...) ). However, I still prefer using a newline as a record delimiter in streams and logs for various long-lasting reasons.
I do not see JSON as a good format for a log file, or as an acceptable format for framing in a stream. Having been around long enough to see the latest "widely supported" general purpose multi-field formats come and go (ASN.1 , XML, JSON, ...), I find that none of them ever have the longevity you'd expect from simple single-line and CSV "scraping" formats, and they all tend to lack the framing qualities desired in streams of information. See if you can find a python or javascript ASN.1 decoder for 1990s messages ...
In any case, having list of metadata fields we would like to have conventions for is where we should probably start (needed regardless of whether or not we use JSON).
StartTime and BaseTime are certainly fields I've found use for already.
Stream format version (and implied histogram encoding version) is on that list too.
A way to identify the originator, e.g. "jHiccup v2.0.6" is also semi-obvious
Gil Tene
Oct 03 2015 23:57 UTC
Then we have things to name the thing being logged. Here I think we end up needing multiple labels and probably support potential hierarchies or tags. E.g. something can be "accounting app", "server A", "cluster D", and "response time" all at the same time. And it could probably use some organization identify on top of that too (org.freecloud.research.lab1).