hm? that isn't the case. also the otep PR is not the spec, it is the process before a spec is written
ah saw your comments on the PR. don't think you should worry about getting in a logz.io mapping/example to the OTEP. -- unless you think it shows something that would require changing the otep, but the otep isn't the spec so the spec isn't in stone after its merger
I didn't bother sending an Erlang mapping like there is for log4j and zap because I don't think it would add anything to the otep, but there certainly will be one when the spec is complete
@tigrannajaryan thank you for all the amazing work and your leadership 👍🏻🙏🍻
Rahul M Chheda
Congrats @tigrannajaryan ! That's great!
Thanks everyone for reviewing and providing valuable feedback. Next: logging libraries, log protocol and log data support in Collector.
@/all I have to move the SIG meeting time today to 11am PT avoid the overlap with OpenTelemetry Community meeting.
I am not sure if this is the correct location to ask this question; however, are there specific individuals that are allowed to participate in the Log SIG meetings? I am looking to listen on the calls, but I am open to contributing. TIA!
Do any of you know eliot? The Python structured logging library?
I'd like to start discussing the API for our logging SDK
Anyone up for it?
I'm starting to define an internal project for logging, and I was hoping to be able to take advantage of open telemetry. I'd be happy to participate in the logging API.
@avik-so That's great.
would opentelemetry logs encompass error/exception reporting? I feel like it is the same data just maybe represented differently
regarding defining a logging API: when the SIG started in late March, this was discussed. see for example the notes for March 25 and April 8––the group at that point decided to focus on integrating with existing logging libraries first, then tackle the logging API. the overall sense was that most projects already had integrated with a library (log4j, ...) and forcing the use of yet something else was going to be an uphill battle.
I somewhat agree
But new projects would probably use our API
I think this is sound logic, but I will also say that personally, I'd love to see OT as a one-stop shop for all telemetry instrumentation, and as you are saying, what about the next 100,000 projects? :)
personally, I think if there's critical mass around defining a logging API among folks participating here, nobody should keep anybody from making progress there, and I do not think the original prioritization was meant to prevent this. it was just that the folks being able to spend time on logging within OT in general at that time wanted to dig into integrating with existing first
I think it is a fine topic to discuss at the SIG meeting
I think everyone likes the idea of a logging API, but the people who've been working on this project since 2019 are more interested in finishing the tracing and metrics APIs. I don't think you'll get full attention on logging APIs until then. (P.S. I've paid zero attention to the logging SIG because I'm in this boat, I've got a metrics API to finish--and yet I've designed and implemented numerous logging APIs in C++ and Go.) I'd be happy if all the people interested in logging would pitch in on tracing and metrics!
2019 member here — I do not think it is a reasonable expectation that everyone be interested in every aspect of the project. Plenty of people only care about a subset of observability data. Also, as the community grows we should leverage all the skillsets available — we have many new additions who are primarily interested in logs including AWS, Elastic, and SumoLogic. While it would be great to be involved in every aspect that is not feasible as the project grows. Of course we should not slow down on the aspects on the path to GA being traces and metrics and @mtwo already covered the goal is to GA later this year with logs likely entering beta.
HI all, I've been thinking about how this Logging sub-project could help more with tracing and metrics. I was thinking that before beginning to work on Logging API design, we could try to standardize the translation from Span and Metric events into structured logging events. This was requested as a feature for the metric SDK (open-telemetry/opentelemetry-specification#617), and I think the same could be said of spans. Right now many OTel repos are including standard-output exporters for both trace and metrics, but there are no standard conventional data formats for them to use, so it's a bit arbitrary what you get. I think it would be great if we had a standard structured log form for both metrics and spans.
@jmacd do you mean a standardized way to encapsulate span data and metric data inside a log record or something else?
say you wanted to simply write spans into elastic search, for example
Got it. Do you want to do an OTEP? (Assuming you want to get distracted from metrics :-) )
I don't. I can't :)
Hi all. I just come up with an idea. As we will transfer logs to a remote collector in some sense, could we leverage dictionary encoding? Just like nano log(https://www.usenix.org/conference/atc18/presentation/yang-stephen). Briefly it makes a dictionary for static parts of every log message and writes only dynamic part as a compact binary log at runtime, which is super fast. This binary form is easier to transfer and analyze. A post processor is needed to make it human readable.