@jeffalder please sync @zenmoto since it appears what you describe overlaps with he is doing / done. Definitely useful to hear about your learnings.
As for "forking the New Relic Java log extensions" it is important to understand the full implications. We went through a process of companies donating their existing solutions to OpenTelemetry (particularly with Java autoinstrumentation library). I am not sure a fork fits this model (due to branding, licensing implications and vendor neutrality stance of OpenTelemetry. You may want to discuss this in more detail with OpenTelemetry TC and Admins and look into the history of Java autoinstrumentation donation as a possible model to follow if you intend to donate solutions. A fork is fine as a proof of concept though if you want to demonstrate a concept.
I believe at this stage it is important to come up with library specs and guidelines (and doing proof-of-concepts is likely necessary for this). The implementations can then follow, based on the agreed guidelines.
As for "forking the New Relic Java log extensions" it is important to understand the full implications. We went through a process of companies donating their existing solutions to OpenTelemetry (particularly with Java autoinstrumentation library). I am not sure a fork fits this model (due to branding, licensing implications and vendor neutrality stance of OpenTelemetry. You may want to discuss this in more detail with OpenTelemetry TC and Admins and look into the history of Java autoinstrumentation donation as a possible model to follow if you intend to donate solutions. A fork is fine as a proof of concept though if you want to demonstrate a concept.
I believe at this stage it is important to come up with library specs and guidelines (and doing proof-of-concepts is likely necessary for this). The implementations can then follow, based on the agreed guidelines.
I'd also hope this could just go in the SDK as a processor and not require a seprate library
I’m not at all proposing the fork as a solution - just a throwaway prototype since I’m sure there will be better ways to accomplish it, and OT’s requirements aren’t New Relic's requirements. I could also simply present on our design principles and why we implemented things the way we did and call it good.
1 reply
@jeffalder have you seen https://logging.apache.org/log4j/2.x/manual/thread-context.html ?
The implementation is different from logback's MDC, but capabilities are similar it seems
@nilebox You’re right, that is very similar. It’s been a while since I reviewed the design. One of the design principles in the New Relic extensions was zero-injection of bytecode. Because of the way the New Relic span creation works, I think there wasn’t a zero-injection way of intercepting the message and adding context in a useful way. We ended up inheriting the MessageFactory … and at that point there was no real point in using ThreadContext when we could use our own Message class. (This is me trying to recapture the decision we made at the time; there may have been other reasons.)
https://github.com/newrelic/java-log-extensions/tree/master/log4j2
Certainly an improvement to consider in the next round!
@/all if you have not reviewed the Data Model proposal yet please do: open-telemetry/oteps#97
If you are a spec approver please approve if you have no objections. If you are not a log expert please invite someone trusted who is an expert and can help you to review.
I would like to make sure we are moving forward and finalize the data model and if possible settle all outstanding data model issues in tomorrow's Log SIG meeting.
The data model is a prerequisite for a lot of work that cannot start until we agree on the data model (including protocol work and many parts of PoC of adding logs to Collector).
If you are a spec approver please approve if you have no objections. If you are not a log expert please invite someone trusted who is an expert and can help you to review.
I would like to make sure we are moving forward and finalize the data model and if possible settle all outstanding data model issues in tomorrow's Log SIG meeting.
The data model is a prerequisite for a lot of work that cannot start until we agree on the data model (including protocol work and many parts of PoC of adding logs to Collector).
IOW if withholding an approval will help us get to 1.0 sooner, I will do that.
@jmacd getting rid of the internal OpenCensus data format is blocked by finalizing the protocol definitions. We are not going to start working on metrics conversion when metrics protocol is in such fluid state since every change in a protocol definition breaks Collector internal code.
Things are better for trace part. Trace-related components in the Core are now converted to OTLP with the exception of Zipkin. Collector core maintainers do not intend to convert contrib components. We will publish guidelines for contributors to convert their components once we are certain trace protocol is final and we are not going to make further changes to it in the near future. I am deliberately delaying this to avoid asking contributors to redo their work again.
Logs work is done by different people (me) and has no effect on this (since I am not doing conversion work - it is primarily Bogdan and others).
Withholding log approval is not going to help with this in any way. Expediting metric work and finalizing metric protocol will help.
Things are better for trace part. Trace-related components in the Core are now converted to OTLP with the exception of Zipkin. Collector core maintainers do not intend to convert contrib components. We will publish guidelines for contributors to convert their components once we are certain trace protocol is final and we are not going to make further changes to it in the near future. I am deliberately delaying this to avoid asking contributors to redo their work again.
Logs work is done by different people (me) and has no effect on this (since I am not doing conversion work - it is primarily Bogdan and others).
Withholding log approval is not going to help with this in any way. Expediting metric work and finalizing metric protocol will help.
@bogdandrutu I believe we were already moving in the direction of having separate approvers for signals, right?
https://github.com/open-telemetry/opentelemetry-collector/issues?q=is%3Aissue+is%3Aopen+label%3Alogs <- here are the three collector
logs issues.
One of the things I'm wondering about is how we'd model logging inside the application.
Just like to share a small tool we used to benchmark our various log agent implementation, it is still a work in progress with many features pending, feel free to contribute or share your opnion: https://github.com/awslabs/amazon-log-agent-benchmark-tool
ah saw your comments on the PR. don't think you should worry about getting in a logz.io mapping/example to the OTEP. -- unless you think it shows something that would require changing the otep, but the otep isn't the spec so the spec isn't in stone after its merger
I didn't bother sending an Erlang mapping like there is for log4j and zap because I don't think it would add anything to the otep, but there certainly will be one when the spec is complete
@/all we have an official data model, congrats! https://github.com/open-telemetry/oteps/blob/master/text/0097-log-data-model.md
