Is there an easy way to add all relevant individuals as editors of a google doc?
A basic summary on our experiences with logging agents:
* fluentd: Too slow, couldn't handle our volume at any reasonable resource usage
* fluentbit: Buggy. They've since fixed their broken JSON handling but there's probably other issues
* filebeat: Doesn't handle rotation and deletion properly
All but filebeat also had the issue where it would buffer internally if its sink went down, rather than backpressuring to its source.
ie. if you're reading from a file, but you can't send the data right now, you read from the file and put it in a new file instead of just...waiting before reading more from the file
IMO agents should be as simple as possible and do the bare minimum to get the data off the node into somewhere more robust (eg. message queue). Then any transforms or extra cross-referencing should be a seperate processing step.
I have a large number of enterprise customers at AWS who are using Fluent Bit, for the specific features that we use, I've not heard of any significant complaints.
I guess it stated with fluent/fluent-bit#1278
last I knew there was no official publishing support for journald, but it is a structured, compact, binary log format that is already used by most servers these days I'd assume
2 replies
@/all Logs vision is now published. I believe this is very important and a must read for OpenTelemetry Logs contributors: https://github.com/open-telemetry/oteps/blob/master/text/0092-logs-vision.md
@tsloughter that reflects our evaluations as well. By the way, Brian Troutwine (creator of cernan) has been advising us on Vector. I'm not sure if you worked with him at Postmates:
- fluentbit: Buggy. They've since fixed their broken JSON handling but there's probably other issues
The JSON issues! For a little bit of context, and to not repeat it here, see:
https://github.com/timberio/vector/issues/727#issuecomment-561004676
Fluentbit reached 1.1 with those issues đź‘€.
That was fixed quite a while ago, so I suspect this experience is a bit old
@PettitWesley we are still able to produce JSON parsing issues, but I agree, the major ones are fixed!
8 replies
In terms of a microservices architecture that used queues to send messages to various parts, is there a way to propagate a trace context via queue messaging. We have used OpenCensus a little bit and can serialize a TraceContext (which is basically the trace id) with which we can then add manually to queue payload. We have (non-http) processors that consume these queues and we would like to re-establish the trace context when processing. However we cannot seem to recreate the context from the serialized trace context. Is there a pattern for this or is this type of functionality not was intended.
10 replies
Please comment.
Direct link to Google doc: https://docs.google.com/document/d/1ix9_4TQO3o-qyeyNhcOmqAc1MTyr-wnXxxsdWgCMn9c/edit#
I just wanted to tell about a typo in the meeting notes: the meeting dates are in year 2010 !
probably a question for @tigrannajaryan
Ah, I know that struggle :) A colleague wrote this tool to automatically upload recordings from recurring zoom meetings to Google Drive folders: https://github.com/graphaelli/zat/ then we just link to the folder with all the recordings for discoverability
@roncohen neat, I'll check that out
@liias OpenTelemetry tracing is fairly mature and is currently in Beta. I know people who use it at least parts of OpenTelemetry in production (e.g. the Collector). Support for logs does not exist yet. We started discussions to support logs a few weeks ago and I do not expect logs to be production-ready for the next few months.
Folks, glad to join this forum. We are a small group of folks working on a full observability stack at https://logiq.ai. Would be happy to contribute to the logging efforts. Look forward to engaging with the community.
@rparth welcome. Feel free to join SIG meetings (meetings are in OpenTelemetry Calendar) and review proposals (recent open proposal is about Data Model: https://docs.google.com/document/d/1ix9_4TQO3o-qyeyNhcOmqAc1MTyr-wnXxxsdWgCMn9c/edit?ts=5e990fe2#)
Hi folks! I'm from New Relic. We recently released a logging product. I wrote the Java Agent integrations with logback 1.2, log4j 1, log4j 2, and java.util.logging, and served as a tech lead across the other languages. The Java Agent writes its own logs using log4j 2. I'm a little late to the design process. Let me know how I can contribute.
@jeffalder you may want to sync with @zenmoto who works on Logging Library spec.
@jeffalder yes, there is a rough plan on what we want to do: https://docs.google.com/document/d/1cX5fWXyWqVVzYHSFUymYUfWxUK5hT97gc23w595LmdM/edit#heading=h.ym810wjhhbxw
I am personally focused on the data model, the protocol and the Collector. @zenmoto works on library specs. There are several other topics that I think nobody owns yet.
I am personally focused on the data model, the protocol and the Collector. @zenmoto works on library specs. There are several other topics that I think nobody owns yet.
Based on last weeks discussion in the SIG, I put up PRs for two suggested edits to the log vision otep.
Quantifiable resource utilization goals for the unified collector: open-telemetry/oteps#101
A concrete list of log sources to support: open-telemetry/oteps#102
Quantifiable resource utilization goals for the unified collector: open-telemetry/oteps#101
A concrete list of log sources to support: open-telemetry/oteps#102
@zenmoto Here's our repo: https://github.com/newrelic/java-log-extensions and our user documentation https://docs.newrelic.com/docs/logs/enable-logs/logs-context-java/configure-logs-context-java
Hi! I am getting a little confused about what open-telemetry can do. I am especially interested in the logging ability where I read
OpenTelemetry captures metrics, distributed traces, resource metadata, and logs (logging support is incubating now) from https://opentelemetry.io/about/ . Where can I find documentation on the incubation work being done? Is there an example of a Go implementation of this? Does OpenTelemetry act like a wrapper around login the case of Go or does it provide it's own solution like logrus or Uber's zap?
24 replies
@tigrannajaryan I was considering forking the New Relic Java log extensions and modifying them for OTel spans. Would that be a useful demo? I can also talk Wednesday on some of the principles and lessons learned that guided New Relic’s library design, if that would be interesting.