I was researching the capability to push K8s logs through
Fluent bit->OpenTelemetry Collector->any output of choice pipeline and found that it's actually plausible with the recent additions to the collector and a (rather small) modification to a FluentBit Helm chart. I have prepared a doc that summarises it and brings rationale behind some newly added issues (mostly related to K8s Processor extensions): https://docs.google.com/document/d/1QlFbXz0eQUaKXK1WrnEs3VqRkcPD2RAdXJdJMjnOSd8/edit?usp=sharing Would love to hear if that ideas sounds reasonable and if so, will be happy to work on that. I could also demo that during the upcoming SIG meeting and discuss it
3 replies
I see the data model https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/logs/data-model.md#log-and-event-record-definition but I'm looking to see if I can specify log data as JSON-LD: https://json-ld.org/
it'd make things easier if I could specify a key/value pair and be able to link it to different concepts
i.e. using an owl-time instant for the logging timestamp https://www.w3.org/TR/owl-time/#time:Instant
so plain old structured logging for the most part
but with a
@context field and being able to mix and max schemas
instead of having to extract it from regex
basically I want to avoid this https://medium.com/palantir/using-static-code-analysis-to-improve-log-parsing-18f0d1843965
Log SIG meeting is today at 3pm PT. It disappeared from Otel calendar, I will chech why. Zoom link https://zoom.us/j/8203130519
Hi everyone, we would like to seek input from the OTEL Community on logging statements for the Logging API in C++. Any suggestions / feedback would be greatly appreciated! The RFC is linked here: open-telemetry/opentelemetry-cpp#356. Thank you!
hi everyone .. is the https://opentelemetry.io/docs/collector/ currently usable for logging arbitrary events? E.g. we have a "player" app which should relatively simplistically log when an item starts playback, when it encounters an error, & when it stops playing.
Elastic Common Schema (https://www.elastic.co/guide/en/ecs),
Splunk Schema (https://docs.splunk.com/Documentation/CIM/4.17.0/User/Overview),
Graylog Schema (https://schema.graylog.org),
Security Logging Standard still an open question (https://www.scip.ch/en/?labs.20180315)
https://ieeexplore.ieee.org/document/8587302
we find the logging practice is adopted highly inconsistently
among different developers both across projects and even one project
Conclusion: Both .. have forced us to question whether it is a reliable means to
understand the runtime behavior of software systems via analyzing the
logs produced by the current logging practice.
I guess we pick one schema and they eventually converge towards a standard.
I think Logging Provider / exporter implementation in SDK may export to whatever scheme. It's a developer choice, not enforcing just one schema. There could be systems already using ELK, others don't. We provide exporters for Logstash, for Graylog, for Microsoft Common Schema, and whatever other org schema. It's a huge hassle to adopt a new schema for an organization that has years of prior experience with their own schema, then years migrating to even a slight change in schema revision, not mentioning moving to entirely new schema. Original logging statements in code must be schema-agnostic as much as possible, with exporter in SDK deciding how to pack / in what logging format to export.
I agree. OpenTelemetry is an open-source project. It should support Vendor Common Models (Microsoft, Oracle, Splunk, Graylog, elastic, etc) because the industry invested in these solutions. The provider model is well understood. However, today's Vendor Common Models will eventually be obsoleted - the systems lifecycle. So Industry Common Model (DMTF CIM simplified or replaced) could emerge once todays Vendor CIM go away. Otherwise we get Vendor Common Model V2 in ten years time and Vendor Common Model V3 in 20 years time.
My guess is that that means logs in a more formal sense than the usage I mentioned.
Regarding sending events, I guess we might use https://www.github.com/open-telemetry/opentelemetry-js/tree/master/packages%2Fopentelemetry-web .. We're currently using https://github.com/DataDog/browser-sdk -- if the Collector understands the format that sends, we might just point it at the Collector and keep on using it.
I looked at https://docs.datadoghq.com/tracing/connect_logs_and_traces/python Tried to add the traceid and spanid in the logs. But the traces and spans generated by opentelemetry and dd are different. Is there a way to get the context traceid/spanid natively in Python ?
1 reply
@/all I am cancelling the Log SIG meeting today. Christian and Przemek can't attend, David and I don't have any topics and I don't see any topics in the agenda doc either. Everybody in Otel is focused on traces and metrics release. Let's meet next week (check the calendar, the time is different).
Hey guys, I'm working on the Logging SDK and I have ran into a design choice problem. The .NET logging implementation, which I was using as reference, has the LoggerProvider contains a list of processors that all the Logger's use when a logging statement is called. However, this prevents different loggers from writing to different processors. An alternate approach that I'm considering is having each Logger contain its own list of processors, such that different loggers can write to different processors. This would be useful if, for example, we wanted a console logger that only writes to console, and a separate logger that writes to a different destination (ElasticSearch for example). Does anyone have any objections to this second approach? Thanks in advance :)
1 reply
I have a question about the Log data model. The usage of Name, in the appendix ( https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/logs/data-model.md#appendix-a-example-mappings ) sometimes Name is describing a single Event (example: EventLogs, CloudTrail, ...) and sometimes a collection of entries (example: Google Cloud Logging). Looking at the description "Short event identifier that does not contain varying parts.", it feels that Google Cloud Logging is wrong... but I see a need to have some kind of grouping key (like : LogName). What's everyone's idea around this?
6 replies
I see the Logs GA definition at https://github.com/open-telemetry/oteps/blob/master/text/logs/0130-logs-1.0ga-definition.md . Is there a rough estimate for when Logs GA might happen? 2020 or 2021? :)
1 reply
Hi Guys,
Has it been decided that what logging libraries would be supported per language, for example for golang will it be Zap or logrus etc?
Also, I understand this is an ongoing project, but would really appreciate if someone gives an idea that it will involve schema validation as well? Or that heavy lifting will be taken on by the collector? Or none of these are requirements.
Has it been decided that what logging libraries would be supported per language, for example for golang will it be Zap or logrus etc?
Also, I understand this is an ongoing project, but would really appreciate if someone gives an idea that it will involve schema validation as well? Or that heavy lifting will be taken on by the collector? Or none of these are requirements.
3 replies
Would it also be possible to directly plugin ECS(Elastic Common Schema) for example https://github.com/elastic/ecs-logging-go-zap into first party application?
Thanks!
All, here is a separate issue about Telemetry Schema: open-telemetry/opentelemetry-specification#1324
Please comment.
Please comment.
Hey guys, I wanted to ask has anyone used @Include keyword on fluentbit in otel's main yml file(otel-collector-config.yml)? I getting issue in including a config file, can someone please provide format for this?
My file for now is:
extensions:
fluentbit:
executable_path: /opt/td-agent-bit/bin/td-agent-bit
tcp_endpoint: 127.0.0.1:8006
@INCLUDE /usr/local/go/src/opentelemetry-collector/examples/tracing/temp.conf
receivers:
fluentforward:
endpoint: 0.0.0.0:8006
