Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Ghost
    @ghost~5b3983f3d73408ce4f9f4600
    Hi all, have the Log SIG meetings recorded? I don't see any recordings posted to the OpenTelemetry Youtube channel.
    1 reply
    uname223
    @uname223
    Hi There, I have a question, as there's ton of discussion regarding different log agents/collectors, is there somewhere a status, or milestones in terms of log format itself?
    4 replies
    Aditya Prajapati
    @Syn3rman
    Hey all,
    I'm interested in participating under community bridge for otel and was going through the issues to find something to work on. Since the project idea is implementing a fluentbit exporter, I was trying to find some newcomer-friendly issues related to either logs or exporter, but most are either assigned, stale, or completed. Can anyone point me to some issues that I can work on to get a feel for the contributing process?
    2 replies
    Przemek Maciolek
    @pmm-sumo
    I was researching the capability to push K8s logs through Fluent bit->OpenTelemetry Collector->any output of choice pipeline and found that it's actually plausible with the recent additions to the collector and a (rather small) modification to a FluentBit Helm chart. I have prepared a doc that summarises it and brings rationale behind some newly added issues (mostly related to K8s Processor extensions): https://docs.google.com/document/d/1QlFbXz0eQUaKXK1WrnEs3VqRkcPD2RAdXJdJMjnOSd8/edit?usp=sharing Would love to hear if that ideas sounds reasonable and if so, will be happy to work on that. I could also demo that during the upcoming SIG meeting and discuss it
    3 replies
    Will Sargent
    @wsargent
    is there a formal RDF/OWL ontology of logs anywhere?
    it'd make things easier if I could specify a key/value pair and be able to link it to different concepts
    i.e. using an owl-time instant for the logging timestamp https://www.w3.org/TR/owl-time/#time:Instant
    so plain old structured logging for the most part
    but with a @context field and being able to mix and max schemas
    Will Sargent
    @wsargent
    the stuff I'm really interested in is code tracing and debugging data like line number, method and class names, and being able to log DEBUG / TRACE flow as RDF statements
    instead of having to extract it from regex
    Tigran Najaryan
    @tigrannajaryan
    Log SIG meeting is today at 3pm PT. It disappeared from Otel calendar, I will chech why. Zoom link https://zoom.us/j/8203130519
    xukaren
    @xukaren
    Hi everyone, we would like to seek input from the OTEL Community on logging statements for the Logging API in C++. Any suggestions / feedback would be greatly appreciated! The RFC is linked here: open-telemetry/opentelemetry-cpp#356. Thank you!
    Will Sargent
    @wsargent
    @xukaren added comment :-)
    Jean Jordaan
    @jean
    hi everyone .. is the https://opentelemetry.io/docs/collector/ currently usable for logging arbitrary events? E.g. we have a "player" app which should relatively simplistically log when an item starts playback, when it encounters an error, & when it stops playing.
    Tigran Najaryan
    @tigrannajaryan
    @jean yes, a log record is an arbitrary event in OpenTelemetry. To send your app's events to the Collector it needs to be in one of the formats that the Collector already understands or you can add support for your own format by implementing a "receiver" component.
    N
    @noelmcloughlin
    Is there plans for open-telemetry to explore Common Logging Schema?

    Elastic Common Schema (https://www.elastic.co/guide/en/ecs),
    Splunk Schema (https://docs.splunk.com/Documentation/CIM/4.17.0/User/Overview),
    Graylog Schema (https://schema.graylog.org),
    Security Logging Standard still an open question (https://www.scip.ch/en/?labs.20180315)

    https://ieeexplore.ieee.org/document/8587302
    we find the logging practice is adopted highly inconsistently
    among different developers both across projects and even one project
    Conclusion: Both .. have forced us to question whether it is a reliable means to
    understand the runtime behavior of software systems via analyzing the
    logs produced by the current logging practice.

    I guess we pick one schema and they eventually converge towards a standard.
    Max Golovanov
    @maxgolov
    I think Logging Provider / exporter implementation in SDK may export to whatever scheme. It's a developer choice, not enforcing just one schema. There could be systems already using ELK, others don't. We provide exporters for Logstash, for Graylog, for Microsoft Common Schema, and whatever other org schema. It's a huge hassle to adopt a new schema for an organization that has years of prior experience with their own schema, then years migrating to even a slight change in schema revision, not mentioning moving to entirely new schema. Original logging statements in code must be schema-agnostic as much as possible, with exporter in SDK deciding how to pack / in what logging format to export.
    N
    @noelmcloughlin
    I agree. OpenTelemetry is an open-source project. It should support Vendor Common Models (Microsoft, Oracle, Splunk, Graylog, elastic, etc) because the industry invested in these solutions. The provider model is well understood. However, today's Vendor Common Models will eventually be obsoleted - the systems lifecycle. So Industry Common Model (DMTF CIM simplified or replaced) could emerge once todays Vendor CIM go away. Otherwise we get Vendor Common Model V2 in ten years time and Vendor Common Model V3 in 20 years time.
    N
    @noelmcloughlin
    Industry Common Schema may be impossible. But Domain Common Schema (IoT, Cloud, Edge) would be useful for the next generation of system engineers.
    Jean Jordaan
    @jean
    @tigrannajaryan thanks! That is what I expected but the website (/about) is a bit scary: "OpenTelemetry captures metrics, distributed traces, resource metadata, and logs (logging support is incubating now)"
    My guess is that that means logs in a more formal sense than the usage I mentioned.
    Jean Jordaan
    @jean
    Regarding sending events, I guess we might use https://www.github.com/open-telemetry/opentelemetry-js/tree/master/packages%2Fopentelemetry-web .. We're currently using https://github.com/DataDog/browser-sdk -- if the Collector understands the format that sends, we might just point it at the Collector and keep on using it.
    Jean Jordaan
    @jean
    Am I mistaken to be looking for a Collector->Snowplow exporter? Is Snowplow an alternative to the Collector?
    4 replies
    Przemek Maciolek
    @pmm-sumo
    Did the schedule for Log SIG change? I don't see it present in the calendar anymore. Me and @zenmoto got on but there was nobody else present. Didn't have any specific topics for today, just curious if I missed something
    Tigran Najaryan
    @tigrannajaryan
    @pmm-sumo Log SIG meeting is biweekly, it should be in the calendar for the next week, Wed, 3pm PT.
    2 replies
    Pavan
    @pathardepavan
    I looked at https://docs.datadoghq.com/tracing/connect_logs_and_traces/python Tried to add the traceid and spanid in the logs. But the traces and spans generated by opentelemetry and dd are different. Is there a way to get the context traceid/spanid natively in Python ?
    1 reply
    Tigran Najaryan
    @tigrannajaryan
    @/all to allow people from more eastern times to join the SIG meeting I am moving it to an earlier hour, 10am PT starting from Oct 28. If the new time doesn't work for some people we will need to look into alternating times.
    1 reply
    Tigran Najaryan
    @tigrannajaryan
    @/all I am cancelling the Log SIG meeting today. Christian and Przemek can't attend, David and I don't have any topics and I don't see any topics in the agenda doc either. Everybody in Otel is focused on traces and metrics release. Let's meet next week (check the calendar, the time is different).
    Mark
    @MarkSeufert
    Hey guys, I'm working on the Logging SDK and I have ran into a design choice problem. The .NET logging implementation, which I was using as reference, has the LoggerProvider contains a list of processors that all the Logger's use when a logging statement is called. However, this prevents different loggers from writing to different processors. An alternate approach that I'm considering is having each Logger contain its own list of processors, such that different loggers can write to different processors. This would be useful if, for example, we wanted a console logger that only writes to console, and a separate logger that writes to a different destination (ElasticSearch for example). Does anyone have any objections to this second approach? Thanks in advance :)
    1 reply
    Alex Van Boxel
    @alexvanboxel
    I have a question about the Log data model. The usage of Name, in the appendix ( https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/logs/data-model.md#appendix-a-example-mappings ) sometimes Name is describing a single Event (example: EventLogs, CloudTrail, ...) and sometimes a collection of entries (example: Google Cloud Logging). Looking at the description "Short event identifier that does not contain varying parts.", it feels that Google Cloud Logging is wrong... but I see a need to have some kind of grouping key (like : LogName). What's everyone's idea around this?
    6 replies
    Amol
    @Amolstar_twitter
    I have used the Open Telemetry based API's for Metrics collection and tracing. Do we have any sample that can be referred to see how open telemetry based logging can be done? Any pointers/document would help. Thanks.
    5 replies
    Otis Gospodnetić
    @otisg
    Hi. What's considered best practice - separate logs and metrics exporters or one that can export both logs and metrics?
    I see the Logs GA definition at https://github.com/open-telemetry/oteps/blob/master/text/logs/0130-logs-1.0ga-definition.md . Is there a rough estimate for when Logs GA might happen? 2020 or 2021? :)
    1 reply
    ayushmpb95
    @ayushmpb95
    Hi Guys,
    Has it been decided that what logging libraries would be supported per language, for example for golang will it be Zap or logrus etc?
    Also, I understand this is an ongoing project, but would really appreciate if someone gives an idea that it will involve schema validation as well? Or that heavy lifting will be taken on by the collector? Or none of these are requirements.
    3 replies
    Would it also be possible to directly plugin ECS(Elastic Common Schema) for example https://github.com/elastic/ecs-logging-go-zap into first party application?
    Thanks!
    realwoobinlee
    @realwoobinlee
    Hello guys :) just a short question: is there any release date planned for first logging Support? just a rough estimation would be fine (ex. First quarter 2021) Thanks!
    2 replies
    Anurag Gupta
    @agup006
    Hey friends, I see on GitHub the meeting time is 13:00 PT and in the calendar invite it shows as 10:00 PT
    5 replies
    Tigran Najaryan
    @tigrannajaryan
    @/all I will be on PTO starting from tomorrow until Jan 4. I will not attend the SIG meeting on Dec 23. I am not sure what the attendance is going to look like given the holidays, but I will keep the meeting on the calendar in case there is people who want to meet.
    Christian Beedgen
    @kumoroku
    happy holidays! FWIW I will also be out next week. maybe folks can speak up if there's a topic they want to discuss.
    Tigran Najaryan
    @tigrannajaryan
    David is also out, so I think keeping it on the calendar is pointless. I will cancel on Dec 23. Happy holidays and see you all next year!
    Tigran Najaryan
    @tigrannajaryan
    All, here is a separate issue about Telemetry Schema: open-telemetry/opentelemetry-specification#1324
    Please comment.
    gagansingh355
    @gagansingh355

    Hey guys, I wanted to ask has anyone used @Include keyword on fluentbit in otel's main yml file(otel-collector-config.yml)? I getting issue in including a config file, can someone please provide format for this?
    My file for now is:
    extensions:
    fluentbit:
    executable_path: /opt/td-agent-bit/bin/td-agent-bit
    tcp_endpoint: 127.0.0.1:8006
    @INCLUDE /usr/local/go/src/opentelemetry-collector/examples/tracing/temp.conf

    receivers:
    fluentforward:
    endpoint: 0.0.0.0:8006

    gagansingh355
    @gagansingh355
    One more doubt, I have created my custom exporter and fluenbit is extension and fluentforwarder is the receiver. Can someone please confirm in what format will I receive logs at exporter side?
    Bryan Wu
    @bryan-wu-gitter
    Hi folks. I'm looking to obfuscate specific fields in logs. Is there any built-in functionality for obsfucation in the .NET SDK or pre-exisiting solutions I can reference? Thanks