by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Omer Katz
    @thedrow
    One of the things I'm wondering about is how we'd model logging inside the application.
    Does the SDK provide a logging mechanism or should we use whatever we'd like and the collector will collect it from files/journald etc.
    4 replies
    Rahul M Chheda
    @rahulchheda
    Hi, would like to contribute to these efforts, went through the previous meeting agendas and results. Could somebody point me towards the repo, or github link for OTEP for logging?
    2 replies
    haojhcwa
    @haojhcwa
    Just like to share a small tool we used to benchmark our various log agent implementation, it is still a work in progress with many features pending, feel free to contribute or share your opnion: https://github.com/awslabs/amazon-log-agent-benchmark-tool
    Jonah Kowall
    @jkowall
    The PR for the spec seems like there are a bunch of vendor specific things. Why do Splunk need SignalFX events and Splunk events? Same with the google/amazon log fields, we could make those generic versus specific.
    Tristan Sloughter
    @tsloughter
    hm? that isn't the case. also the otep PR is not the spec, it is the process before a spec is written
    ah saw your comments on the PR. don't think you should worry about getting in a logz.io mapping/example to the OTEP. -- unless you think it shows something that would require changing the otep, but the otep isn't the spec so the spec isn't in stone after its merger
    I didn't bother sending an Erlang mapping like there is for log4j and zap because I don't think it would add anything to the otep, but there certainly will be one when the spec is complete
    Tigran Najaryan
    @tigrannajaryan
    Mark Carter
    @markcartertm_twitter
    @tigrannajaryan thank you for all the amazing work and your leadership πŸ‘πŸ»πŸ™πŸ»
    Rahul M Chheda
    @rahulchheda
    Congrats @tigrannajaryan ! That's great!
    Omer Katz
    @thedrow
    :tada:
    Tigran Najaryan
    @tigrannajaryan
    Thanks everyone for reviewing and providing valuable feedback. Next: logging libraries, log protocol and log data support in Collector.
    Reece Bradley
    @reecebradley
    πŸŽˆπŸŽ‰
    Tigran Najaryan
    @tigrannajaryan
    @/all I have to move the SIG meeting time today to 11am PT avoid the overlap with OpenTelemetry Community meeting.
    Pablo
    @voomik
    I am not sure if this is the correct location to ask this question; however, are there specific individuals that are allowed to participate in the Log SIG meetings? I am looking to listen on the calls, but I am open to contributing. TIA!
    Christian Beedgen
    @kumoroku
    calls are open, please do dial in!
    Pablo
    @voomik
    Fantastic. I will join :-)
    jeffalder
    @jeffalder
    What’s the room for the 11am? Still https://zoom.us/j/8203130519 ?
    Pablo
    @voomik
    You were in the right room @jeffalder . Meeting ended already.
    Leighton Chen
    @lzchen
    Wasn't the meeting mved to 11am?
    Pablo
    @voomik
    Yes 11am PT
    Omer Katz
    @thedrow
    So how come it already ended?
    Or maybe I got the timezones confused
    Pablo
    @voomik
    Seems like the updates were light? That was my first time attending, so I am not familiar with the procedures.
    Christian Beedgen
    @raychaser
    yes, this was a very quick meeting today! we often go for the full hour
    generally, check out the meeting agenda/notes doc:
    https://docs.google.com/document/d/1cX5fWXyWqVVzYHSFUymYUfWxUK5hT97gc23w595LmdM ––read backwards a bit and it will hopefully provide a bunch of context
    as Tigran said, getting the Log Data Model approved was a big step that will now allow some implementation to start https://github.com/open-telemetry/oteps/blob/master/text/0097-log-data-model.md
    Omer Katz
    @thedrow
    But we need to decide how we want to do this
    Do any of you know eliot? The Python structured logging library?
    Tigran Najaryan
    @tigrannajaryan
    @/all please review a tiny amendment to Log Data Model: open-telemetry/oteps#109
    Omer Katz
    @thedrow

    Do any of you know eliot? The Python structured logging library?

    I'd like to start discussing the API for our logging SDK

    Anyone up for it?
    Avi Kessner
    @avik-so
    I'm starting to define an internal project for logging, and I was hoping to be able to take advantage of open telemetry. I'd be happy to participate in the logging API.
    Omer Katz
    @thedrow
    @avik-so That's great.
    Anyone else?
    28 replies
    Tristan Sloughter
    @tsloughter
    would opentelemetry logs encompass error/exception reporting? I feel like it is the same data just maybe represented differently
    4 replies
    Christian Beedgen
    @kumoroku
    regarding defining a logging API: when the SIG started in late March, this was discussed. see for example the notes for March 25 and April 8––the group at that point decided to focus on integrating with existing logging libraries first, then tackle the logging API. the overall sense was that most projects already had integrated with a library (log4j, ...) and forcing the use of yet something else was going to be an uphill battle.
    Omer Katz
    @thedrow
    I somewhat agree
    But new projects would probably use our API
    Christian Beedgen
    @kumoroku
    I think this is sound logic, but I will also say that personally, I'd love to see OT as a one-stop shop for all telemetry instrumentation, and as you are saying, what about the next 100,000 projects? :)
    personally, I think if there's critical mass around defining a logging API among folks participating here, nobody should keep anybody from making progress there, and I do not think the original prioritization was meant to prevent this. it was just that the folks being able to spend time on logging within OT in general at that time wanted to dig into integrating with existing first
    I think it is a fine topic to discuss at the SIG meeting
    Omer Katz
    @thedrow
    I agree
    Joshua MacDonald
    @jmacd
    I think everyone likes the idea of a logging API, but the people who've been working on this project since 2019 are more interested in finishing the tracing and metrics APIs. I don't think you'll get full attention on logging APIs until then. (P.S. I've paid zero attention to the logging SIG because I'm in this boat, I've got a metrics API to finish--and yet I've designed and implemented numerous logging APIs in C++ and Go.) I'd be happy if all the people interested in logging would pitch in on tracing and metrics!
    Steve Flanders
    @flands
    2019 member here β€” I do not think it is a reasonable expectation that everyone be interested in every aspect of the project. Plenty of people only care about a subset of observability data. Also, as the community grows we should leverage all the skillsets available β€” we have many new additions who are primarily interested in logs including AWS, Elastic, and SumoLogic. While it would be great to be involved in every aspect that is not feasible as the project grows. Of course we should not slow down on the aspects on the path to GA being traces and metrics and @mtwo already covered the goal is to GA later this year with logs likely entering beta.
    10 replies
    Joshua MacDonald
    @jmacd
    HI all, I've been thinking about how this Logging sub-project could help more with tracing and metrics. I was thinking that before beginning to work on Logging API design, we could try to standardize the translation from Span and Metric events into structured logging events. This was requested as a feature for the metric SDK (open-telemetry/opentelemetry-specification#617), and I think the same could be said of spans. Right now many OTel repos are including standard-output exporters for both trace and metrics, but there are no standard conventional data formats for them to use, so it's a bit arbitrary what you get. I think it would be great if we had a standard structured log form for both metrics and spans.
    I would add that the OTel metrics API was conceived as a logging API from the start. The data model describes metric events, which can be recorded as a faithful way to capture metrics, so we would just need to standardize how they are structured as log records. Here's the event structure: https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/metrics/api.md#metric-event-format
    Tigran Najaryan
    @tigrannajaryan
    @jmacd do you mean a standardized way to encapsulate span data and metric data inside a log record or something else?