Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Tigran Najaryan
    @tigrannajaryan
    @murphyzlaw it maybe worth attending the next Log SIG meeting. They are open for anyone to attend. You can also track progress in specification and collector repos (filter issues/PRs by "logs" label). FYI, @zenmoto also works on an OpenTelemtry-compliant Java logging PoC.
    Ron Murphy
    @murphyzlaw
    Thanks for the replies - will join this Wednesday. I have looked at most of the collateral you guys have produced via working documents, gitter, etc. Will take a look at fluent-logger-java and fluent-logger-node.
    Tigran Najaryan
    @tigrannajaryan
    Log SIG meeting tomorrow overlaps with OpenTelemetry Community meeting. I was out for a few days and don't have any updates since the last week. Does anyone have anything to discuss tomorrow? I can reschedule the meeting tomorrow after the Community meeting, but if there is nothing to discuss we can skip this time.
    Tigran Najaryan
    @tigrannajaryan
    Cancelling since I don't see any objections.
    Tigran Najaryan
    @tigrannajaryan
    I added an overview of how logging works with OpenTelemetry to the specs, please review: open-telemetry/opentelemetry-specification#737
    Dr. Anton Chuvakin
    @anton_chuvakin_twitter
    Thanks to whoever invited me here - this is a very fun group, to be sure.
    Greg Shriver
    @gshriver
    Hello to the group - sorry for joining the SIG meeting uninvited. I see now, after reading this chat, that one should be invited before joining. Apologies for the breach of etiquette. I work in the mainframe division at Broadcom, and we have been working on several items that seem to be congruent with OT. First, we have been socializing live streaming of log data from the mainframe to a central sink, combining it with log data from non-mainframe sources, to allow easier log analysis for multi-platform applications, portions of which run on the mainframe. One of our other use cases centers around OT support (in our APM product) for distributed tracing.
    Christian Beedgen
    @kumoroku
    hi Greg––welcome. and, no worries: you don't need an invite!
    Greg Shriver
    @gshriver
    Thanks Christian!
    Tigran Najaryan
    @tigrannajaryan
    @gshriver welcome. OpenTelemetry SIG meetings are open for attendance, there is no need for invitation.
    Tigran Najaryan
    @tigrannajaryan
    @/all a reminder, please review the Logs Overview document: open-telemetry/opentelemetry-specification#737
    It sets the stage for several things that we want to do.
    Daniel Jaglowski
    @djaglowski

    Hi - I’m implementing a logs receiver, and wish to call consumer.ConsumeLogs(ctx context.Context, ld pdata.Logs) error.

    I have a struct representing log records, and wish to convert this to pdata.Logs. However, I’m not finding a way to do this that doesn’t require direct interaction with internal packages.

    Am I right to think this should be possible? Is there an expected approach to this that someone could point me to?

    10 replies
    Tigran Najaryan
    @tigrannajaryan
    @/all there is going to be a demo of another log collection agent tomorrow in Collector's SIG meeting. I have no updates on logs for tomorrow and unless someone has updates or anything to discuss about logs I suggest if you are interested in the demo join the Collector SIG meeting and we cancel the Log SIG this time.
    2 replies
    Avi Kessner
    @avik-so
    What time is the sig meeting?
    Hi, I have a question about logging context data.
    I want all of my logs to include the relevant request-id or session id.
    I also want to be able to log an event from deep within the call stack.
    I also don't want to pass my request or context object into every single function call.
    How is this problem normally solved in async /single-threaded systems like nodejs?
    Tigran Najaryan
    @tigrannajaryan
    Meeting times are in OpenTelemetry calendar: https://github.com/open-telemetry/community#calendar
    1 reply
    You may want to ask the nodejs question in nodejs channel. Logs are no different from traces and metrics from the perspective of context passing and nodejs SIG likely already thought about this.
    2 replies
    Tigran Najaryan
    @tigrannajaryan
    Please review Logs 1.0 GA scope / todo list: open-telemetry/oteps#130
    Andre Guerlain
    @aguerlain-r7
    Is the call ongoing? I don't see anyone in the LOG SIG Zoom
    Tigran Najaryan
    @tigrannajaryan
    Log SIG meeting is at 3pm PT this week. Check OpenTelemetry calendar.
    Andre Guerlain
    @aguerlain-r7
    Ah sorry, timezone issue
    Tigran Najaryan
    @tigrannajaryan
    Does anyone have anything to discuss today? I have been focused on Otel tasks unrelated to logs for the last week and don't have anything new.
    Christian Beedgen
    @kumoroku
    nothing from my end and will also not make it today actually
    Tigran Najaryan
    @tigrannajaryan
    OK, I am going to cancel Log SIG meeting this time since there appears to be nothing to discuss.
    Devon Burriss
    @dburriss
    Hello. Not sure if this is the correct place to ask. Is the logging spec going to be setting any constraints /guidelines on whether logs are lossy? Let me explain in case I am not using correct terminology. Metrics (at least with some push based agents) as I understand send via UTP, logs and maybe some APM via TCP. I also saw some mention in notes about sampling. I would assume of logs that if log 1 and 3 are present I should expect log 2 to be there if the log call was hit. On the sampling side I can see the usefulness of being able to opt-in to sampling if you have a log line when something is hit a million times...
    Tigran Najaryan
    @tigrannajaryan
    There is no sampling specification planned as part of Otel log spec for now. Spec is also independent from the protocol that is used for log transfer. Otel defines its own protocol, but it is only one of the supported protocols and not a mandated one. The reliability of delivery and whether any logs will be or may be lost dependents on many factors of which sampling and the protocol are only a subset. TLDR: the Log spec does not say anything about sampling, reliability or lossiness today. It may be worth exploring this in the future.
    1 reply
    Ron Murphy
    @murphyzlaw
    Question about log processing in OpenTelemetry Collector (I also asked this in the log SIG meeting a few weeks ago). We are considering a path application => OTC => Kafka (possibly with fluentbit tailing in between app and OTC). I found that currently, OTC has a defined exporter for Kafka for traces. Could this be adapted to produce a similar exporter for logs, and how difficult would that be? How much community interest do we anticipate?
    Tigran Najaryan
    @tigrannajaryan
    @murphyzlaw yes, it is possible to extend kafkaexporter to output logs. Should not be very difficult. The primary question will be in what format you write to kafka. kafkaexporter currently supports a couple serialization formats for traces (otlp, jaeger). We will need to decide what to support for logs.
    1 reply
    Daniel Jaglowski
    @djaglowski
    Hi all, tomorrow in the collector SIG, I’ll be demonstrating a golang native logs receiver built around observIQ’s stanza log agent.
    Tigran Najaryan
    @tigrannajaryan
    Does anyone have topics for tomorrow? I am personally focused on completing tracing spec and most of Otel as well. For logs most of the planned work is in Collector for which we have its own meeting. I wonder if we should reduce the frequency of Log SIG meeting to bi-weekly for now. We can increase again in the future if the pace of logging work picks up.
    1 reply
    Tigran Najaryan
    @tigrannajaryan
    I don't see any objections so I am going to cancel this week's Log SIG meeting. Starting from the next week we will meet biweekly (US morning slot which appears to work better for the majority of participants for now). Note that Collector SIG meeting remains weekly and is a good place for topics around Logs support in the Collector.
    Tigran Najaryan
    @tigrannajaryan
    @/all the Log SIG meeting is cancelled today (see above).
    Tigran Najaryan
    @tigrannajaryan
    @mtwo can you please remove the Log meeting from calendar today? ^^^
    Morgan McLean
    @mtwo
    Done!
    Ovidiu
    @ovidiubuligan
    Hello , I am new to this open telemetry space , and I am curious if the future is combining fluentBit with open telementry collector like in this proposal : open-telemetry/opentelemetry-collector#955
    1 reply
    Ovidiu
    @ovidiubuligan
    I have a few questions 1 .Does the collector support windows containers ?
    2'nd: Windows has a tracing subsystem ETW subsystem that we use in one of our windows container microservice . We pipelined the ETW logs to stdout for fluentBit to collect .Now we have traces as stdout. Can we combine this stdout that contains a timestamp as a trace info without changing the app code ?
    Chamila
    @chamilad
    @tigrannajaryan I'd like to contribute to open-telemetry/opentelemetry-collector#1655 as a starting point. Currently looking at https://github.com/open-telemetry/opentelemetry-collector/blob/master/testbed/README.md and will continue from there. Please let me know if I've started down the wrong path.
    Ghost
    @ghost~5b3983f3d73408ce4f9f4600
    Hi all, have the Log SIG meetings recorded? I don't see any recordings posted to the OpenTelemetry Youtube channel.
    1 reply
    uname223
    @uname223
    Hi There, I have a question, as there's ton of discussion regarding different log agents/collectors, is there somewhere a status, or milestones in terms of log format itself?
    4 replies
    Aditya Prajapati
    @Syn3rman
    Hey all,
    I'm interested in participating under community bridge for otel and was going through the issues to find something to work on. Since the project idea is implementing a fluentbit exporter, I was trying to find some newcomer-friendly issues related to either logs or exporter, but most are either assigned, stale, or completed. Can anyone point me to some issues that I can work on to get a feel for the contributing process?
    2 replies
    Przemek Maciolek
    @pmm-sumo
    I was researching the capability to push K8s logs through Fluent bit->OpenTelemetry Collector->any output of choice pipeline and found that it's actually plausible with the recent additions to the collector and a (rather small) modification to a FluentBit Helm chart. I have prepared a doc that summarises it and brings rationale behind some newly added issues (mostly related to K8s Processor extensions): https://docs.google.com/document/d/1QlFbXz0eQUaKXK1WrnEs3VqRkcPD2RAdXJdJMjnOSd8/edit?usp=sharing Would love to hear if that ideas sounds reasonable and if so, will be happy to work on that. I could also demo that during the upcoming SIG meeting and discuss it
    3 replies
    Will Sargent
    @wsargent
    is there a formal RDF/OWL ontology of logs anywhere?
    it'd make things easier if I could specify a key/value pair and be able to link it to different concepts
    i.e. using an owl-time instant for the logging timestamp https://www.w3.org/TR/owl-time/#time:Instant
    so plain old structured logging for the most part
    but with a @context field and being able to mix and max schemas
    Will Sargent
    @wsargent
    the stuff I'm really interested in is code tracing and debugging data like line number, method and class names, and being able to log DEBUG / TRACE flow as RDF statements
    instead of having to extract it from regex