Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Morgan McLean
    @mtwo
    for future meetings
    Tigran Najaryan
    @tigrannajaryan
    @/all based on what Morgan said here is a Doodle for the next SIG meeting shifted to later time: https://doodle.com/poll/pta9dhiscb93nf6i
    For future meetings we will use this for alternate weeks.
    Tigran Najaryan
    @tigrannajaryan
    @/all an OTEP to add Log-related concepts to our vocabulary: open-telemetry/oteps#91
    Wesley Pettit
    @PettitWesley

    Hey all, based on our discussions, I've started a document for housing agent evaluations and debating concerns for logging/telemetry agents.

    https://docs.google.com/document/d/15EMUKw-dv4z-oF28CKh5CZXYJO-5a-dMoev7_Ka-u10/edit#heading=h.6bp9stv65rh3

    I will bring this up in the Log SIG meeting today
    Is there an easy way to add all relevant individuals as editors of a google doc?
    Tristan Sloughter
    @tsloughter
    in case it is useful, I poked the person at Postmates who has been responsible for our logging infrastructure:
    A basic summary on our experiences with logging agents:
    * fluentd: Too slow, couldn't handle our volume at any reasonable resource usage
    * fluentbit: Buggy. They've since fixed their broken JSON handling but there's probably other issues
    * filebeat: Doesn't handle rotation and deletion properly
    All but filebeat also had the issue where it would buffer internally if its sink went down, rather than backpressuring to its source.
    ie. if you're reading from a file, but you can't send the data right now, you read from the file and put it in a new file instead of just...waiting before reading more from the file
    
    IMO agents should be as simple as possible and do the bare minimum to get the data off the node into somewhere more robust (eg. message queue). Then any transforms or extra cross-referencing should be a seperate processing step.
    Wesley Pettit
    @PettitWesley

    fluentbit: Buggy. They've since fixed their broken JSON handling

    That was fixed quite a while ago, so I suspect this experience is a bit old

    I have a large number of enterprise customers at AWS who are using Fluent Bit, for the specific features that we use, I've not heard of any significant complaints.
    Tristan Sloughter
    @tsloughter
    good to know
    I guess it stated with fluent/fluent-bit#1278
    Gregor Zeitlinger
    @zeitlinger
    is there a date for the next zoom meeting already?
    Tigran Najaryan
    @tigrannajaryan
    Yes, Log SIG meetings are on Wednesdays. See OpenTelemetry calendar (time alternates every week).
    Ted Young
    @tedsuo
    Perhaps a naive question, but what’s wrong with syslog? I am old, explain to me why my trusty tools are no longer sufficient.
    8 replies
    Tristan Sloughter
    @tsloughter
    speaking of protocols like syslog. is journald being considered?
    last I knew there was no official publishing support for journald, but it is a structured, compact, binary log format that is already used by most servers these days I'd assume
    2 replies
    Tigran Najaryan
    @tigrannajaryan
    @/all Logs vision is now published. I believe this is very important and a must read for OpenTelemetry Logs contributors: https://github.com/open-telemetry/oteps/blob/master/text/0092-logs-vision.md
    Reiley Yang
    @reyang
    :thumbsup:
    Binary Logic
    @binarylogic

    @tsloughter that reflects our evaluations as well. By the way, Brian Troutwine (creator of cernan) has been advising us on Vector. I'm not sure if you worked with him at Postmates:

    • fluentbit: Buggy. They've since fixed their broken JSON handling but there's probably other issues

    The JSON issues! For a little bit of context, and to not repeat it here, see:

    https://github.com/timberio/vector/issues/727#issuecomment-561004676

    Fluentbit reached 1.1 with those issues đź‘€.

    That was fixed quite a while ago, so I suspect this experience is a bit old

    @PettitWesley we are still able to produce JSON parsing issues, but I agree, the major ones are fixed!

    8 replies
    dlane771
    @dlane771
    In terms of a microservices architecture that used queues to send messages to various parts, is there a way to propagate a trace context via queue messaging. We have used OpenCensus a little bit and can serialize a TraceContext (which is basically the trace id) with which we can then add manually to queue payload. We have (non-http) processors that consume these queues and we would like to re-establish the trace context when processing. However we cannot seem to recreate the context from the serialized trace context. Is there a pattern for this or is this type of functionality not was intended.
    10 replies
    Tigran Najaryan
    @tigrannajaryan
    @/all Log data model draft OTEP: open-telemetry/oteps#97
    Please comment.
    Wesley Pettit
    @PettitWesley
    I read through it. I'm very new to OT, but have a little bit of experience with logs. I really liked the proposed data model and overall thoroughness of the doc. I've not yet been able to think of anything obvious which it seems to be missing.
    Tigran Najaryan
    @tigrannajaryan
    @/all Log SIG meeting tomorrow conflicted with Community meeting. I moved Log SIG meeting to be one hour earlier to avoid the conflict. Calendar is updated.
    Olivier Mengué
    @dolmen
    Hi. New here. I'm just discovering the project as a future user.
    I just wanted to tell about a typo in the meeting notes: the meeting dates are in year 2010 !
    Ron Cohen
    @roncohen
    Hi folks! Is there a recording of the meeting on Apr 8, 2010?
    probably a question for @tigrannajaryan
    Tigran Najaryan
    @tigrannajaryan
    @roncohen AFAIK the recordings are not published automatically (we don't have a way to do it). I believe it is a manual process. @mtwo do you know this?
    Ron Cohen
    @roncohen
    Ah, I know that struggle :) A colleague wrote this tool to automatically upload recordings from recurring zoom meetings to Google Drive folders: https://github.com/graphaelli/zat/ then we just link to the folder with all the recordings for discoverability
    Morgan McLean
    @mtwo
    yeah, it's manual ... apparently Kubernetes has a new way to automate it but I haven't had the time to investigate it
    @roncohen neat, I'll check that out
    Tigran Najaryan
    @tigrannajaryan
    @/all Unfortunately both Morgan and I have a conflict. I am moving today's SIG meeting by 30 minutes so that we can make it. Sorry for last minute notice, we realized too late we have a conflict with Community meeting today and had to reschuffle the schedule.
    Ron Cohen
    @roncohen
    If anyone wants to join early and chit chat I'm there :)
    Eduardo Silva
    @edsiper
    ack
    Madis Liias
    @liias
    Hi there. Quick question. If I am interested in tracing, logging (java applications) in production. Is it too early to consider open-telemetry and I should look into OpenCensus instead?
    Tigran Najaryan
    @tigrannajaryan
    @liias OpenTelemetry tracing is fairly mature and is currently in Beta. I know people who use it at least parts of OpenTelemetry in production (e.g. the Collector). Support for logs does not exist yet. We started discussions to support logs a few weeks ago and I do not expect logs to be production-ready for the next few months.
    rparth
    @rparth
    Folks, glad to join this forum. We are a small group of folks working on a full observability stack at https://logiq.ai. Would be happy to contribute to the logging efforts. Look forward to engaging with the community.
    Tigran Najaryan
    @tigrannajaryan
    @rparth welcome. Feel free to join SIG meetings (meetings are in OpenTelemetry Calendar) and review proposals (recent open proposal is about Data Model: https://docs.google.com/document/d/1ix9_4TQO3o-qyeyNhcOmqAc1MTyr-wnXxxsdWgCMn9c/edit?ts=5e990fe2#)
    Ron Cohen
    @roncohen
    @tigrannajaryan the doc is no longer publicly accessible. Any idea?
    jeffalder
    @jeffalder
    Hi folks! I'm from New Relic. We recently released a logging product. I wrote the Java Agent integrations with logback 1.2, log4j 1, log4j 2, and java.util.logging, and served as a tech lead across the other languages. The Java Agent writes its own logs using log4j 2. I'm a little late to the design process. Let me know how I can contribute.
    Tigran Najaryan
    @tigrannajaryan
    @roncohen I accidentally changed the permission. Should be accessible now.
    @jeffalder you may want to sync with @zenmoto who works on Logging Library spec.
    jeffalder
    @jeffalder
    @zenmoto lmk how I can help!
    jeffalder
    @jeffalder
    @tigrannajaryan Is there a work plan? Once the current OTEP 97 is completed, what needs to get done next? Is there anything that can be done in parallel besides the Library spec?
    Tigran Najaryan
    @tigrannajaryan
    @jeffalder yes, there is a rough plan on what we want to do: https://docs.google.com/document/d/1cX5fWXyWqVVzYHSFUymYUfWxUK5hT97gc23w595LmdM/edit#heading=h.ym810wjhhbxw
    I am personally focused on the data model, the protocol and the Collector. @zenmoto works on library specs. There are several other topics that I think nobody owns yet.
    Wesley Pettit
    @PettitWesley
    Based on last weeks discussion in the SIG, I put up PRs for two suggested edits to the log vision otep.
    Quantifiable resource utilization goals for the unified collector: open-telemetry/oteps#101
    A concrete list of log sources to support: open-telemetry/oteps#102