Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Juraci Paixão Kröhling
    @jpkrohling
    @jamiegs jaeger won't give you that kind of analysis, but people have been using it with Kafka as storage and then creating custom consumers to extract the information they need. As you might indeed want an APM from a SaaS provider, you are missing as couple from your list: honeycomb, datadog, splunk, aws ... Perhaps a good source is to go over the list of otel-collector exporters and check which ones are related to commercial vendors? https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/master/exporter
    Steve Flanders
    @flands
    @jamiegs Also note that OpenTelemetry is not GA yet and the spec is still changing. Once GA hits vendors should begin to fully support the spec and its features. If anything is missing on the Splunk/SignalFx side let me know and happy to assist.
    2 replies
    Ghost
    @ghost~60008d26d73408ce4ff8d1b7
    Can someone talk to me about OpenTelemetry as a sidecar, mixed with prometheus receivers (for scraping the container)? I'm losing labels I'm trying to set in the sidecar. My architecture is, a workload deployed in kubernetes with an otel sidecar, the workload either pushes via opencensus/otlp to the sidecar, or the prometheus receiver on the sidecar scrapes the workload directly. Labels are applied under receivers.prometheus.config.global.external_labels. This sidecar then exports via OTLP to a central collector on the cluster that is scraped by a master prometheus instance. I get/can see the metrics, but my external_labels aren't being set. I'm expecting this isn't a supported configuration and the global block in the receivers is just ignored?
    I'm interested to see either a working example using prometheus in this way, or a better way in general.
    Tigran Najaryan
    @tigrannajaryan
    @/all if you are interested in logs please check out the proposal to contribute Stanza to OpenTelemetry with the goal of using it as the log collection library for OpenTelemetry Collector: open-telemetry/community#605
    Ghost
    @ghost~60008d26d73408ce4ff8d1b7
    To my above question, nevermind. I figured out my labeling issue; the labels have to be applied under the static_configs block, the global.external_labels isn't honored here.
    Joshua MacDonald
    @jmacd

    The @opentelemetry Metrics Workshop happens tomorrow at 9:30 PST at https://zoom.us/j/8203130519

    9:30 - 9:45am PST: Opening remarks, organization of the day, how we got here, and the many streams of work. We have an API, language SDKs, a Protocol, a Collector, Receivers, Exporters, Semantic Conventions, and a connection to Tracing [organizer: OTel Committee]

    9:45-10:15am PST: Community building: @opentelemetry has first-class @OpenMetricsIO support, and @PrometheusIO users are first-class users. These projects are meant to get along, and committed to it. [organizer: Alolita Sharma]

    10:15 - 11:00am PST: @opentelemetry Collector deployment models (e.g., Daemonset vs. Statefulset), Agents, Sidecars, and first-class support for Prometheus Remote-Write [organizer: Jaana Dogan]

    11:30 - 12:15pm The Metrics API and Data Model, how we integrated the @opencensusio feature set (Tracing and Metrics, combined!) with the @OpenMetricsIO and StatsD data models. Presenting the six @opentelemetry metric instruments [organizer: Bogdan Drutu]

    12:15 - 13:00pm Histograms! How we’ll support high-resolution and variable-boundary histograms and the connection to sampling metric events [organizers: Josh MacDonald, Michael Gerstenhaber]

    13:30 - 15:00pm Questions and answers

    4 replies
    Devi
    @asldevi
    hello, Is there anyone working on adaptive sampling as described in this issue or differently? jaegertracing/jaeger#365
    7 replies
    Stephen Hong
    @stephenhong
    Hello team, according to Resource SDK, a resource can be associated with the TracerProvider when the TracerProvider is created or it can be associated with the MeterProvider when the MeterProvider is created. So does this mean that an application would need to use both providers in order for the traces and metrics to be linked to each other? I'm understanding this as, by using both providers, the traces and metrics from an application share the same resource attributes and therefore, are able to be linked to each other. Am I correct to assume this?
    5 replies
    taman9333
    @taman9333

    Hello everyone I had a problem when installing OTEL with istio
    I had set b3-propagator as here https://www.npmjs.com/package/@opentelemetry/propagator-b3
    and it works but I get a warning in all my spans " invalid parent span IDs "?

    Screen Shot 2021-01-19 at 4.51.35 PM.png

    5 replies
    Jonatan Ivanov
    @jonatan-ivanov
    Are there plans to enable GH Discussions on https://github.com/open-telemetry/opentelemetry-specification too as it was enabled for other repos?
    Ulrich Romahn
    @uromahn
    Quick question: are people/companies are using OTEL in production or is most of the work just PoC and explorations?
    5 replies
    Trying to figure out if we should go with OTEL or better stick with OpenTracing for now.
    Not sure if this is the right conversation for this question ...
    Naga
    @tannaga
    Hello .. I am writing a custom processor in otel collector. Is there a way to access the content in the request headers? I was looking to extract user agent, x-forwarded-for, etc info from the header
    Juraci Paixão Kröhling
    @jpkrohling
    @tannaga there isn't a way to do it reliably, as an aggregating processor might be running before your processor, like the batch processor
    1 reply
    in other cases, you should have access to the HTTP (or only gRPC, not sure now) metadata contained in the context
    paullly
    @paullly
    Hello guys, we are planning to use opentelemetry in our Program. I was wondering if there is anything like a weaver in order to add traces for every method. Is the Otel tracing designed to be used like that, or would you recommend opening spans in every method manual? Thanks
    1 reply
    Juraci Paixão Kröhling
    @jpkrohling
    it's typically an anti-pattern to trace all method calls -- you probably should use an instrumentation library to get data about your boundaries only (like incoming and outgoing HTTP calls )
    if you need more info, then you instrument the parts that you need to understand better
    paullly
    @paullly
    Thank you very much for your answer! I will look into it!
    arjantop-cai
    @arjantop-cai

    Hi, I have trouble setting up prometheus kubelet/node scraping with otel-collector. I have this config:

    - job_name: k8s/kubelet
        scheme: https
        tls_config:
          ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
          insecure_skip_verify: true
        bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
        kubernetes_sd_configs:
          - role: node
        relabel_configs:
          - action: labelmap
            regex: __meta_kubernetes_node_label_(.+)

    This is what I see in logs:

    2021-01-21T21:43:21.433Z    warn    internal/metricsbuilder.go:104    Failed to scrape Prometheus endpoint    {"component_kind": "receiver", "component_type": "prometheus", "component_name": "prometheus", "scrape_timestamp": 1611265401788, "target_labels": "map[beta_kubernetes_io_arch:amd64 beta_kubernetes_io_instance_type:k3s beta_kubernetes_io_os:linux instance:k3d-dev-server-0 job:k8s/kubelet k3s_io_hostname:k3d-dev-server-0 k3s_io_internal_ip:172.20.0.3 kubernetes_io_arch:amd64 kubernetes_io_hostname:k3d-dev-server-0 kubernetes_io_os:linux node_kubernetes_io_instance_type:k3s node_role_kubernetes_io_master:true]"}
    Bhavesh Mistry
    @bmistry13
    I have OTEL instrumented via Spring Sleuth and OpenTenemetry agent
    but traceid for Spring Sleuth is different from OpenTenemetry agent
    How can I make traceid to be same so they both report same traceid
    arjantop-cai
    @arjantop-cai

    Hi, I have trouble setting up prometheus kubelet/node scraping with otel-collector. I have this config:

    - job_name: k8s/kubelet
        scheme: https
        tls_config:
          ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
          insecure_skip_verify: true
        bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
        kubernetes_sd_configs:
          - role: node
        relabel_configs:
          - action: labelmap
            regex: __meta_kubernetes_node_label_(.+)

    This is what I see in logs:

    2021-01-21T21:43:21.433Z    warn    internal/metricsbuilder.go:104    Failed to scrape Prometheus endpoint    {"component_kind": "receiver", "component_type": "prometheus", "component_name": "prometheus", "scrape_timestamp": 1611265401788, "target_labels": "map[beta_kubernetes_io_arch:amd64 beta_kubernetes_io_instance_type:k3s beta_kubernetes_io_os:linux instance:k3d-dev-server-0 job:k8s/kubelet k3s_io_hostname:k3d-dev-server-0 k3s_io_internal_ip:172.20.0.3 kubernetes_io_arch:amd64 kubernetes_io_hostname:k3d-dev-server-0 kubernetes_io_os:linux node_kubernetes_io_instance_type:k3s node_role_kubernetes_io_master:true]"}

    Had to enable debug logging to see that the error is actually forbidden, my role was missing the node endpoint

    1 reply
    I feel like this should be logged at error log level
    Poonai
    @poonai
    Anyone used tempo for tracing?
    1 reply
    Arsene
    @Tochemey
    Hello folks. I hope we are all doing well. I would like to know whether there is a good tutorial with gRPC.
    7 replies
    Eric Mustin
    @ericmustin
    :wave: is the contrib flavor of the otel-collector packaged in the aws cloudwatch agent now for ec2?
    Stephen Hong
    @stephenhong
    Hello team, I'm searching for any resources for creating a custom exporter for OpenTelemetry collector. I'm trying to send traces and metrics data to a specific destination and I need to create a custom exporter from scratch. This exporter also needs to be able to work with the Otel collector. Are there any good starting points, docs, tutorials, etc?
    4 replies
    Morgan McLean
    @mtwo

    FYI all, I've updated all Zoom meetings to require a password: 77777. This is now embedded into our calendar and meeting notes links, and is referenced on the community GitHub README.md.

    We had to do this after one of the meetings got Zoom bombed! If you access the meetings via the calendar, you won't notice any changes.

    also, we've enabled GitHub discussions on the community repo: https://github.com/open-telemetry/community/discussions
    if you want invites to SIG meetings (with the updated Zoom links) in your calendar (rather than opening a separate one), I recommend joining https://groups.google.com/forum/#!forum/opentelemetry-calendar
    Ted Young
    @tedsuo
    /@all :point_up:
    Emmanuel Courreges
    @ecourreges-orange
    Hi all, has anyone performed performance comparisons for a simple java tomcat api that calls one rest backend?
    I am interested at getting a rough percentage impact of tracing at let's say 1% tracing and 100% tracing with default attributes, with either spring-cloud-sleuth, opentelemetry-java, and opentelemetry-java-instrumentation, the latter being the preferred choice if it is equivalent to the former two.
    This will allow me recommending instrumentation choices and sampling probability for production apps.
    I know you might answer "It depends on the complexity of your code", that's why here I am talking of an almost "pass-through" code, in which instrumenting server and client spans will have the most performance impact, so it's a worst case performance impact. Thanks.
    1 reply
    For example I did a test with jaeger tracing simple requests on an Apache server C++ module Rest API which does one call to cassandra, and the impact of 8 spans per request with 100% tracing was a 50% rise in CPU usage, not for production, but 1% tracing was less than 3% CPU compared to no instrumentation, so that is acceptable for production use
    Stavros Kontopoulos
    @skonto
    Hi all I am exporting some metrics at the collector container side (port 8889) using the prometheus exporter there. However, I dont see any labels coming from my resource set correctly in the controller at my app. I do see the resource labels printed locally in stdout when the metric points are received (debug mod). Any ideas? Is this supported or resource labels are removed?
    Weyert de Boer
    @weyert
    This easycla stuff is not easy at all
    jaguar75
    @jaguar75_twitter
    Hi, is there a way I can get context propagation without really emitting traces?
    28 replies
    Weyert de Boer
    @weyert
    Does anyone know how to opentelemetry logging should work? Doesn't mean I would need to attach events to the current span when my logger class is calling log('message')?
    think of trying to write an instrumentation for winston. I think what I am describing is happening in the log4j but my Java ain't great
    Bhavesh Mistry
    @bmistry13
    HI All, I would like to set attributes for all spans to have global attributes span.setAttribute("ui_Session", "session_uuid"); ? How do I achieve this with OTEL ?
    Bhavesh Mistry
    @bmistry13
    hi All, @WithSpan annotation is only applies to METHOD can we change behavior to applicable to Class to ALL methods will get instrumented it is very annoying to add for each method
    1 reply
    Maarten De Wispelaere
    @BitProcessor
    Hey - We're thinking about integrating either opentracing or open-telemetry into several of our Java projects (and some NodeJS stuff). The latter (Open-Telemetry) seems like the way to go, but I wonder if it's stable enough at this time to use in production projects?
    Juraci Paixão Kröhling
    @jpkrohling
    the tracing part as a whole has been deemed stable some time ago, and the java sdk is a "tier 1" sdk. At this point, I'd say that it's quite safe to use it in terms of API stability
    John Watson
    @jkwatson
    Just FYI, we uncovered a couple small API tweaks we need to make in the Java APIs before 1.0. But, they are small tweaks, and we always call out any breaking changes in our release notes and CHANGELOG.md with every release (and, once we hit 1.0, breaking changes are forbidden).
    Weyert de Boer
    @weyert
    anyone know if you can join the zoom meetings via the browser?
    5 replies
    Bryce Buchanan
    @bryce-b
    Hey @SergeyKanzhelev & @nachoBonafonte, would you all be open to starting a Swift SIG? I'd like to open a forum to discuss development of that project.
    1 reply
    Maarten De Wispelaere
    @BitProcessor
    @jpkrohling @jkwatson Thanks guys!
    Patrick White
    @operations-ivy
    Hello, I am working on a migration from opentracing to opentelemetry, I'm curious if there might be any issues that occur if I'm using the same cassandra cluster backend for opentracing collector and opentelemetry collector?