Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Austin Parker
    @austinlparker
    I know there's a Kubernetes operator for the collector already, but I'm not sure if there's a helm chart. These are good questions! I'd open an issue for them on the collector repo.
    Naseem
    @naseemkullah
    WIll do, thanks!
    Steve Flanders
    @flands
    No helm chart yet — PRs welcomed!
    Naseem
    @naseemkullah
    sounds good! I've created an issue open-telemetry/opentelemetry-collector#752 asking for a helm-charts repo.
    Naseem
    @naseemkullah
    I'm having issues chaining spans together, nginx ingress is using jaeger client to instrument, and can chain to another jaeger instrumented service, but it does not seem to chain with otel instrumented services that also emit spans. Any ideas?
    2 replies
    neith00
    @neith00
    I'm actively trying to better monitor a distributed system which has a message broker between its component. My idea, is to provide a tool to component developers to track the messages that goes through the system. For instance, sometimes during development we (hardly) notice that some messages are lost/dropped/whatever during the process. Could OpenTelemetry help me to build a tool which would give us the status/location of each messages going through our system?
    Austin Parker
    @austinlparker
    @neith00 yes, this would be a good use case for tracing.
    neith00
    @neith00
    @austinlparker I was thinbking using prometheus as backend.
    Austin Parker
    @austinlparker
    ehh... probably? you'd have really high cardinality because you'd need a separate bucket per message.
    neith00
    @neith00
    well I don't think creating a bucket per message is worth unless I delete the bucket right after get rid of the message though
    In this case I should push the metrics to prometheus rather beeing scraped
    13217 Amasia
    @13217amasia
    Where can I find information on OT overhead/throughput loss over different sampling ratios? I understand this might be difficult question to answer but maybe there are some existing benchmarks...
    I also would like to learn if it's possible to control sampling ratio, when an application is just instrumented without any intrusive code changes? E.g. possibly by injecting a sampling ratio parameter to instrumentation parameters?
    Thank you!
    Naseem
    @naseemkullah
    This message was deleted
    2 replies
    Naseem
    @naseemkullah
    @13217amasia regarding sampling rate control: if using opentelemetry collector, sampling could be configured there (see https://github.com/open-telemetry/opentelemetry-collector/tree/master/processor#sampling-processor). If not it would be nice that a sampling rate be able to be passed as env var, I have a PR open for this in the js implementation: open-telemetry/opentelemetry-js#1069
    Ayushi Singh
    @ayusheesingh
    I'm trying to use the azuremonitorexporter but am getting an "invalid instrumentation key" error even though it matches the one I see for my Application Insights resource on Azure Portal - any recommendations on how to fix?
    Nicolas MacBeth
    @Nicolas-MacBeth
    Hey, not sure if this is where should ask this but here goes: It's my understanding that the Prometheus receiver for the OTel collector should just work with any Prometheus exporter, but I'm having issues with https://github.com/prometheus/mysqld_exporter . I get this message logged in the collector: {"level":"info","ts":1591373779.4030387,"caller":"internal/metrics_adjuster.go:357","msg":"Adjust - skipping unexpected point","type":"UNSPECIFIED"}, followed by (i think this is just Stackdriver exporter not happy with the prior error^) {"level":"info","ts":1591373782.0302048,"caller":"scrape/scrape.go:930","msg":"","level":"warn","scrape_pool":"mysql_exporter_otel","target":"http://localhost:9104/metrics","msg":"append failed","err":"[rpc error: code = ResourceExhausted desc = Your metric descriptor quota has been exhausted.; rpc error: code = ResourceExhausted desc = Your metric descriptor quota has been exhausted.; rpc error: code = ResourceExhausted desc = Your metric descriptor quota has been exhausted.; rpc error: code = ResourceExhausted desc = Your metric descriptor quota has been exhausted.; rpc error: code = ResourceExhausted desc = Your metric descriptor quota has been exhausted.]"}
    4 replies
    Does the mysql exporter need to be modified to format the metrics according to the collector, or is there something I'm missing in the collector config?
    dcwangmit01
    @dcwangmit01_twitter

    Hi. I've got comments/questions after a few days of playing with OTel.

    1) Why is UpDownCounter instrument not called Gauge? The six instrument types all have very similar naming components (Counter, UpDownCounter, ValueRecorder, SumObserver, UpDownSumObserver, ValueObserver). I'm sure I'll get used to this over time, but as a newb the naming pattern is hard to spot and requires a lot of spec checking while coding.

    2) The UpDownCounter instrument, which is equivalent to Gauge, is updated via an Add() function. How do I update this counter when I already have the final value? Right now it appears to me that I need to track the prior value, do the math to find the delta, and then call the Add() function with the delta as the value. I would rather just update the metric with the current value. Is there a way to directly set the value for an UpDownCounter instrument?

    3) It took me a while to realize that ValueRecorder and ValueObserver instruments were summary metrics that had multiple values (MinMaxSumCount), and thus would be dropped by the OTel exporter for Prometheus. To me, "Value" suggests a single scalar value. I would have expected the exporter for Prometheus to export the LastValue and just work for Value* instruments. This burned at least a day of debugging for me.

    Thanks all, for this amazing project. I've read some of the spec discussions and the decisions made have all been well considered.

    John Watson
    @jkwatson
    @dcwangmit01_twitter I'm not sure I get why you say that UpDownCounter is the same as a Gauge. From what I know of gauges, they generally only aggregate as the LastValue, not as a sum.
    23 replies
    Mouliksoni
    @Mouliksoni
    Hi, I have a quick question about exporter config (agent side). I'm looking to specify certain attributes to the exporter, I see that there is an way to specify attributes when I create a new instance of the exporter (in my case new new CollectorExporter for nodejs). I was looking for a way to specify the attributes via a config file instead, like how it's possible to do it on the collector side?
    Austin Parker
    @austinlparker
    @Mouliksoni not at this time, although I think there's some interest in such a thing
    3 replies
    Naseem
    @naseemkullah

    What are pros/cons of having a trace begin at a load balancer rather than at the first hop server-side?

    Presumably the trace can begin before that, at the client device, what are pros/cons of having the root span there?

    Would be nice to have a chart listing these three locations where a trace can begin along with pros/cons
    4 replies
    Erik LaBianca
    @easel
    Is there a convention in opentelemtry for exposing the trace id / span id in http responses? I’d like to be able to hit an api endpoint, grab the trace id from the header and drop that into jaeger or whatever and see the trace.
    Austin Parker
    @austinlparker
    i'll give a pretty strong "it depends" here, but if you send trace context to an endpoint then you'd get the trace ID in the headers on the response
    you'd have to look at the instrumentation for the HTTP/gRPC framework - if it injects the trace context into the response even on the root span and you don't have anything stripping those headers, I don't see why not
    John Watson
    @jkwatson
    I think I'd be pretty surprised to see instrumentation inject headers into a response
    Can you show some instrumentation that does that, @austinlparker ?
    Anthony Mirabella
    @Aneurysm9
    the Go net/http instrumentation does, though I'm not positive it's a great idea
    trace context is probably fine, baggage maybe not
    John Watson
    @jkwatson
    what span id does it inject?
    Anthony Mirabella
    @Aneurysm9
    that would be the span ID of the span created by the instrumentation wrapper
    John Watson
    @jkwatson
    is that going to be useful to the caller, I wonder?
    I don't see in that code where it's injecting headers into the response (I also don't really read go, so I might be missing it)
    Anthony Mirabella
    @Aneurysm9
    the first link was to a ResponseWriter wrapper that wraps the WriteHeader method to trap the status code and inject into the outbound Header
    the context it uses is created at the second link, where the instrumentation creates a span for an incoming request
    John Watson
    @jkwatson
    gotcha
    Austin Parker
    @austinlparker
    if there's an incoming span context than the instrumentation will inject it into the response, but its also one of those "well why would it matter" because at that point you've got the trace ID from the outgoing request
    John Watson
    @jkwatson
    I thought that the W3C spec had specifically removed this "backward propagation" case, but I don't know if it explicitly rejects it from being done.
    Austin Parker
    @austinlparker
    i'm also not sure, i'd need to do some research
    John Watson
    @jkwatson
    But, this is only what my probably faulty memory tells me
    Robert Wolujewicz
    @robertwol
    hello Community.
    How one add/include the collector community contrib receivers/exporters to the collector code/image for personal use?
    2 replies
    Lisa Crispin
    @lisacrispin
    The teams in our R&D org are being pushed to get going with OpenTelemetry and observability. A few teams have done several really small experiments with OpenTel, some using Honeycomb but most using ElasticSearch. Our code is C# and JS. (And low code, as the product is a low code platform, but let's not go there yet). Do any of you have recommended links for devs who are total newbies to get going with OpenTel in .Net? I have a list of my own but I'm not a coder and not an expert on this, I'm just trying to support people with good information.
    2 replies
    Amol
    @Amolstar_twitter
    Hi, We have a requirement to store the metrics in Oracle database. Please let me know if we have an exporter for storing this? Any suggestions on how to go about this? Thanks
    Amol
    @Amolstar_twitter
    Hi, Any suggestions on the above query. Thanks.
    John Watson
    @jkwatson
    I don't believe that a metric exporter that writes to a SQL database has been implemented by anyone.
    1 reply
    Chris Young
    @worldofchris
    Hi, I am a complete newbie in the world of open-telemetry. Using the Python libs I have set it up to instrument a video processing workflow which is made up of a number of Google Cloud Functions which communicate with each other via http and Pub/Sub. I love the potential of Open Telemetry and am excited by the results I'm getting with what I've put in place so far. Where in the Open Telemetry community is the best place to share this experience and get feedback on whether I am going in the right direction? Aware that I could be misunderstanding some things / cutting against the grain. Thanks!
    Chris Young
    @worldofchris
    @zeus92bgr is that 'Nf' in response to my question above? Thanks.
    Oh it's been deleted now.
    Andrey Afoninsky
    @afoninsky
    hello all
    I'm trying to deliver traces into two system (jaeger / grafana tempo) in order to play with them, so I use opentelemetry-collector as a receiver
    when I send traces directly to collector - I see them in logs but downstream services don't have them (if I send directly to jaeger everything is ok)
    could you point the way how can I debug it and see where I failed? config and commands I use: https://gist.github.com/afoninsky/7496c4bb89cc461aa33ffeb64318a634
    thank you
    Lucas Harms
    @lmickh
    Are there any examples of using the k8sprocessor on the collector?