Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Alessandro Re
    @akiross

    Hello, I'm trying to get some tracing and metric data into otel collector (on the same host, default port). I'm using tracing for tracing data, and otel for metric data. I managed to get traces into the collector, but apparently I'm failing sending it the metrics.

    The code I'm using is similar to this: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=822f60525e95b24ec0c81a7f25dfa35f

    What am I doing wrong?

    4 replies
    Nisheeth Barthwal
    @nbaztec

    Could someone point me to a documentation on what are the use cases for different meters, as specified here:
    https://docs.rs/opentelemetry/0.14.0/opentelemetry/metrics/index.html

    From examples it seems as if both gauges and histograms can be implemented as ValueRecorders But then there are other things in there like Observers, BoundedCounter, etc. finding it a bit hard to figure out how to simply record a single value like bytes and incoming_requests and the use-cases for other meters.

    Thanks

    1 reply
    Glade Miller
    @glademiller
    I'm trying to use actix-web-opentelemetry and get the trace id and span id in my own middleware. For everything I have tried I always get a 0 trace id and span id is there a way to do this. Additionally I would love for my own middleware to be able to add a span that is in the context of the span from the actix-web-opentelemetry middleware and haven't had any luck making that work.
    Glade Miller
    @glademiller
    Perhaps the question should be phrased as how can I get the a trace id or span id that isn't 0. At what point are those assigned?
    The context for this is I need to return a header with those values in a certain format
    Glade Miller
    @glademiller
    I've narrowed down my problem and it is that the span created by actix-web-opentelemetry is not the current span in my own middleware. I'll be looking at actix's code a bit to see if there is a way to make sure the middleware layer correctly.
    Nate Mara
    @nate-onesignal
    Hello :wave: is it possible to directly set the sampling flag on an existing root-level opentelemetry span context? I'm using opentelemetry with tracing-opentelemetry and want to be able to set certain traces as non-sampled depending on some runtime data.
    9 replies
    Nisheeth Barthwal
    @nbaztec
    Hello, is it necessary to store the reference to ValueObserver created via meter.u64_value_observer.init() in static scope or can it be safely ignored within an underlying function?
    Derek Leverenz
    @derekleverenz
    Hi folks. I noticed a method i was using, https://docs.rs/opentelemetry/0.13.0/opentelemetry/trace/trait.TraceContextExt.html#tymethod.remote_span_context has been removed in 0.14. Is there another way I should be retrieving the remote span context now?
    2 replies
    Alessandro Re
    @akiross

    Is it possible to build a custom aggregator selector using opentelemetry_otlp 0.6.0? new_metrics_pipeline returns a OtlpMetricPipelineBuilder<Selector, ...> but the with_aggregator_selector is bound to Selector and not to a different type, as it happens in version 0.7.0.

    I wanted to use this to get some histogram metrics, but I cannot find a workaround. Can I get histograms in version 0.6?

    2 replies
    Alessandro Re
    @akiross

    for some reason, I cannot get counter instruments to work: I've set up the otel collector to use stdout exporter, and when counter data is exported, I get this:

    2021-06-01T10:15:33.577Z        INFO    loggingexporter/logging_exporter.go:57  MetricsExporter {"#metrics": 1}
    2021-06-01T10:15:33.577Z        DEBUG   loggingexporter/logging_exporter.go:63  ResourceMetrics #0
    InstrumentationLibraryMetrics #0
    InstrumentationLibrary example
    Metric #0
    Descriptor:
         -> Name: example.even
         -> Description:
         -> Unit:
         -> DataType: DoubleSum
         -> IsMonotonic: true
         -> AggregationTemporality: AGGREGATION_TEMPORALITY_CUMULATIVE
    DoubleDataPoints #0
    StartTimestamp: 2021-06-01 10:15:13.557887018 +0000 UTC
    Timestamp: 2021-06-01 10:15:33.559754668 +0000 UTC
    Value: 0.000000

    that is, Value is always zero. This is how I'm using it: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=c81f22d4ae8c428688dddedde55315d9

    21 replies
    Jeroen Vankelecom
    @jeroen-vankelecom
    Hi, I hope my question is at the right place here... We would like to annotate our metrics with traceIDs (exemplars)... looking at the opentelemetry-otlp code, it seems already some structure is in place to store Exemplars, but I'm missing the bigger picture. Any idea if and when exemplars will be supported in OpenTelemetry? Any good reference to how accomplish this would be highly appreciated! Thx
    1 reply
    wise0wl
    @wise0wl:jobmachine.org
    [m]
    Hello! I have an actix http web service that I have instrumented with opentelemetry using actix-web-opentelemetry. Currently it produces prometheus metrics and exports traces with the jaeger exporter. This works, however to do proper correlation I need to have the trace ID in my logs as well. Do any of you have suggestions for instrumenting logging with span and trace id's included?
    2 replies
    justin0mcateer
    @justin0mcateer
    I was wondering if there are plans to add support for logging in the opentelemetry-rust implementation
    wise0wl
    @wise0wl:jobmachine.org
    [m]
    I've seen talk of plans, but it is as yet unimplemented
    Zhongyang Wu
    @TommyCpp
    Yes, we do have plans to add log support. Currently, our main focus is still on tracing size(and metrics once it's stabilized)
    wise0wl
    @wise0wl:jobmachine.org
    [m]
    That did expose things, thank you. The traces and spans id's are all blank---for whatever reason. I'm trying a few things.
    Adrien Jeser
    @ajeser_gitlab

    Hello world !

    I am trying to send traces to Jeager in a Wasm application.

    My code:

    use opentelemetry::global;
    use opentelemetry::sdk::trace::{self, Sampler};
    use opentelemetry::trace::Tracer;
    
    fn main() {
        console_error_panic_hook::set_once();
        global::set_text_map_propagator(opentelemetry_jaeger::Propagator::new());
        let tracer = opentelemetry_jaeger::new_pipeline()
            .with_collector_endpoint("http://localhost:6831")
            .with_service_name("my_app")
            .with_trace_config(trace::config().with_sampler(Sampler::AlwaysOn))
            .install_simple()
            .unwrap();
    
        tracer.in_span("doing_work", |cx| for i in 1..100 {});
    
        global::shutdown_tracer_provider(); // export remaining spans
    }

    The feature "wasm_collector_client" is active on opentelemetry-jaeger.

    No traces are transmitted on the network. Does anyone have an idea?

    1 reply
    Zhongyang Wu
    @TommyCpp
    Hmm that should work. Will take a look today
    Kirill Mironov
    @Drevoed

    Hello everyone! For some reason I get message "unknown service opentelemetry.proto.collector.trace.v1.TraceService" when trying to send data to opentelemetry-collector.

    let tracer = opentelemetry_otlp::new_pipeline()
            .with_endpoint("http://localhost:55680")
            .with_tonic()
            .install_batch(opentelemetry::runtime::Tokio)?;
    
        let telemetry = tracing_opentelemetry::layer().with_tracer(tracer);

    Cargo.toml:

    opentelemetry-otlp = "0.8.0"
    opentelemetry = { version = "0.15.0", features = ["rt-tokio"] }
    tracing-opentelemetry = "0.14.0"

    otel-collector-config.yaml:

    receivers:
      otlp:
        protocols:
          grpc:
    
    exporters:
      loki:
        endpoint: "http://loki:3100/loki/api/v1/push"
        tenant_id: "example"
        labels:
          attributes:
            severity: "severity"
    service:
      pipelines:
        logs:
          receivers: [ otlp ]
          exporters: [ loki ]

    docker-compose otel service:

      otel-collector:
        image: otel/opentelemetry-collector-contrib-dev
        command: [ "--config=/etc/otel-collector-config.yaml" ]
        volumes:
          - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
        ports:
        - "4317:4317"
        - "55680:55680"
        networks:
          - loki
    14 replies
    Otel-collector is up and running that's for sure:
    otel-collector_1  | 2021-07-11T18:33:45.879Z    info    builder/pipelines_builder.go:51 Pipeline is starting... {"pipeline_name": "logs", "pipeline_datatype": "logs"}
    otel-collector_1  | 2021-07-11T18:33:45.879Z    info    builder/pipelines_builder.go:62 Pipeline is started.    {"pipeline_name": "logs", "pipeline_datatype": "logs"}
    otel-collector_1  | 2021-07-11T18:33:45.879Z    info    service/service.go:192  Starting receivers...
    otel-collector_1  | 2021-07-11T18:33:45.879Z    info    builder/receivers_builder.go:70 Receiver is starting... {"kind": "receiver", "name": "otlp"}
    otel-collector_1  | 2021-07-11T18:33:45.879Z    info    otlpreceiver/otlp.go:75 Starting GRPC server on endpoint 0.0.0.0:4317   {"kind": "receiver", "name": "otlp"}
    bbigras
    @bbigras:matrix.org
    [m]
    In jaeger my spans are not linked (like with their parents). I'm using async code with the serenity discord lib which uses tokio and I use async-std in my code. Any ideas why the spans are not linked?
    Zhongyang Wu
    @TommyCpp
    Is the spans in the same host as the parent span or the parent span is in other hosts(like a remote span)?
    bbigras
    @bbigras:matrix.org
    [m]
    same host in the same program. I wonder if serenity does something special to run my async code. Or could a crate like tracing interfere with that?
    Zhongyang Wu
    @TommyCpp
    We do have a good integration with tracing so that should be OK. If it’s the same host the we can rule out propagator problems. Need to look at how serenity to have a better understanding
    bbigras
    @bbigras:matrix.org
    [m]
    I could try to make a minimal example, but that would probably require running serenity and connect a test bot to discord.
    Zhongyang Wu
    @TommyCpp
    That would be helpful. Mind open a issue if you get an example handy?
    bbigras
    @bbigras:matrix.org
    [m]
    I don't have the problem if I use current_with_span instead of current_with_value. What is the difference between the 2 https://github.com/open-telemetry/opentelemetry-rust/blob/55d798e7c238961e4734d345575ee2626fd3a776/examples/async/src/main.rs#L31-L45 ?
    Zhongyang Wu
    @TommyCpp
    So current_with_span will wrap your span into a SynchronizedSpan type. And when you create a new span, it will check and see if the parent context has SynchronizedSpan. That's why current_with_span works but current_with_value doesn't
    bbigras
    @bbigras:matrix.org
    [m]
    Gotcha. Thank you very much. and sorry for the noise.
    Zhongyang Wu
    @TommyCpp
    No worries. It's nice to have some conversations in the channel :smile:
    wise0wl
    @wise0wl:jobmachine.org
    [m]

    Hello! I have most of my apps instrumented with OpenTelemetry metrics (tonic / tower gRPC and Actix HTTP) and am looking to add tracing. The problem I have found is that there doesn't seem to be a good way of adding the TraceId and SpanId to logs, so that I can correlate my access logs with the traces and then follow it through the system.

    I have looked into tracing_subscriber but am not sure if this is the route I should take. All the examples show the ability to turn a trace into a log, but they do not include the IDs, so it's a bit moot.

    Does anyone have any suggestions?

    1 reply
    shekhinah (she/her)
    @she:khinah.xyz
    [m]
    How do I set histogram boundaries for different meters? So far I've only found opentelemetry_prometheus::ExporterBuilder::with_default_histogram_boundaries.
    Jan Steinke
    @jan-xyz
    I think that's a current limitation and is not supported at the moment. You can only set the boundaries globally for all meters with the Prometheus exporter. At least that was the state a few months ago and I am also struggling with it.
    David Lewis
    @davidarmstronglewis
    Hey all, I'm trying to get started with using opentelemetry to instrument some Rust Lambdas I'm running in AWS by pushing traces to DataDog. It looks like the examples are assuming I have an exporter running as a sidecar container reachable on the local network. That doesn't make as much sense in the context of AWS Lambda. Does anyone have any suggestions of where to start or where I can look for guidance? Thanks all <3
    1 reply
    wise0wl
    @wise0wl:jobmachine.org
    [m]

    Hello! I have an interesting situation. I am attempting to instrument logging my trace id for my Actix based services and have tried both using the Actix Logger middleware directly with custom_request_replace to inject a trace id, and by creating my own Logger middleware based directly off theirs. In both cases they can add a trace id to my logs. HOWEVER, later on in the actual HTTP route handler, when I use #[instrument] my trace id is different.

    I have looked at and used the Actix tracing library actix-web-opentelemetry which does fully work, and if I add a logging call in that library's Service call() method I get a consistent trace id. My question is, why? What is causing my trace scope or context to not be available in my later http route handlers?

    The only difference that I see between the two libraries Service implementations is that the actix-web-opentelemetry returns a Box::pin future where the request is await'd and the Actix Logger returns a struct with the bare future as a member.
    Josh Triplett
    @joshtriplett

    I'm trying to migrate from opentelemetry-rust 0.14 to 0.16. I'm using OTLP HTTP. In 0.14, I had this:

            opentelemetry_otlp::TraceExporter::new_http(
                opentelemetry_otlp::ExporterConfig {
                    endpoint: "https://api.honeycomb.io/v1/traces".to_string(),
                    protocol: opentelemetry_otlp::Protocol::HttpBinary,
                    ..Default::default()
                },
                opentelemetry_otlp::HttpConfig {
                    headers: Some(headers),
                    ..Default::default()
                },
            )

    But in 0.16, HttpConfig doesn't seem to be public anymore, so I can't create one to call new_http. I tried creating an HttpExporterBuilder, but once I'm done building, I don't see how to get from there to a SpanExporter, because SpanExporterBuilder::build_span_exporter is private.

    What's the supported way to build a SpanExporter in 0.16?
    cqx820
    @cqx820:matrix.org
    [m]

    Hi everyone,

    I'm trying to send trace data to Google Cloud Trace, I tried a python script with my_key.json and it's working for me, but I'm unable to do the same with rust. The Cargo.toml looks like

    opentelemetry = { version = "0.16.0", default_features = false, features = ["rt-tokio", "rt-async-std", "rt-tokio-current-thread", "trace", "metrics"] }
    opentelemetry-stackdriver = { version = "0.12.0", default_features = false, features = ["gcp_auth", "yup-authorizer"] }
    tracing = "0.1.28"
    tracing-futures = "0.2.5"
    tracing-opentelemetry = "0.15.0"
    tracing-subscriber = "0.2.24"

    My workflow is like this:

    1. Get exporter
      let handle = tokio::runtime::Handle::current();
      let spawner = tokio_adapter::TokioSpawner::new(handle);
      let exporter = StackDriverExporter::connect(
       GcpAuthorizer::new().await.unwrap(), &spawner, None, None).await.unwrap();  // Tried YupAuthorizer as well, didn't work too.
    2. Get tracer
      let tracer = sdk::trace::TracerProvider::builder()
         .with_simple_exporter(exporter)
         .build()
         .tracer("tracer_name", Some(VERSION));
    3. Get subscriber
      let subscriber = tracing_subscriber::registry().with(tracing_opentelemetry::layer().with_tracer(tracer));
    4. Set global default
      tracing::subscriber::set_global_default(subscriber);
    5. Call tracing::span!(tracing::Level::TRACE, span_name, ...)

    Above is my entire workflow, after running I can't find any traces on Google Cloud Trace, I'm sure the cred file has enough permission. I've also tried raising the trace levels but no lucks. There were no runtime errors found. I wonder if I'm doing wrong with getting the exporter and the tracer.

    I also wrote some sample codes below just for testing, but they didn't work for me too

    let exporter = StackDriverExporter::connect(
        GcpAuthorizer::new().await.unwrap(),
        &spawner, Some(Duration::from_secs(5)), Some(5usize),
      ).await.unwrap();
    
    let tracer = sdk::trace::TracerProvider::builder()
          .with_simple_exporter(exporter)
          .build()
          .tracer("tracer_name", Some(VERSION));
    
    let attributes =
          vec![KeyValue::new("version", "0.0.0")];
    
    let _span = tracer
            .span_builder("my_test_span")
            .with_attributes(attributes)
            .start(&tracer);
    
    tracer.with_span(_span, |cx| {
          sleep(Duration::from_millis(500));
        });

    Any suggestions would be appreciated, thank you very much!!

    I also found another issue about the YupAuthorizer, I provided a path for persistent_token_file, but I found it's not persisting the token to disk too.
    Jan Steinke
    @jan-xyz:matrix.org
    [m]
    Good morning (in Europe and Africa), I filed yesterday a PR to bump the Prometheus dependency and noticed that some of the examples are pinned to old versions, while others use path directives. Would you be interested in a PR which changes everything to path directives so that the examples don't go out of sync?
    1 reply
    Cecile Tonglet
    @cecton

    Hello I have a question about TraceState.

    Context: If I understand right I should pass a HTTP header "tracestate" to the sub services I call using this. (So I can see the trace across multiple services.)

    More context: I'm using tokio's tracing with opentelemetry-rust and jaeger to vizualize. I do see the spans I created with tracing in jaeger. Everything seems to be working for now.

    I want to make this trace state header for the subservices. So I do this before calling the subservice:

    let trace_state_header = opentelemetry::trace::get_active_span(|span| {
        let span_context = span.span_context();
        span_context.trace_state().header()
    });

    The problem is that I always get an empty string. The SpanRef's debug looks like this: SpanRef(SynchronizedSpan { span_context: SpanContext { trace_id: TraceId(0), span_id: SpanId(0), trace_flags: TraceFlags(0), is_remote: false, trace_state: TraceState(None) }, inner: None }) which doesn't look great (TraceState(None) looks wrong).

    I have the impression that there is a difference between tracing's span and opentelemetry's span. Maybe I'm missing a step somewhere?

    6 replies
    altanozlu
    @altanozlu:matrix.org
    [m]
    Hi every one, i would like to ask why there isn’t another repository for contrib modules like go and python does
    1 reply
    Matteo Giachino
    @matteosister

    Hello everybody. I'm trying to have a "version" tag for every span in our rust service. Right now we are using the opentelemetry-zipkin exporter, but we tried also with jaeger, and otel with the same result.
    We are doing:

    let trace_config = trace::config().with_resource(Resource::new(vec![
        KeyValue::new("version", version),
    ]));
    let tracer = opentelemetry_zipkin::new_pipeline()
            .with_collector_endpoint("http://otel_url")
            .with_service_name("app_name")
            .with_trace_config(trace_config)
            .install_batch(opentelemetry::runtime::TokioCurrentThread)
            .expect("failed to install opentelemetry_zipkin exporter pipeline");

    The problem is that I don't see the version tag in the reported spans. Is this the correct way to do this?

    7 replies
    Jan Steinke
    @jan-xyz:matrix.org
    [m]
    You will need to add the tag to each span separately
    gsson
    @gsson:matrix.org
    [m]

    👋
    Hello! OpenTelemetry/tracing noob here!

    I'm trying to use opentelemetry-prometheus to add some metrics to my app.

    I would like to add request information as attributes to to the meters when recording the response metrics, but I can't quite figure out how. I've played around a bit with span fields or attributes but no luck. I my test I'm using the new and shiny axum server, but I guess I would run into similar issues everywhere.

    I have created a very simple tower_http::trace::TraceLayer like so:

    .layer(TraceLayer::new_for_http()
        .on_response(move |response: &Response<_>, latency: Duration, _span: &Span| {
            let attributes = [
                KeyValue::new("status", response.status().as_str().to_string()),
                // KeyValue::new("path", ???),
                // KeyValue::new("method", ???),
            ];
            server_request_recorder.record(latency.as_secs_f64(), &attributes);
        },
    ));

    Can anyone help me fill in the blanks? :)

    I put the full minimal-ish example here: https://github.com/gsson/mini-web-rs

    9 replies
    Matteo Giachino
    @matteosister
    Is there a way to add some kind of hook to tracing-subscriber to add a custom tag to every span generated?
    3 replies
    bbigras
    @bbigras:matrix.org
    [m]
    I got OpenTelemetry trace error occurred. Exporter jaeger encountered the following error(s): thrift agent failed with message too long. Any way to dump the offending message?
    bbigras
    @bbigras:matrix.org
    [m]