Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    bbigras
    @bbigras:matrix.org
    [m]
    I don't have the problem if I use current_with_span instead of current_with_value. What is the difference between the 2 https://github.com/open-telemetry/opentelemetry-rust/blob/55d798e7c238961e4734d345575ee2626fd3a776/examples/async/src/main.rs#L31-L45 ?
    Zhongyang Wu
    @TommyCpp
    So current_with_span will wrap your span into a SynchronizedSpan type. And when you create a new span, it will check and see if the parent context has SynchronizedSpan. That's why current_with_span works but current_with_value doesn't
    bbigras
    @bbigras:matrix.org
    [m]
    Gotcha. Thank you very much. and sorry for the noise.
    Zhongyang Wu
    @TommyCpp
    No worries. It's nice to have some conversations in the channel :smile:
    wise0wl
    @wise0wl:jobmachine.org
    [m]

    Hello! I have most of my apps instrumented with OpenTelemetry metrics (tonic / tower gRPC and Actix HTTP) and am looking to add tracing. The problem I have found is that there doesn't seem to be a good way of adding the TraceId and SpanId to logs, so that I can correlate my access logs with the traces and then follow it through the system.

    I have looked into tracing_subscriber but am not sure if this is the route I should take. All the examples show the ability to turn a trace into a log, but they do not include the IDs, so it's a bit moot.

    Does anyone have any suggestions?

    1 reply
    shekhinah (she/her)
    @she:khinah.xyz
    [m]
    How do I set histogram boundaries for different meters? So far I've only found opentelemetry_prometheus::ExporterBuilder::with_default_histogram_boundaries.
    Jan Steinke
    @jan-xyz
    I think that's a current limitation and is not supported at the moment. You can only set the boundaries globally for all meters with the Prometheus exporter. At least that was the state a few months ago and I am also struggling with it.
    David Lewis
    @davidarmstronglewis
    Hey all, I'm trying to get started with using opentelemetry to instrument some Rust Lambdas I'm running in AWS by pushing traces to DataDog. It looks like the examples are assuming I have an exporter running as a sidecar container reachable on the local network. That doesn't make as much sense in the context of AWS Lambda. Does anyone have any suggestions of where to start or where I can look for guidance? Thanks all <3
    1 reply
    wise0wl
    @wise0wl:jobmachine.org
    [m]

    Hello! I have an interesting situation. I am attempting to instrument logging my trace id for my Actix based services and have tried both using the Actix Logger middleware directly with custom_request_replace to inject a trace id, and by creating my own Logger middleware based directly off theirs. In both cases they can add a trace id to my logs. HOWEVER, later on in the actual HTTP route handler, when I use #[instrument] my trace id is different.

    I have looked at and used the Actix tracing library actix-web-opentelemetry which does fully work, and if I add a logging call in that library's Service call() method I get a consistent trace id. My question is, why? What is causing my trace scope or context to not be available in my later http route handlers?

    The only difference that I see between the two libraries Service implementations is that the actix-web-opentelemetry returns a Box::pin future where the request is await'd and the Actix Logger returns a struct with the bare future as a member.
    Josh Triplett
    @joshtriplett

    I'm trying to migrate from opentelemetry-rust 0.14 to 0.16. I'm using OTLP HTTP. In 0.14, I had this:

            opentelemetry_otlp::TraceExporter::new_http(
                opentelemetry_otlp::ExporterConfig {
                    endpoint: "https://api.honeycomb.io/v1/traces".to_string(),
                    protocol: opentelemetry_otlp::Protocol::HttpBinary,
                    ..Default::default()
                },
                opentelemetry_otlp::HttpConfig {
                    headers: Some(headers),
                    ..Default::default()
                },
            )

    But in 0.16, HttpConfig doesn't seem to be public anymore, so I can't create one to call new_http. I tried creating an HttpExporterBuilder, but once I'm done building, I don't see how to get from there to a SpanExporter, because SpanExporterBuilder::build_span_exporter is private.

    What's the supported way to build a SpanExporter in 0.16?
    cqx820
    @cqx820:matrix.org
    [m]

    Hi everyone,

    I'm trying to send trace data to Google Cloud Trace, I tried a python script with my_key.json and it's working for me, but I'm unable to do the same with rust. The Cargo.toml looks like

    opentelemetry = { version = "0.16.0", default_features = false, features = ["rt-tokio", "rt-async-std", "rt-tokio-current-thread", "trace", "metrics"] }
    opentelemetry-stackdriver = { version = "0.12.0", default_features = false, features = ["gcp_auth", "yup-authorizer"] }
    tracing = "0.1.28"
    tracing-futures = "0.2.5"
    tracing-opentelemetry = "0.15.0"
    tracing-subscriber = "0.2.24"

    My workflow is like this:

    1. Get exporter
      let handle = tokio::runtime::Handle::current();
      let spawner = tokio_adapter::TokioSpawner::new(handle);
      let exporter = StackDriverExporter::connect(
       GcpAuthorizer::new().await.unwrap(), &spawner, None, None).await.unwrap();  // Tried YupAuthorizer as well, didn't work too.
    2. Get tracer
      let tracer = sdk::trace::TracerProvider::builder()
         .with_simple_exporter(exporter)
         .build()
         .tracer("tracer_name", Some(VERSION));
    3. Get subscriber
      let subscriber = tracing_subscriber::registry().with(tracing_opentelemetry::layer().with_tracer(tracer));
    4. Set global default
      tracing::subscriber::set_global_default(subscriber);
    5. Call tracing::span!(tracing::Level::TRACE, span_name, ...)

    Above is my entire workflow, after running I can't find any traces on Google Cloud Trace, I'm sure the cred file has enough permission. I've also tried raising the trace levels but no lucks. There were no runtime errors found. I wonder if I'm doing wrong with getting the exporter and the tracer.

    I also wrote some sample codes below just for testing, but they didn't work for me too

    let exporter = StackDriverExporter::connect(
        GcpAuthorizer::new().await.unwrap(),
        &spawner, Some(Duration::from_secs(5)), Some(5usize),
      ).await.unwrap();
    
    let tracer = sdk::trace::TracerProvider::builder()
          .with_simple_exporter(exporter)
          .build()
          .tracer("tracer_name", Some(VERSION));
    
    let attributes =
          vec![KeyValue::new("version", "0.0.0")];
    
    let _span = tracer
            .span_builder("my_test_span")
            .with_attributes(attributes)
            .start(&tracer);
    
    tracer.with_span(_span, |cx| {
          sleep(Duration::from_millis(500));
        });

    Any suggestions would be appreciated, thank you very much!!

    I also found another issue about the YupAuthorizer, I provided a path for persistent_token_file, but I found it's not persisting the token to disk too.
    Jan Steinke
    @jan-xyz:matrix.org
    [m]
    Good morning (in Europe and Africa), I filed yesterday a PR to bump the Prometheus dependency and noticed that some of the examples are pinned to old versions, while others use path directives. Would you be interested in a PR which changes everything to path directives so that the examples don't go out of sync?
    1 reply
    Cecile Tonglet
    @cecton

    Hello I have a question about TraceState.

    Context: If I understand right I should pass a HTTP header "tracestate" to the sub services I call using this. (So I can see the trace across multiple services.)

    More context: I'm using tokio's tracing with opentelemetry-rust and jaeger to vizualize. I do see the spans I created with tracing in jaeger. Everything seems to be working for now.

    I want to make this trace state header for the subservices. So I do this before calling the subservice:

    let trace_state_header = opentelemetry::trace::get_active_span(|span| {
        let span_context = span.span_context();
        span_context.trace_state().header()
    });

    The problem is that I always get an empty string. The SpanRef's debug looks like this: SpanRef(SynchronizedSpan { span_context: SpanContext { trace_id: TraceId(0), span_id: SpanId(0), trace_flags: TraceFlags(0), is_remote: false, trace_state: TraceState(None) }, inner: None }) which doesn't look great (TraceState(None) looks wrong).

    I have the impression that there is a difference between tracing's span and opentelemetry's span. Maybe I'm missing a step somewhere?

    6 replies
    altanozlu
    @altanozlu:matrix.org
    [m]
    Hi every one, i would like to ask why there isn’t another repository for contrib modules like go and python does
    1 reply
    Matteo Giachino
    @matteosister

    Hello everybody. I'm trying to have a "version" tag for every span in our rust service. Right now we are using the opentelemetry-zipkin exporter, but we tried also with jaeger, and otel with the same result.
    We are doing:

    let trace_config = trace::config().with_resource(Resource::new(vec![
        KeyValue::new("version", version),
    ]));
    let tracer = opentelemetry_zipkin::new_pipeline()
            .with_collector_endpoint("http://otel_url")
            .with_service_name("app_name")
            .with_trace_config(trace_config)
            .install_batch(opentelemetry::runtime::TokioCurrentThread)
            .expect("failed to install opentelemetry_zipkin exporter pipeline");

    The problem is that I don't see the version tag in the reported spans. Is this the correct way to do this?

    7 replies
    Jan Steinke
    @jan-xyz:matrix.org
    [m]
    You will need to add the tag to each span separately
    gsson
    @gsson:matrix.org
    [m]

    👋
    Hello! OpenTelemetry/tracing noob here!

    I'm trying to use opentelemetry-prometheus to add some metrics to my app.

    I would like to add request information as attributes to to the meters when recording the response metrics, but I can't quite figure out how. I've played around a bit with span fields or attributes but no luck. I my test I'm using the new and shiny axum server, but I guess I would run into similar issues everywhere.

    I have created a very simple tower_http::trace::TraceLayer like so:

    .layer(TraceLayer::new_for_http()
        .on_response(move |response: &Response<_>, latency: Duration, _span: &Span| {
            let attributes = [
                KeyValue::new("status", response.status().as_str().to_string()),
                // KeyValue::new("path", ???),
                // KeyValue::new("method", ???),
            ];
            server_request_recorder.record(latency.as_secs_f64(), &attributes);
        },
    ));

    Can anyone help me fill in the blanks? :)

    I put the full minimal-ish example here: https://github.com/gsson/mini-web-rs

    9 replies
    Matteo Giachino
    @matteosister
    Is there a way to add some kind of hook to tracing-subscriber to add a custom tag to every span generated?
    3 replies
    bbigras
    @bbigras:matrix.org
    [m]
    I got OpenTelemetry trace error occurred. Exporter jaeger encountered the following error(s): thrift agent failed with message too long. Any way to dump the offending message?
    bbigras
    @bbigras:matrix.org
    [m]
    Eric Kidd
    @emk
    Good morning. I've been trying to figure out how to use tracing-opentelemetry to integrate a Rust app with an external trace. This mostly seems to work, but I can't figure out how to add new keys to the TraceState. It appears that tracing fields and OpenTelemetry attributes get added to events, like I'd expect, but I haven't figured out how to add a key to the TraceState, which is separate from OpenTelemetry attributes. Am I missing something obvious here?
    1 reply
    bbigras
    @bbigras:matrix.org
    [m]
    @emk: by "remote context" do you mean you extracted the trace id from http headers?
    1 reply
    I have no idea what tracestat and baggage are yet. Still have things to learn.
    Eric Kidd
    @emk
    Clearly, I do, too.
    bbigras
    @bbigras:matrix.org
    [m]

    if you need to extract the trace stuff from headers, here's how I do it:

    let parent_cx = global::get_text_map_propagator(|propagator| {
        propagator.extract(&HeaderExtractor(&headers))
    });
    
    let tracer = global::tracer("example");
    let span = tracer
        .span_builder("something")
        .with_kind(SpanKind::Server)
        .with_parent_context(parent_cx)
        .start(&tracer);

    I'm using global::set_text_map_propagator(TraceContextPropagator::new()); on the client side.

    Josh Triplett
    @joshtriplett

    I'm trying to migrate from opentelemetry-rust 0.14 to 0.16. I'm using OTLP HTTP. In 0.14, I had this:

            opentelemetry_otlp::TraceExporter::new_http(
                opentelemetry_otlp::ExporterConfig {
                    endpoint: "https://api.honeycomb.io/v1/traces".to_string(),
                    protocol: opentelemetry_otlp::Protocol::HttpBinary,
                    ..Default::default()
                },
                opentelemetry_otlp::HttpConfig {
                    headers: Some(headers),
                    ..Default::default()
                },
            )

    But in 0.16, HttpConfig doesn't seem to be public anymore, so I can't create one to call new_http. I tried creating an HttpExporterBuilder, but once I'm done building, I don't see how to get from there to a SpanExporter, because SpanExporterBuilder::build_span_exporter is private.
    What's the supported way to build a SpanExporter in 0.16?

    Josh Triplett
    @joshtriplett
    @jtescher found and pushed a solution, thank you!
    Mark Van de Vyver
    @taqtiqa-mark
    Hi, I'm trying to get my head around the Rust debugging story. In particular around the using the tracing crate with Jaeger. As best I understand, this means coming to grips with how these crates hang together: tracing -> tracing-subscriber -> tracing-opentelemetry -> opentelemetry - opentelemetry-tracing. Before clogging this channel, is there any doc/blog giving a high level over view? Or even lower level overview of the relations and intended uses?
    3 replies
    Mark Van de Vyver
    @taqtiqa-mark
    If not, one immediate question is: Should I expect the Tracing crates #[instrument] in a 3rd party library to generate OT spans to Jaeger? Assuming my app has somthing like the following working:
    async fn main() {
        // Build a Jaeger batch span processor
        // Setup Tracer Provider
        // Get new Tracer from TracerProvider
        // Create a layer ( `layer`) with the configured Tracer
         tracing_subscriber::registry()
                            .with(layer)
                            .try_init()
                            .expect("Global default subscriber.");
         let root = trace_span!("my_app_start").entered();
    1 reply
    Robert F. Dickerson
    @rfdickerson

    I'm struggling to get batched reporting working- but simple works fine.

        let tracer = opentelemetry_jaeger::new_pipeline()
        .with_service_name("report_example")
        .install_simple()?;
    
        // let tracer = opentelemetry_jaeger::new_pipeline()
        //     .with_service_name("orchestrator")
        //     .install_batch(opentelemetry::runtime::Tokio)?;
    
        let opentelemetry = tracing_opentelemetry::layer().with_tracer(tracer);
    
        tracing_subscriber::registry()
            .with(opentelemetry)
            .try_init()
            .expect("Global default subscriber.");

    If I enable batching (uncomment and comment the other) I get the following error:

    OpenTelemetry trace error occurred. Exporter jaeger encountered the following error(s): thrift agent failed with message too long
    3 replies
    restioson
    @restioson:breadpirates.chat
    [m]
    hi all, wondering if it's a known bug that (with tracing) recording a f64/f32 into a span doesn't display correctly in jaeger? i don't really know which part of the tracing-opentelemetry-jaeger stack could be causing it, so maybe someone here has seen this before?
    1 reply
    restioson
    @restioson:breadpirates.chat
    [m]
    this is what i mean, saving should be around 25.71 as printed by dbg macro
    restioson
    @restioson:breadpirates.chat
    [m]
    I have not tried instrumenting with OT directly, I can try that
    Could also be a Jaeger bug
    Andrey Cherkashin
    @andoriyu
    Is it normal that propagator from opentracing_jaeger does inject sometimes?
    Or rather, what's the proper way to go from tracing'span to injecting headers before making a remote call?
    Atanas Yankov
    @atanunq
    Hi all, yesterday I wanted to experiment with sending traces to an OTel collector with the opentelemetry-otlp crate. I hoped that the Quickstart section will be enough to get me started but it turned out that tonic needs an async executor and I had to go on a massive detour to figure out features, executors, etc. Am I missing something or is the guide there outdated? I'd love to improve the docs but I need to figure it out myself first
    2 replies
    Ankit Gupta
    @agoops

    :wave: I'd like to use opentelemetry to persist my application's spans and events so I can export it to an otel compatible backend later.

    While writing an Exporter to persist json-serialized SpanData, I noticed that SpanData and its associated Events are not exported until the span fully completes.

    Running this example produces this sample-trace-data.jsonl, where each line is a complete Span.

    If my app hangs or crashes before completing a Span, I'd certainly like to capture any Events emitted before hang or crash, so that I can debug.

    Is this possible using the opentelemetry-rustsdk? Is there a good place for me to hook into upstream of SpanExporter that I can capture and persist span/event data as it's generated?