Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Dirkjan Ochtman
    @djc
    :thumbsup:
    Dawid Nowak
    @dawid-nowak

    hey; cools stuff about 0.13.0 let's hope it is a lucky number :) anyway, small question about metrics again. if I get it right for ValueRecorders the aggregator is set once when the metric pipeline is created for example :

      controller = Some(opentelemetry_otlp::new_metrics_pipeline(tokio::spawn, delayed_interval)
                      .with_export_config(export_config)
                      .with_period(std::time::Duration::from_secs(open_telemetry.metric_window.unwrap_or_else(|| 30)))
                      .with_aggregator_selector(selectors::simple::Selector::Histogram(vec![0.0,0.1,0.2,0.3,0.5,0.8,1.3,2.1]))
                      .build()?);

    My understanding is that this will apply to all ValueRecorders and all histograms will have same buckets.
    Isn't it a bit of a limitation ?
    From my application perspective, I would like to be able to set different buckets for different metrics.

    10 replies
    janxyz
    @janxyz:matrix.org
    [m]
    Thanks! It's getting late in Europe, I'll check it first thing in the morning 🙂
    Zhongyang Wu
    @TommyCpp
    :+1:
    janxyz
    @janxyz:matrix.org
    [m]
    wow maybe we can even re-use the Textmap propagater from the example by just creating a textmap and setting the fields in it with the stuff from the proto message
    vbrandl
    @vbrandl:matrix.vsund.de
    [m]

    i'm trying to use opentelemetry in an existing actix-web application, that already uses tracing to export into jaeger. i try to configure everything like this:

    use opentelemetry::{global, runtime::TokioCurrentThread};
    use tracing::{subscriber::set_global_default, Subscriber};
    use tracing_bunyan_formatter::{BunyanFormattingLayer, JsonStorageLayer};
    use tracing_log::LogTracer;
    use tracing_subscriber::{layer::SubscriberExt, EnvFilter, Registry};
    
    pub fn get_subscriber(name: &str, env_filter: &str) -> impl Subscriber + Send + Sync {
        let env_filter =
            EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new(env_filter));
    
        let formatting_layer = BunyanFormattingLayer::new(name.to_string(), std::io::stdout);
    
        global::set_text_map_propagator(opentelemetry_jaeger::Propagator::new());
        let tracer = opentelemetry_jaeger::new_pipeline()
            .with_service_name(name)
            .install_batch(TokioCurrentThread)
            .expect("cannot install jaeger pipeline");
        let telemetry = tracing_opentelemetry::layer().with_tracer(tracer);
    
        Registry::default()
            .with(telemetry)
            .with(env_filter)
            .with(JsonStorageLayer)
            .with(formatting_layer)
    }
    
    pub fn init_subscriber(subscriber: impl Subscriber + Send + Sync) {
        LogTracer::init().expect("Failed to set logger");
        set_global_default(subscriber).expect("Failed to set tracing subscriber");
    }

    and will later call it init_subscriber(get_subscriber("app_name", "info"))

    but this fails because opentelemetry::sdk::trace::Tracer does not implement the following traits: opentelemetry::trace::tracer::Tracer, PreSampledTracer.
    i don't know what i'm missing here...

    Dawid Nowak
    @dawid-nowak
    @vbrandl:matrix.vsund.de I would double check that tracing/opentelemetry dependencies are in sync between your project and actix-web
    vbrandl
    @vbrandl:matrix.vsund.de
    [m]
    what exactly do you mean by that? is there a list of compatible versions?
    Dawid Nowak
    @dawid-nowak
    i believe that actix has dependency on tracing, tracing has dependency on opentelemetry; in your app you probably have a dependency on opentelemety; all of dependencies have to match
    Zhongyang Wu
    @TommyCpp
    Could you share your Cargo.toml?
    i just saw tracing-opentelemetry still uses opentelemetry 0.12, so i have to bring everything down: https://github.com/tokio-rs/tracing/blob/master/tracing-opentelemetry/Cargo.toml#L25
    Zhongyang Wu
    @TommyCpp
    Yeah you will have to use 0.12 for opentelemetry and 0.11 for opentelemetry-jaeger
    Dawid Nowak
    @dawid-nowak
    also, check which version of telemetry is actix using
    vbrandl
    @vbrandl:matrix.vsund.de
    [m]
    ok it compiles now, but i get a runtime error there is no reactor running, must be called from the context of a Tokio 1.x runtime because opentelemetry 0.12 pulls in tokio 1.0 while actix-web stable is still on 0.*
    but i guess i'll just try upgrading yo actix 4 beta...
    vbrandl
    @vbrandl:matrix.vsund.de
    [m]
    @tom
    @TommyCpp @dawid-nowak ty for your help. it works now
    janxyz
    @janxyz:matrix.org
    [m]
    Is there a way to configure the buckets the valueRecorder uses?
    Dawid Nowak
    @dawid-nowak
    this works for me .with_aggregator_selector(selectors::simple::Selector::Histogram(vec![0.0, 0.1, 0.2, 0.3, 0.5, 0.8, 1.3, 2.1]))
    janxyz
    @janxyz:matrix.org
    [m]
    Thanks! If I understand that correctly, it's a function on the pipeline, right? So all my histograms have the same buckets?
    Dawid Nowak
    @dawid-nowak
    opentelemetry_otlp::new_metrics_pipeline(tokio::spawn, delayed_interval)
                        .with_export_config(export_config)
                        .with_period(std::time::Duration::from_secs(open_telemetry.metric_window.unwrap_or_else(|| 30)))
                .with_aggregator_selector(selectors::simple::Selector::Histogram(vec![0.0, 0.1, 0.2, 0.3, 0.5, 0.8, 1.3, 2.1]))
                        .build()?,
    there is a PR open so you can customize aggregators and buckets that they might be using based on metric descriptor name open-telemetry/opentelemetry-rust#497
    Zhongyang Wu
    @TommyCpp
    I think our coverage broke somehow. Haven't seen a coverage report for a while
    Found this error in build log
    ./target/debug/deps/sha2-0dbd11ac79fa6266.gcno:version '408*', prefer 'A93*'
    find: ‘gcov’ terminated by signal 11
    Julian Tescher
    @jtescher
    hm
    sorta odd, we don't specify much for that?
    Zhongyang Wu
    @TommyCpp
    It works until open-telemetry/opentelemetry-rust#465, but we haven't change anything in CI pipeline around that PR
    Dawid Nowak
    @dawid-nowak
    This message was deleted

    hey all, i am running this example here

    fn init_meter() -> metrics::Result<PushController> {
        let export_config = ExporterConfig {
            endpoint: "http://localhost:4317".to_string(),
            protocol: Protocol::Grpc,
            ..ExporterConfig::default()
        };
        opentelemetry_otlp::new_metrics_pipeline(tokio::spawn, delayed_interval)
            .with_export_config(export_config)
        .with_aggregator_selector(selectors::simple::Selector::Exact)
            .build()
    }
    
    #[tokio::main]
    async fn main() -> Result<(), Box<dyn Error + Send + Sync + 'static>> {
        let _started = init_meter()?;
        let meter = global::meter("test2");
        let value_counter = meter.u64_counter("blah9").init();   
        let labels = vec![KeyValue::new("key1","val1")];
        for i in 0..100{
        let j = i%4;
        println!("{} {}",i,j);
        let mut labels = vec![];//labels.clone();
        let kv = match j{
            0=> KeyValue::new("method","GET"),
            1=> KeyValue::new("method","POST"),
            2=> KeyValue::new("method","PUT"),
            3=> KeyValue::new("method","DELETE"),
            _=> KeyValue::new("method","HEAD"),
        };
        labels.push(kv);
    //    labels.push(KeyValue::new("key4",j.to_string()));
    
        value_counter.add(1, &labels);
        tokio::time::sleep(Duration::from_secs(1)).await;
        }
    
        // wait for 1 minutes so that we could see metrics being pushed via OTLP every 10 seconds.
        tokio::time::sleep(Duration::from_secs(60)).await;
    
        shutdown_tracer_provider();
    
        Ok(())
    }

    At the end, in Prometheus dashboard I am getting
    image.png

    agent_blah9{collector="pf", instance="otel-agent:8889", job="otel-collector", method="PUT", type="docker"}   25

    where I would expect

    agent_blah9{collector="pf", instance="otel-agent:8889", job="otel-collector", method="PUT", type="docker"}   25
    agent_blah9{collector="pf", instance="otel-agent:8889", job="otel-collector", method="POST", type="docker"}   25
    agent_blah9{collector="pf", instance="otel-agent:8889", job="otel-collector", method="GET", type="docker"}   25
    agent_blah9{collector="pf", instance="otel-agent:8889", job="otel-collector", method="DELETE", type="docker"}   25

    Any ideas ? I had a look at agent opentelemetry-agent logs and there seems to be only one type of metric there

    Data point labels:
         -> method: PUT
    12 replies
    Zhongyang Wu
    @TommyCpp
    I tried it and it seems to be the right data
    Dawid Nowak
    @dawid-nowak
    More interested in the general approach. Are you guys using some sophisticated tooling or pepper the code with printfs
    Zhongyang Wu
    @TommyCpp
    Clion has a debugger for Rust, it’s not as powerful as the one for Java but it works fine I think
    Dawid Nowak
    @dawid-nowak
    Yeah, it crashed when I was loading the project 🤣
    Zhongyang Wu
    @TommyCpp
    @jtescher Should we consider push a new patch version for opentelemetry-otlp to (hopefully) fix the broken doc?
    Julian Tescher
    @jtescher
    @TommyCpp could do a patch release for it
    Zhongyang Wu
    @TommyCpp
    That’s great. Thanks!
    Ido Barkan
    @barkanido
    hey folks. I am writing a service which reports to datadog. should I use opentelemtry-rust ? how stable for production is it?
    7 replies
    Jasper
    @jbg
    hi all, i'm trying to build something that depends on the opentelemetry-otlp rust crate, and I get this cryptic error from opentelemetry-otlp's build.rs:
    10 replies
    error: failed to run custom build command for `opentelemetry-otlp v0.5.0`
    965    
    966    Caused by:
    967      process didn't exit successfully: `/.../target/release/build/opentelemetry-otlp-13bb7928af03e4cb/build-script-build` (exit code: 101)
    968      --- stderr
    969      thread 'main' panicked at 'Error generating protobuf: Os { code: 2, kind: NotFound, message: "No such file or directory" }', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/opentelemetry-otlp-0.5.0/build.rs:28:10
    970
    it's happening when opentelemetry-otlp's build.rs calls tonic_build, so i thought my build container might be missing protoc, but it's installed
    i could hack the build.rs to provide a useful error message but it's about 5 dependencies deep so will involve a fair amount of dependency patching. has anyone seen this error before and knows what might be wrong?
    Nisheeth Barthwal
    @nbaztec
    Hey folks, of the late seems otel (jaeger) propagator has stopped working on the tonic gRPC clients even though the traceId is in the current context (via withContext) and the propagator set. My other Go service thus cannot join spans correctly. Anyone experience and/or have an idea about this? Thanks!
    7 replies
    Noel Campbell
    @nlcamp
    I'm running into errors trying to adapt the basic-otlp example to have a SumObserver with a callback that calls observe() using a variable instead of a constant. Is there a recommended approach for this? Here's a diff showing my changes to basic-otlp/src/main.rs, which currently don't compile:
    diff --git a/examples/basic-otlp/src/main.rs b/examples/basic-otlp/src/main.rs
    index 51ac886..6a981ac 100644
    --- a/examples/basic-otlp/src/main.rs
    +++ b/examples/basic-otlp/src/main.rs
    @@ -51,6 +51,7 @@ lazy_static::lazy_static! {
         ];
     }
    
    +
     #[tokio::main]
     async fn main() -> Result<(), Box<dyn Error + Send + Sync + 'static>> {
         let _ = init_tracer()?;
    @@ -59,7 +60,8 @@ async fn main() -> Result<(), Box<dyn Error + Send + Sync + 'static>> {
         let tracer = global::tracer("ex.com/basic");
         let meter = global::meter("ex.com/basic");
    
    -    let one_metric_callback = |res: ObserverResult<f64>| res.observe(1.0, COMMON_LABELS.as_ref());
    +    let mut f64_metric_val: f64 = 1.0;
    +    let one_metric_callback = |res: ObserverResult<f64>| res.observe(f64_metric_val, COMMON_LABELS.as_ref());
         let _ = meter
             .f64_value_observer("ex.com.one", one_metric_callback)
             .with_description("A ValueObserver set to 1.0")
    @@ -101,8 +103,15 @@ async fn main() -> Result<(), Box<dyn Error + Send + Sync + 'static>> {
             });
         });
    
    -    // wait for 1 minutes so that we could see metrics being pushed via OTLP every 10 seconds.
    -    tokio::time::sleep(Duration::from_secs(60)).await;
    +    let mut count = 0u32;
    +    loop {
    +        tokio::time::sleep(Duration::from_secs(10)).await;
    +        count += 1;
    +        f64_metric_val += 1.0;
    +        if count == 6 {
    +            break;
    +        }
    +    }
    
         shutdown_tracer_provider();
    3 replies
    elbaro
    @elbaro

    Hi, I am trying out the tracing api with local jaeger instance,
    only a short lived #[instrument] appears in jaeger ui, and ui shows no unfinished span or event.

    Even this simple example does not work; it only shows 'doing_work' span and no event at all. what is wrong?

    fn main() {
        opentelemetry::global::set_text_map_propagator(opentelemetry_jaeger::Propagator::new());
        let tracer = opentelemetry_jaeger::new_pipeline()
            .install_simple()
            .unwrap();
        use opentelemetry::trace::Tracer;
        tracing::error!("first error event");
        tracer.in_span("doing_work", |cx| {
            tracing::error!("nested error event");
            tracing::info!("nested info event");
            tracing::warn!("nested warn event");
            tracing::debug!("nested debug event");
            tracing::trace!("nested trace event");
        });
        opentelemetry::global::shutdown_tracer_provider();
    }
    image.png
    image.png
    Julian Tescher
    @jtescher
    @elbaro you probably want to stick with either the tracing or the otel API to not run into issues when mixing the two in the same app
    then you can switch the in_span calls to https://docs.rs/tracing/0.1.25/tracing/span/struct.Span.html#method.in_scope and you should see both your spans and the logs
    (jaeger doesn't show logs outside of spans, so the "first error event" log won't appear there)