Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Nuutti Kotivuori
    @nakedible-p
    Thank you for the help!
    janxyz
    @janxyz:matrix.org
    [m]

    That can all be written in a single line with

    global::tracer("my-component").start("span-name");

    And there are also other quality of life functions like in_span which executes a closure the same way you asked for it before:
    https://docs.rs/opentelemetry/0.13.0/opentelemetry/trace/trait.Tracer.html#method.in_span

        global::tracer("my-component").in_span("span-name", |_cx| {
            // anything happening in functions we call can still access the active span...
            my_other_function();
        })

    I don't think it is fair to the maintainers to complain about the maturity of the library in a 0.13.0 pre-release version. If you are lacking functions then I am sure they would appreciate a contribution or issue to see if others are also interested in such a function.

    Zhongyang Wu
    @TommyCpp
    Thanks for the feedback. Here are my two cents around the issue. So the reason why we have so many different "internal pieces" defined and exposed is that we have to comply with the opentelemetry specifications. I think many users are currently leverage tracing-opentelemetry to have a nicer customer interface. Feel free to open a feature request and we can have some discussion there.
    Nuutti Kotivuori
    @nakedible-p
    To be clear - I wasn't complaining as in the sense that "this should be better by now!" - I was complaining in the sense of trying to highlight what might be missing as a form of feedback. For me, trying to use the library has been really obtuse, as not a lot has been explained from the viewpoint of library user - even the documentation mostly explains the rules from the opentelemetry specifications rather than what the user needs. I will certainly try to make convenience wrappers myself, and see if I come up with something useful to contribute - or open a discussion in a ticket once there's something concrete to discuss.
    I've also used tracing-opentelemetry for a bit - and while that is much more geared towards user convenience, many parts of the whole tracing ecosystem rubbed me wrong, and I hate the additional complexity of chaining one framework to another.
    Nuutti Kotivuori
    @nakedible-p
    But, please don't take anything I say as entitlement on my part or discouragement to you - I'm just trying to accurately convey what I'm seeing as a user of the framework, rather than a builder.
    Zhongyang Wu
    @TommyCpp
    No worries :grimacing: I think it's nice to have some user feedback especially when we are talking about how to make this library easier to use.
    Nuutti Kotivuori
    @nakedible-p
    This is me not understanding opentelemetry, but what is the purpose of the tracer name? I mean, I can use global::tracer("foo") to create a named tracer by the name of "foo", but what I'm seeing is that it really sets "foo" as the name of instrumentatio library! That looks really odd to me - if that's the case, then it seems more that tracer objects should be created just once per application as a singleton, or something?
    New Tracer instances are always created through a TracerProvider (see API). The name and version arguments supplied to the TracerProvider must be used to create an InstrumentationLibrary instance which is stored on the created Tracer.
    And indeed, install_simple() on a pipeline does let tracer = provider.get_tracer("opentelemetry", Some(env!("CARGO_PKG_VERSION"))); and then returns that tracer, as if to signal that this is the tracer instance that should be used for all spans in the application/library
    But this is totally at odds with the usage in https://github.com/open-telemetry/opentelemetry-rust/blob/main/examples/async/src/main.rs for example
    I mean there there's a new tracer for every function
    async fn run(addr: &SocketAddr) -> io::Result<usize> {
        let tracer = global::tracer("runner");
        let span = tracer.start(format!("running: {}", addr));
        let cx = Context::current_with_span(span);
    
        let mut stream = connect(addr).with_context(cx.clone()).await?;
        write(&mut stream).with_context(cx).await
    }
    Nuutti Kotivuori
    @nakedible-p
    ...but since you can't define a "default tracer" (only default tracer provider) and you can't get a "current tracer" from context, the tracer instance would need to be either "global" or be passed on to each function separately. Passing it on to each function separately completely destroys the point of Context, so global would make sense - but it doesn't rub me right either
    I'd have to do something like:
    lazy_static! {
        static ref TRACER: GenericTracer = stdout::new_pipeline()
            .with_trace_config(
                sdktrace::config()
                    .with_sampler(sdktrace::Sampler::AlwaysOn)
                    .with_id_generator(sdktrace::XrayIdGenerator::default()),
            )
            .install_simple();
    }
    And even in that - it's fine for the application, but what should I do in a library?
    Nuutti Kotivuori
    @nakedible-p
    JS docs say something like this:
    const api = require("@opentelemetry/api");
    
    const tracer = api.trace.getTracer("my-library-name", "0.2.3");
    
    async function doSomething() {
      const span = tracer.startSpan("doSomething");
      try {
        const result = await doSomethingElse();
        span.end();
        return result;
      } catch (err) {
        span.setStatus({
          // use an appropriate status code here
          code: api.SpanStatusCode.ERROR,
          message: err.message,
        });
        span.end();
        return null;
      }
    }
    It still doesn't make sense to say that it is an instrumentation library - but maybe that's just a weirdness of opentelemetry that should be explained. I mean sure, it makes sense if all the spans are created by auto-instrumentation, but it's hella weird if writing spans manually.
    But on the rust side, that would mean I'd stick this everywhere:
    lazy_static! {
        static ref TRACER: GenericTracer = global::tracer(env!("CARGO_PKG_NAME"), Some(env!("CARGO_PKG_VERSION"));
    }
    Nuutti Kotivuori
    @nakedible-p
        /// Sets the status of the `Span`. If used, this will override the default `Span`
        /// status, which is `Unset`. `message` MUST be ignored when the status is `OK` or `Unset`
        fn set_status(&self, code: StatusCode, message: String) {
            self.with_data(|data| {
                if code == StatusCode::Error {
                    data.status_message = message;
                }
                data.status_code = code;
            });
        }
    Doesn't that mean that it's possible to get a non-empty status_message for status OK by first setting status error with message, and then setting status OK? I mean, that's what the spec says, but I think what they mean is that status_message should be cleared if status is OK or Unset... or who knows
    Nuutti Kotivuori
    @nakedible-p
    Man am I bad at Rust... but this sorta works how I think it should work, although the calling convention is not up to snuff:
    async fn fut_with_span<T, Fut>(fut: Fut, tracer_name: &'static str, span_name: &'static str) -> Result<T, Error>
    where
        Fut: Future<Output = Result<T, Error>>
    {
        let tracer = global::tracer(tracer_name);
        let span = tracer.start(span_name);
        let cx = Context::current_with_span(span);
        let spanref = cx.span();
        let ret = fut.with_context(cx.clone()).await;
        match ret {
            Ok(v) => {
                spanref.set_status(StatusCode::Ok, "".to_string());
                Ok(v)
            },
            Err(e) => {
                spanref.record_exception(&e);
                spanref.set_status(StatusCode::Error, e.to_string());
                Err(e)
            },
        }
    }
    Nuutti Kotivuori
    @nakedible-p
    Finally, with a lot of trial and error and help from Rust discord, I managed to transform the above to something I can actually call on a future. It requires the async-trait crate, though.
    #[async_trait]
    trait FutureSpan: Future {
        async fn with_span<T, E>(self, tracer_name: &'static str, span_name: &'static str) -> Result<T, E>
        where
            E: std::error::Error,
            Self: std::future::Future<Output = Result<T, E>> + Sized + Send,
        {
            let tracer = global::tracer(tracer_name);
            let span = tracer.start(span_name);
            let cx = Context::current_with_span(span);
            let spanref = cx.span();
            let ret = self.with_context(cx.clone()).await;
            match ret {
                Ok(v) => {
                    spanref.set_status(StatusCode::Ok, "".to_string());
                    Ok(v)
                },
                Err(e) => {
                    spanref.record_exception(&e);
                    spanref.set_status(StatusCode::Error, e.to_string());
                    Err(e)
                },
            }
        }
    }
    
    impl<T, E, F> FutureSpan for F
    where
      F: Future<Output = Result<T, E>>
    {}
    Example usage:
        let outbound_stream = TcpStream::connect(proxy_addr).with_span("process_connection", "connect").await?;
    Nuutti Kotivuori
    @nakedible-p
    Next up I'll probably need to make a macro or helper to do something like Context::current().span().set_attribute(Key::new("key2").string("hello")
    should be span_attr!("key2", "hello") or something :-)
    Nuutti Kotivuori
    @nakedible-p
    Does anyone have any feedback on the tracer name vs. instrumentation lib thing? It's one of the bigger things I don't really understand in this whole thing...
    9 replies
    Nuutti Kotivuori
    @nakedible-p
    Another question - how do I create a span that does not have the currently active span as parent? It seems if I do tracer.span_builder().start(), then as parent context is set to None, it will still default to currently active span - which is kinda weird, as tracer.start() will especially use the method that will set Some(Context::current) as the parent context - but if the value is left as None, then the trace implementation does the same as well.
    3 replies
    bbigras
    @bbigras:matrix.org
    [m]
    Does opentelemetry-prometheus export metrics for all the span I'm using for opentelemetry?
    bbigras
    @bbigras:matrix.org
    [m]
    I'm guessing it's not the case.
    Julian Tescher
    @jtescher
    no you have to use the otel metrics api for that
    Nuutti Kotivuori
    @nakedible-p

    In AWS XRay, spans are often exported twice. First soon after creation, with in_progress: true, and then a second time after the span is ended. It is mandated that a span can only be delivered either once, when it is ended, or twice, first with in_progress true and second with in_progress false.

    Is there any mechanism in OpenTelemetry to export in-progress spans? I know a SpanProcessor does get start and end events, but it's not an Exporter.

    1 reply
    Zhongyang Wu
    @TommyCpp
    You can call exporter from the span processor though. You will have to implement your own span processor to do that
    Nuutti Kotivuori
    @nakedible-p
    Right, kind of figured that out myself - Exporter doesn't really care if they are in progress or complete - but I need to write my own SpanProcessor that will also export spans on start. The only snag-ish in that is that the end method gets a SpanData which is taken in by the Exporters, but start only gets a Span and Context, so I need to compose SpanData myself if I want to use that.
    Noel Campbell
    @nlcamp

    What's the reason behind having the interval parameter in opentelemetry_otlp::new_metrics_pipeline() and not just relying on the specified or default period value to control how often metrics are pushed? https://docs.rs/opentelemetry-otlp/0.5.0/opentelemetry_otlp/fn.new_metrics_pipeline.html

    Is there a known use case where someone would want to specify an interval other than the one seen in all the metrics examples? :

    // Skip first immediate tick from tokio, not needed for async_std.
    fn delayed_interval(duration: Duration) -> impl Stream<Item = tokio::time::Instant> {
        opentelemetry::util::tokio_interval_stream(duration).skip(1)
    }
    1 reply
    Giselle Serate
    @giselleserate-okta
    Hi all, I've been trying to get udp messages sent to an agent (instead of a collector) from a jaeger pipeline. I believe that when the max packet size is decreased, too-big packets just aren't getting sent, since when I watch the packets in wireshark, fewer packets come through with a smaller max packet size (as opposed to more, if they were being split up or something). Is this a bug, or an intended behavior?
    2 replies
    Nuutti Kotivuori
    @nakedible-p

    This is more of a OpenTelemetry question than related to Rust - but looking at the opentelemetry protobuf definition, then Span "attributes" is a list of KeyValues. And KeyValues always have a string key, and AnyValue as value. And AnyValue has can contain arrays and key value lists itself, meaning that a KeyValue list can easily be mapped from an object with arbitrary nesting - it's like JSON, except that there's also "int64" and "bytes" datatypes.

    However, when looking at the specification, it says that Span attributes can contain primitive types and homogenous arrays of primitive types only - so no nesting allowed.

    So two questions:
    1) Why is the attributes limited to primitive values in the spec even though the protobuf allows for more?
    2) Is it customary to just use encoded JSON as the value to overcome this limitation in all the cases where nesting required?

    2 replies
    Josh Triplett
    @joshtriplett
    I'm currently using tracing in my server, with text output at the moment, and I'm looking to start using tracing-opentelemetry and a remote service (possibly Honeycomb) for observability. My main server can just use telemetry-otlp and its batch mechanism directly, but that main server spawns various short-lived servers (separate instances) that may only run for seconds, and those short-lived servers can't afford to make multi-millisecond API calls to a remote service. I think it would make sense to have my main server accept telemetry from the short-lived servers, and upload it on their behalf in batch. I'm trying to figure out what people normally use for that; what Rust crates do I want to look into for accepting telemetry data and passing it on via the same OLTP connection I'm already using?
    Josh Triplett
    @joshtriplett
    The short-lived servers can count on the main server being continuously available, and any work done on the short-lived servers to get tracing data to the long-running main server should be as lightweight as possible.
    I don't know if I should be looking for a Rust in-process OpenTelemetry Collector to run on the main server, or if there's some much more lightweight mechanism I can use to get trace information from the short-lived servers to the main server and then let the main server handle exporting it via OTLP.
    9 replies
    Nuutti Kotivuori
    @nakedible-p

    This may be a bit controversial - and I don't know if the OpenTelemetry specification specifically mandates that default TraceId:s must be random 128-bits - but have you considered changing the default TraceId generator to use ULIDs instead of being completely random?

    https://github.com/ulid/spec - tl;dr on the spec: 48-bit milliseconds since epoch + 80 bit random

    And I don't mean adopting the string representation, and other stuff - just that instead of 128-bit random the beginning would be a timestamp by default. I think this will "magically" make a lot of stuff perform better.

    4 replies
    Nuutti Kotivuori
    @nakedible-p
    I suck at Rust - is there a more reasonable way to do the conversion of span Attributes into JSON than this?
                if sd.attributes.len() > 0 {
                    let mut map = Map::with_capacity(sd.attributes.len());
                    for (k, v) in &sd.attributes {
                        map.insert(k.to_string(), match v {
                            Value::Bool(x) => x.to_owned().into(),
                            Value::I64(x) => x.to_owned().into(),
                            Value::F64(x) => x.to_owned().into(),
                            Value::String(x) => x.to_owned().into(),
                            Value::Array(Array::Bool(x)) => x.to_owned().into(),
                            Value::Array(Array::I64(x)) => x.to_owned().into(),
                            Value::Array(Array::F64(x)) => x.to_owned().into(),
                            Value::Array(Array::String(x)) => x.to_owned().into(),
                        });
                    }
                    msg["attributes"] = map.into();
                }
    2 replies
    Alessandro Re
    @akiross

    Hello, I'm trying to get some tracing and metric data into otel collector (on the same host, default port). I'm using tracing for tracing data, and otel for metric data. I managed to get traces into the collector, but apparently I'm failing sending it the metrics.

    The code I'm using is similar to this: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=822f60525e95b24ec0c81a7f25dfa35f

    What am I doing wrong?

    4 replies
    Nisheeth Barthwal
    @nbaztec

    Could someone point me to a documentation on what are the use cases for different meters, as specified here:
    https://docs.rs/opentelemetry/0.14.0/opentelemetry/metrics/index.html

    From examples it seems as if both gauges and histograms can be implemented as ValueRecorders But then there are other things in there like Observers, BoundedCounter, etc. finding it a bit hard to figure out how to simply record a single value like bytes and incoming_requests and the use-cases for other meters.

    Thanks

    1 reply
    Glade Miller
    @glademiller
    I'm trying to use actix-web-opentelemetry and get the trace id and span id in my own middleware. For everything I have tried I always get a 0 trace id and span id is there a way to do this. Additionally I would love for my own middleware to be able to add a span that is in the context of the span from the actix-web-opentelemetry middleware and haven't had any luck making that work.
    Glade Miller
    @glademiller
    Perhaps the question should be phrased as how can I get the a trace id or span id that isn't 0. At what point are those assigned?
    The context for this is I need to return a header with those values in a certain format
    Glade Miller
    @glademiller
    I've narrowed down my problem and it is that the span created by actix-web-opentelemetry is not the current span in my own middleware. I'll be looking at actix's code a bit to see if there is a way to make sure the middleware layer correctly.
    Nate Mara
    @nate-onesignal
    Hello :wave: is it possible to directly set the sampling flag on an existing root-level opentelemetry span context? I'm using opentelemetry with tracing-opentelemetry and want to be able to set certain traces as non-sampled depending on some runtime data.
    9 replies
    Nisheeth Barthwal
    @nbaztec
    Hello, is it necessary to store the reference to ValueObserver created via meter.u64_value_observer.init() in static scope or can it be safely ignored within an underlying function?
    Derek Leverenz
    @derekleverenz
    Hi folks. I noticed a method i was using, https://docs.rs/opentelemetry/0.13.0/opentelemetry/trace/trait.TraceContextExt.html#tymethod.remote_span_context has been removed in 0.14. Is there another way I should be retrieving the remote span context now?
    2 replies
    Alessandro Re
    @akiross

    Is it possible to build a custom aggregator selector using opentelemetry_otlp 0.6.0? new_metrics_pipeline returns a OtlpMetricPipelineBuilder<Selector, ...> but the with_aggregator_selector is bound to Selector and not to a different type, as it happens in version 0.7.0.

    I wanted to use this to get some histogram metrics, but I cannot find a workaround. Can I get histograms in version 0.6?

    2 replies