i just saw tracing-opentelemetry still uses opentelemetry 0.12, so i have to bring everything down: https://github.com/tokio-rs/tracing/blob/master/tracing-opentelemetry/Cargo.toml#L25
ok it compiles now, but i get a runtime error
but i guess i'll just try upgrading yo actix 4 beta...
there is no reactor running, must be called from the context of a Tokio 1.x runtime because opentelemetry 0.12 pulls in tokio 1.0 while actix-web stable is still on 0.*but i guess i'll just try upgrading yo actix 4 beta...
@TommyCpp @dawid-nowak ty for your help. it works now
Thanks! If I understand that correctly, it's a function on the pipeline, right? So all my histograms have the same buckets?
opentelemetry_otlp::new_metrics_pipeline(tokio::spawn, delayed_interval)
.with_export_config(export_config)
.with_period(std::time::Duration::from_secs(open_telemetry.metric_window.unwrap_or_else(|| 30)))
.with_aggregator_selector(selectors::simple::Selector::Histogram(vec![0.0, 0.1, 0.2, 0.3, 0.5, 0.8, 1.3, 2.1]))
.build()?,
Found this error in build log
./target/debug/deps/sha2-0dbd11ac79fa6266.gcno:version '408*', prefer 'A93*'
find: ‘gcov’ terminated by signal 11
sorta odd, we don't specify much for that?
It works until open-telemetry/opentelemetry-rust#465, but we haven't change anything in CI pipeline around that PR
hey all, i am running this example here
fn init_meter() -> metrics::Result<PushController> {
let export_config = ExporterConfig {
endpoint: "http://localhost:4317".to_string(),
protocol: Protocol::Grpc,
..ExporterConfig::default()
};
opentelemetry_otlp::new_metrics_pipeline(tokio::spawn, delayed_interval)
.with_export_config(export_config)
.with_aggregator_selector(selectors::simple::Selector::Exact)
.build()
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error + Send + Sync + 'static>> {
let _started = init_meter()?;
let meter = global::meter("test2");
let value_counter = meter.u64_counter("blah9").init();
let labels = vec![KeyValue::new("key1","val1")];
for i in 0..100{
let j = i%4;
println!("{} {}",i,j);
let mut labels = vec![];//labels.clone();
let kv = match j{
0=> KeyValue::new("method","GET"),
1=> KeyValue::new("method","POST"),
2=> KeyValue::new("method","PUT"),
3=> KeyValue::new("method","DELETE"),
_=> KeyValue::new("method","HEAD"),
};
labels.push(kv);
// labels.push(KeyValue::new("key4",j.to_string()));
value_counter.add(1, &labels);
tokio::time::sleep(Duration::from_secs(1)).await;
}
// wait for 1 minutes so that we could see metrics being pushed via OTLP every 10 seconds.
tokio::time::sleep(Duration::from_secs(60)).await;
shutdown_tracer_provider();
Ok(())
}At the end, in Prometheus dashboard I am getting
agent_blah9{collector="pf", instance="otel-agent:8889", job="otel-collector", method="PUT", type="docker"} 25where I would expect
agent_blah9{collector="pf", instance="otel-agent:8889", job="otel-collector", method="PUT", type="docker"} 25
agent_blah9{collector="pf", instance="otel-agent:8889", job="otel-collector", method="POST", type="docker"} 25
agent_blah9{collector="pf", instance="otel-agent:8889", job="otel-collector", method="GET", type="docker"} 25
agent_blah9{collector="pf", instance="otel-agent:8889", job="otel-collector", method="DELETE", type="docker"} 25Any ideas ? I had a look at agent opentelemetry-agent logs and there seems to be only one type of metric there
Data point labels:
-> method: PUT
12 replies
error: failed to run custom build command for `opentelemetry-otlp v0.5.0`
965
966 Caused by:
967 process didn't exit successfully: `/.../target/release/build/opentelemetry-otlp-13bb7928af03e4cb/build-script-build` (exit code: 101)
968 --- stderr
969 thread 'main' panicked at 'Error generating protobuf: Os { code: 2, kind: NotFound, message: "No such file or directory" }', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/opentelemetry-otlp-0.5.0/build.rs:28:10
970
it's happening when opentelemetry-otlp's build.rs calls tonic_build, so i thought my build container might be missing protoc, but it's installed
i could hack the build.rs to provide a useful error message but it's about 5 dependencies deep so will involve a fair amount of dependency patching. has anyone seen this error before and knows what might be wrong?
Hey folks, of the late seems otel (jaeger) propagator has stopped working on the tonic gRPC clients even though the traceId is in the current context (via
withContext) and the propagator set. My other Go service thus cannot join spans correctly. Anyone experience and/or have an idea about this? Thanks!
7 replies
I'm running into errors trying to adapt the basic-otlp example to have a SumObserver with a callback that calls observe() using a variable instead of a constant. Is there a recommended approach for this? Here's a diff showing my changes to basic-otlp/src/main.rs, which currently don't compile:
diff --git a/examples/basic-otlp/src/main.rs b/examples/basic-otlp/src/main.rs
index 51ac886..6a981ac 100644
--- a/examples/basic-otlp/src/main.rs
+++ b/examples/basic-otlp/src/main.rs
@@ -51,6 +51,7 @@ lazy_static::lazy_static! {
];
}
+
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error + Send + Sync + 'static>> {
let _ = init_tracer()?;
@@ -59,7 +60,8 @@ async fn main() -> Result<(), Box<dyn Error + Send + Sync + 'static>> {
let tracer = global::tracer("ex.com/basic");
let meter = global::meter("ex.com/basic");
- let one_metric_callback = |res: ObserverResult<f64>| res.observe(1.0, COMMON_LABELS.as_ref());
+ let mut f64_metric_val: f64 = 1.0;
+ let one_metric_callback = |res: ObserverResult<f64>| res.observe(f64_metric_val, COMMON_LABELS.as_ref());
let _ = meter
.f64_value_observer("ex.com.one", one_metric_callback)
.with_description("A ValueObserver set to 1.0")
@@ -101,8 +103,15 @@ async fn main() -> Result<(), Box<dyn Error + Send + Sync + 'static>> {
});
});
- // wait for 1 minutes so that we could see metrics being pushed via OTLP every 10 seconds.
- tokio::time::sleep(Duration::from_secs(60)).await;
+ let mut count = 0u32;
+ loop {
+ tokio::time::sleep(Duration::from_secs(10)).await;
+ count += 1;
+ f64_metric_val += 1.0;
+ if count == 6 {
+ break;
+ }
+ }
shutdown_tracer_provider();
Hi, I am trying out the tracing api with local jaeger instance,
only a short lived #[instrument] appears in jaeger ui, and ui shows no unfinished span or event.
Even this simple example does not work; it only shows 'doing_work' span and no event at all. what is wrong?
fn main() {
opentelemetry::global::set_text_map_propagator(opentelemetry_jaeger::Propagator::new());
let tracer = opentelemetry_jaeger::new_pipeline()
.install_simple()
.unwrap();
use opentelemetry::trace::Tracer;
tracing::error!("first error event");
tracer.in_span("doing_work", |cx| {
tracing::error!("nested error event");
tracing::info!("nested info event");
tracing::warn!("nested warn event");
tracing::debug!("nested debug event");
tracing::trace!("nested trace event");
});
opentelemetry::global::shutdown_tracer_provider();
}
Can see an example init here that sets up the tracing registry https://github.com/open-telemetry/opentelemetry-rust/blob/main/examples/actix-http-tracing/src/main.rs#L32-L43
then you can switch the
in_span calls to https://docs.rs/tracing/0.1.25/tracing/span/struct.Span.html#method.in_scope and you should see both your spans and the logs
(jaeger doesn't show logs outside of spans, so the
"first error event" log won't appear there)
Thanks, didn't realize the tokio-tracing and otel APIs could conflict.
Now I am left with an incomplete span problem. For example, in the code below, jaeger shows async_fn2 (child) with its parent span id, but the parent span itself is missing.
#[instrument]
async fn async_fn2() {
tracing::info!("enter2");
tokio::time::sleep(std::time::Duration::from_secs(5)).await;
tracing::info!("exit2");
}
#[instrument]
async fn async_fn() {
tracing::info!("enter");
async_fn2().await;
// panic!(); // <- this flushes all spans
tokio::time::sleep(std::time::Duration::from_secs(1000)).await;
tracing::info!("exit");
}
#[tokio::main]
async fn main() -> Result<()> {
let tracer = opentelemetry_jaeger::new_pipeline()
.with_service_name("timer")
.install_simple()?;
tracing_subscriber::registry()
.with(tracing_opentelemetry::subscriber().with_tracer(tracer))
.try_init()?;
async_fn().await;
Ok(())
}Is there a way to flush long-running span periodically? Is this a limitation of jaeger, opentelemetry interface, or rust implementation?
It would be great to have a 'growing' span: the span's name and start time is immediately visible in jaeger (batching delay is ok), and its end time and additional log/context is updated over time.
Hi! I'm trying to use a tracer to simply trace an async function. It seems that the helper for that is with_context from FutureExt. However, it seems horribly difficult to use? I mean, I need to create a tracer, and a span, and then set that span in the current context and then pass that context on to the with_context helper - and it still doesn't seem to automatically set the span status based on the response, or to record any errors in the span from the future.
You can use
with_current_context() to simply attach the currently active context. https://docs.rs/opentelemetry/0.13.0/opentelemetry/trace/trait.FutureExt.html#method.with_current_context
Should the example be updated to reflect that? https://github.com/open-telemetry/opentelemetry-rust/blob/main/examples/async/src/main.rs - I'm still not sure how that makes things easier - and it still doesn't seem to be handling setting the error response in to the span.