kamon-io/Kamon

The Open Source tool for monitoring applications running on the JVM

Yaroslav Derman

@yarosman

Hello. Does anyone try to use kamon with zio ?

@ivantopo Have you tried to run kamon with zio, because I don't have context propagation and correct span creation ((

Ivan Topolnjak

@ivantopo

no, didn't try it myself

didn't even try ZIO yet

jmendesky

@jmendesky

Hi, is there any interest to allow Kamon's Context to be serialised in HTTP according to the W3C Correlation Context proposal? https://w3c.github.io/correlation-context/
I could see this as a big plus for interoperability in a mixed-tech architecture. It's not standardised yet, but once finalised, would you consider it/would you be open for a PR or does something speak against it in general?
I could see it implemented similar to Trace Context Codecs so that as a user you could choose how to serialise it.
ATM the only difference in how Kamon serialises context and the W3C proposal is that key-value pairs are separated with ; in Kamon and with , in the proposal (apart from the actual name of the header value - which is configurable already)

5 replies

Franco Albornoz

@dannashirn

Hey guys, so I had a shared library with Kamon 1.x that used the SpanCustomizer which I am trying to replace now with a PreStartHook. The thing is I used to have the customizer with a constructor with two parameters which were used to then customize the spans. The thing is PreStartHook needs to have a parameterless constructor, but I'm not sure how I'd be able to customize the spans more dynamically with it.

6 replies

jmendesky

@jmendesky

Hi all, is there a preferred way to propagate trace contexts and context tags via GRPC? I can see this issue has been around for a while kamon-io/Kamon#616 (assuming this is for akka-grpc) and there is also this repo: https://github.com/nezasa/kamon-akka-grpc. Is there a plan for the future?

4 replies

Ilya

@squadgazzz

Hello, Kamon dev. Here’s my config https://controlc.com/cf392b75. I can’t send anything to influxdb with Kamon because of

ERROR kamon.influxdb.InfluxDBReporter - Metrics POST to InfluxDB failed with status code [404], response body: {"error":"database not found: \"mydb\””}

error. I can’t understand where did it find mydb? I have db0 in configs.

7 replies

Ramakrishna Hande

@rhande-mdsol

Hi, Is the header name X-B3-TraceId renamed to trace-id ? We were using some old version of zipkin but now after some upgrade we have response having header trace-id. We are using Zipkin version 2.12.3. Thanks in advance

3 replies

Ilya

@squadgazzz

Hello, there! Is it possible to add multiple custom fields to histogram? Not the tags. And is there any way to create a custom metric with Kamon without fields p50.0 p70.0 p90.0 p95.0 p99.0 p99.9 service?

2 replies

Khal!l

@redkhalil

Hi, after upgrading to bundle and prometheus 2.1.2 I'm getting the following exception upon startup

Unable to start Kanela Agent. Please remove -javaagent from your startup arguments and contact Kanela support.: java.lang.NoClassDefFoundError: com/datastax/driver/core/Session

6 replies

TjarkoG

@TjarkoG

I don't want to ask like this but since my pull-request is being ignored completely for almost 2 months i don't know how else to contact. Is there something that i forgot to do?

17 replies

Alexis Hernandez

@AlexITC

BTW, the gitter activity on the right hasn't been updated for a year

1 reply

Alexis Hernandez

@AlexITC

Last item is from 2019

Ivan Topolnjak

@ivantopo

ha, didn't see that before

Enzo Bonggio

@enzobonggio

Hi I’m using kanela-agent on a play project and I want to add support for httpok. So I added on my build.sbt -> "io.kamon" %% "kamon-okhttp" % "1.0.3” and the configuration to enable it as it says on reference.conf
The error that I’m facing right now is the following

Caused by: java.lang.ClassNotFoundException: kanela.agent.api.instrumentation.KanelaInstrumentation
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at kanela.agent.util.classloader.ChildFirstURLClassLoader.loadClass(ChildFirstURLClassLoader.java:58)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at kanela.agent.InstrumentationLoader.lambda$loadInstrumentation$b85f757d$1(InstrumentationLoader.java:58)
    at kanela.agent.libs.io.vavr.control.Try.of(Try.java:75)
    at kanela.agent.InstrumentationLoader.loadInstrumentation(InstrumentationLoader.java:56)
    at kanela.agent.InstrumentationLoader.lambda$null$1(InstrumentationLoader.java:46)
    at kanela.agent.libs.io.vavr.collection.List.flatMap(List.java:898)
    at kanela.agent.InstrumentationLoader.lambda$load$4(InstrumentationLoader.java:46)
    at kanela.agent.libs.io.vavr.collection.List.map(List.java:1038)
    at kanela.agent.InstrumentationLoader.load(InstrumentationLoader.java:43)
    at kanela.agent.Kanela.lambda$null$0(Kanela.java:79)
    at kanela.agent.util.classloader.InstrumentationClassPath.use(InstrumentationClassPath.java:84)
    at kanela.agent.Kanela.lambda$start$1(Kanela.java:68)
    at kanela.agent.util.Execution.timed(Execution.java:32)
    at kanela.agent.util.Execution.runWithTimeSpent(Execution.java:41)
    at kanela.agent.Kanela.start(Kanela.java:67)
    at kanela.agent.Kanela.premain(Kanela.java:47)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386)
    at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401)

Should I add another dependency ? Also I could not find any examples of this so I’m lost

3 replies

Krunoslav Uzelac

@Reeebuuk

Hi guys, I'm having the Unable to start Kanela Agent. Please remove -javaagent from your startup arguments and contact Kanela support issue

javaAgents += "io.kamon" % "kanela-agent" % "1.0.5",

addSbtPlugin("io.kamon" % "sbt-kanela-runner" % "2.0.6")
addSbtPlugin("com.lightbend.sbt" % "sbt-javaagent" % "0.1.5")

any hints? :)

will try 2.1.3

Ivan Topolnjak

@ivantopo

hey @Reeebuuk, indeed, try 2.1.3

Pavel Pliachystsik

@pavel.pliachystsik_gitlab

Hello everybody! I have Play framework application (2.8.1) and subproject with Akka cluster (2.6.3). I'm trying to integrate kamon, added play agent and javaagent to pluguins, enabled it in root project and see some attach errors:
[Attach Listener] ERROR 2020-07-10 09:23:15 Logger : Error => akka.dispatch.Envelope with message Cannot resolve type description for scala.Product. Class loader: java.net.URLClassLoader@2eea9be8: java.lang.IllegalStateException: Cannot resolve type description for scala.Product
at kanela.agent.libs.net.bytebuddy.pool.TypePool$Resolution$Illegal.resolve(TypePool.java:159)
at kanela.agent.libs.net.bytebuddy.pool.TypePool$Default$LazyTypeDescription$TokenizedGenericType.toErasure(TypePool.java:6241)
at kanela.agent.libs.net.bytebuddy.pool.TypePool$Default$LazyTypeDescription$GenericTypeToken$Resolution$Raw$RawAnnotatedType.of(TypePool.java:3412)
at kanela.agent.libs.net.bytebuddy.pool.TypePool$Default$LazyTypeDescription$GenericTypeToken$Resolution$Raw$RawAnnotatedType$LazyRawAnnotatedTypeList.get(TypePool.java:3511)
at kanela.agent.libs.net.bytebuddy.pool.TypePool$Default$LazyTypeDescription$GenericTypeToken$Resolution$Raw$RawAnnotatedType$LazyRawAnnotatedTypeList.get(TypePool.java:3456)
at java.base/java.util.AbstractList$Itr.next(AbstractList.java:371)
at kanela.agent.libs.net.bytebuddy.description.type.TypeList$Generic$AbstractBase.accept(TypeList.java:275)
at kanela.agent.libs.net.bytebuddy.dynamic.scaffold.InstrumentedType$Factory$Default$1.represent(InstrumentedType.java:362)
at kanela.agent.libs.net.bytebuddy.ByteBuddy.rebase(ByteBuddy.java:851)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$TypeStrategy$Default$1.builder(AgentBuilder.java:2007)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$Transformation$Simple$Resolution.apply(AgentBuilder.java:10117)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10494)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10460)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1500(AgentBuilder.java:10223)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10907)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10845)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10413)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)
at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)
at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:167)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$RedefinitionStrategy$Dispatcher$ForJava6CapableVm.retransformClasses(AgentBuilder.java:6865)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$RedefinitionStrategy$Collector$ForRetransformation.doApply(AgentBuilder.java:7142)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$RedefinitionStrategy$Collector.apply(AgentBuilder.java:6991)
at kanela.agent.libs.net.bytebuddy.agent.bui

But after restart they always disappear, so I assume it's all because subpoject is still compiling. Running it in IntellijIdea

ang-mic

@ang-mic

Hi everyone, happy Sunday :) I am going through this tutorial that involves Zipkin and Prometheus. When I go to the URL for Prometheus, I can see the text but not the UI. What am I doing wrong? Thanks in advance.

Pavel Pliachystsik

@pavel.pliachystsik_gitlab

@ang-mic prometheus is standalone app, what kamon do, is export it in vew which prometheus can collect. You need to install prometheus to see ui, and grafana to ger great ui for your monitorring tool

ang-mic

@ang-mic

@pavel.pliachystsik_gitlab Thank you for coming back to me. That clears the water, I wll look into it :)

Alex

@forrana

Hi.

  # kamon.context.codecs.string-keys {
  #   request-id = "X-Request-ID"
  # }

Didn't managed to find anywhere - is it possible to setup fallback values for those?

Ivan Topolnjak

@ivantopo

hey

Alex

@forrana

The only way I've managed to do it - is to check the key in Kamon.currentContext and then create a new context with a key(if it's None) which then I use in a wrapper on HTTP call. Seems to me there should be an easier way?

Ivan Topolnjak

@ivantopo

correct me if I'm wrong, but are you asking if there is a way to generate a value for "request-id" if there was none in the current context?

Alex

@forrana

So the idea is I need to propagate a header "X-Request-ID" but if it's not set there should be a default value, yes.

Ivan Topolnjak

@ivantopo

there is no built-in way to go about it

but let me ask you a few questions to understand the problem better

you mentioned wrapping an HTTP call, is that a client or server HTTP call?

Alex

@forrana

So that's the way it works

val XKey = Key.broadcastString("x-key")
val context = 
    Kamon.currentContext().get(XKey) match {
      case None =>
        Kamon.currentContext().withKey(XKey, Some("XKey Value"))
      case _ => Kamon.currentContext()
    }

Kamon.withContext(context) {
    for {
        result <- httpClient(Get(uri))
    } yield result
}

I just hoped there should be an easier one

nmkarvekar

@nmkarvekar

I have question on kamon usage in production. Is it free to use kamon with instrumentation integrated with prometheus in production without APM? Is there any restriction to use that in terms of number of services?

Ivan Topolnjak

@ivantopo

hey there

yeap, it is free

no restrictions at all

basically, if you can find it on our github then its free :joy:

nmkarvekar

@nmkarvekar

Thanks Ivan. I was little bit confused on this https://kamon.io/apm/pricing/, Starter: up to 2 instances per service,up to 3 services, but as I understand it is if we use APM.

Ivan Topolnjak

@ivantopo

correct, that pricing is only for APM

nmkarvekar

@nmkarvekar

ok. thank you.

Vish Ramachandran

@vishramachandran

@ivantopo A few weeks earlier, I had asked a question about draining metrics to reporter before shutdown. The suggestion was to run Kamon.stopModules() from a shutdown hook to ensure that all metrics are drained to reporters and backends. I precisely did that and waited for the returned future to complete, but every now and then, I see missed samples. There is clear evidence from logs that the metric was published and stopModules call finished successfully, but the sample that was incremented moments before shutdown was not drained to reporters. Any other calls to be performed to drain ?

10 replies

Alexis Hernandez

@AlexITC

I have this setting for the module logging slow queries but it seems it's ignored by kamon, any ideas?

kamon {
  instrumentation.jdbc.statements.threshold = 5 seconds
}

4 replies

Challen Herzberg-Brovold

@ChallenHB

Hello everyone

I am trying to use Kamon with an http4s app to report metrics to New Relic, and I have a couple questions:
1) The documentation says to call Kamon.init() as the very first operation in your code. I am deploying my service as a war, and I'm getting an java.lang.reflect.InvocationTargetException when I call it in the Context Listener. Where should I be calling it?
2) When I'm using the Kamon new relic reporter, does it use new relic transactions at all? I.e. will my service be able to integrate with New Relic's service map?

6 replies

Where communities thrive

People

Repo info

Activity