Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
Repo info
  • Mar 23 12:54
    ivantopo commented #1094
  • Mar 23 09:34
    ivantopo commented #1094
  • Mar 23 08:43
    ivantopo commented #1094
  • Mar 23 08:42
    ivantopo commented #1144
  • Mar 23 08:36
    ivantopo opened #1147
  • Mar 23 02:34
    mofei100 commented #1144
  • Mar 23 02:33
    mofei100 closed #1144
  • Mar 21 15:36
    seglo edited #1145
  • Mar 21 15:33
    seglo synchronize #1145
  • Mar 21 08:25
    sebarys opened #1146
  • Mar 20 18:16
    sebarys commented #793
  • Mar 20 18:14
    sebarys commented #793
  • Mar 19 23:41
    seglo opened #1145
  • Mar 18 14:33
    cmcmteixeira commented #1127
  • Mar 18 02:42
    mofei100 edited #1144
  • Mar 18 02:22
    mofei100 opened #1144
  • Mar 17 19:48
    hughsimpson commented #1127
  • Mar 17 19:10
    hughsimpson commented #1127
  • Mar 17 11:55
    ivantopo closed #1046
  • Mar 17 11:55
    ivantopo commented #1046
Jakub Kozłowski
if that's not possible, I can see I can call .tag on the span directly, but I can't read it that way - or can I?
Hi kamon aficionados, I would like to get your opinion on this subject matter. I've gotten feedback from a colleague that kamon has issues with the byte code generation, that it could damage your artifact, well in short that it's not reliable. And of course he heard this from colleagues who are not around anymore, so I can't verify this. Anybody has had such an experience before?
5 replies
Daniel van der Ende
Hi, I'm trying to integrate Kamon with an Akka HTTP app. The docs seem to suggest that it should work out of the box (e.g. add dependency to build.sbt, run Kamon.init()). I can see JVM and other system metrics, but 0 Akka HTTP metrics. Any idea what the problem here could be? Thanks!
34 replies


we use for our Scala application the Kanela agent 1.0.5. Our application runs with openjdk-14.01 in a docker container.
When the kanela agent is initialized it partially stops.

_  __                _        ______
| |/ /               | |       \ \ \ \
| ' / __ _ _ __   ___| | __ _   \ \ \ \
|  < / _` | '_ \ / _ \ |/ _` |   ) ) ) )
| . \ (_| | | | |  __/ | (_| |  / / / /
|_|\_\__,_|_| |_|\___|_|\__,_| /_/_/_/

Running with Kanela, the Kamon Instrumentation Agent :: (v1.0.5)
[main] INFO 2020-06-29 12:45:02  Logger : The Module: Executor Service Capture on Submit Instrumentation is disabled
[main] INFO 2020-06-29 12:45:02  Logger : The Module: Akka Remote Instrumentation is disabled
[main] INFO 2020-06-29 12:45:02  Logger : Loading Akka Instrumentation
[main] INFO 2020-06-29 12:45:02  Logger :  ==> Loading kamon.instrumentation.akka.instrumentations.EnvelopeInstrumentation
[main] INFO 2020-06-29 12:45:03  Logger :  ==> Loading kamon.instrumentation.akka.instrumentations.SystemMessageInstrumentation
[main] INFO 2020-06-29 12:45:03  Logger :  ==> Loading kamon.instrumentation.akka.instrumentations.RouterInstrumentation
[main] INFO 2020-06-29 12:45:03  Logger :  ==> Loading kamon.instrumentation.akka.instrumentations.ActorInstrumentation
[main] INFO 2020-06-29 12:45:03  Logger :  ==> Loading kamon.instrumentation.akka.instrumentations.ActorLoggingInstrumentation
[main] INFO 2020-06-29 12:45:03  Logger :  ==> Loading kamon.instrumentation.akka.instrumentations.AskPatternInstrumentation
[main] INFO 2020-06-29 12:45:03  Logger :  ==> Loading kamon.instrumentation.akka.instrumentations.EventStreamInstrumentation
[main] INFO 2020-06-29 12:45:03  Logger :  ==> Loading kamon.instrumentation.akka.instrumentations.ActorRefInstrumentation
[main] INFO 2020-06-29 12:45:03  Logger :  ==> Loading kamon.instrumentation.akka.instrumentations.akka_25.DispatcherInstrumentation
[main] INFO 2020-06-29 12:45:03  Logger :  ==> Loading kamon.instrumentation.akka.instrumentations.akka_26.DispatcherInstrumentation
[main] INFO 2020-06-29 12:45:04  Logger : Loading Executor Service Instrumentation
[main] INFO 2020-06-29 12:45:04  Logger :  ==> Loading kamon.instrumentation.executor.ExecutorTaskInstrumentation

Without docker we don't have this problem. With openjdk-13 it also works with docker. Does anyone know this behaviour?

9 replies
Yaroslav Derman

Hello. Does anyone try to use kamon with zio ?

@ivantopo Have you tried to run kamon with zio, because I don't have context propagation and correct span creation ((

Ivan Topolnjak
no, didn't try it myself
didn't even try ZIO yet
Hi, is there any interest to allow Kamon's Context to be serialised in HTTP according to the W3C Correlation Context proposal? https://w3c.github.io/correlation-context/
I could see this as a big plus for interoperability in a mixed-tech architecture. It's not standardised yet, but once finalised, would you consider it/would you be open for a PR or does something speak against it in general?
I could see it implemented similar to Trace Context Codecs so that as a user you could choose how to serialise it.
ATM the only difference in how Kamon serialises context and the W3C proposal is that key-value pairs are separated with ; in Kamon and with , in the proposal (apart from the actual name of the header value - which is configurable already)
5 replies
Franco Albornoz
Hey guys, so I had a shared library with Kamon 1.x that used the SpanCustomizer which I am trying to replace now with a PreStartHook. The thing is I used to have the customizer with a constructor with two parameters which were used to then customize the spans. The thing is PreStartHook needs to have a parameterless constructor, but I'm not sure how I'd be able to customize the spans more dynamically with it.
6 replies
Hi all, is there a preferred way to propagate trace contexts and context tags via GRPC? I can see this issue has been around for a while kamon-io/Kamon#616 (assuming this is for akka-grpc) and there is also this repo: https://github.com/nezasa/kamon-akka-grpc. Is there a plan for the future?
4 replies

Hello, Kamon dev. Here’s my config https://controlc.com/cf392b75. I can’t send anything to influxdb with Kamon because of

ERROR kamon.influxdb.InfluxDBReporter - Metrics POST to InfluxDB failed with status code [404], response body: {"error":"database not found: \"mydb\””}

error. I can’t understand where did it find mydb? I have db0 in configs.

7 replies
Ramakrishna Hande
Hi, Is the header name X-B3-TraceId renamed to trace-id ? We were using some old version of zipkin but now after some upgrade we have response having header trace-id. We are using Zipkin version 2.12.3. Thanks in advance
3 replies
Hello, there! Is it possible to add multiple custom fields to histogram? Not the tags. And is there any way to create a custom metric with Kamon without fields p50.0 p70.0 p90.0 p95.0 p99.0 p99.9 service?
2 replies
Hi, after upgrading to bundle and prometheus 2.1.2 I'm getting the following exception upon startup
Unable to start Kanela Agent. Please remove -javaagent from your startup arguments and contact Kanela support.: java.lang.NoClassDefFoundError: com/datastax/driver/core/Session
6 replies
I don't want to ask like this but since my pull-request is being ignored completely for almost 2 months i don't know how else to contact. Is there something that i forgot to do?
17 replies
Alexis Hernandez
BTW, the gitter activity on the right hasn't been updated for a year
1 reply
Alexis Hernandez
Last item is from 2019
Ivan Topolnjak
ha, didn't see that before
Enzo Bonggio

Hi I’m using kanela-agent on a play project and I want to add support for httpok. So I added on my build.sbt -> "io.kamon" %% "kamon-okhttp" % "1.0.3” and the configuration to enable it as it says on reference.conf
The error that I’m facing right now is the following

Caused by: java.lang.ClassNotFoundException: kanela.agent.api.instrumentation.KanelaInstrumentation
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at kanela.agent.util.classloader.ChildFirstURLClassLoader.loadClass(ChildFirstURLClassLoader.java:58)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at kanela.agent.InstrumentationLoader.lambda$loadInstrumentation$b85f757d$1(InstrumentationLoader.java:58)
    at kanela.agent.libs.io.vavr.control.Try.of(Try.java:75)
    at kanela.agent.InstrumentationLoader.loadInstrumentation(InstrumentationLoader.java:56)
    at kanela.agent.InstrumentationLoader.lambda$null$1(InstrumentationLoader.java:46)
    at kanela.agent.libs.io.vavr.collection.List.flatMap(List.java:898)
    at kanela.agent.InstrumentationLoader.lambda$load$4(InstrumentationLoader.java:46)
    at kanela.agent.libs.io.vavr.collection.List.map(List.java:1038)
    at kanela.agent.InstrumentationLoader.load(InstrumentationLoader.java:43)
    at kanela.agent.Kanela.lambda$null$0(Kanela.java:79)
    at kanela.agent.util.classloader.InstrumentationClassPath.use(InstrumentationClassPath.java:84)
    at kanela.agent.Kanela.lambda$start$1(Kanela.java:68)
    at kanela.agent.util.Execution.timed(Execution.java:32)
    at kanela.agent.util.Execution.runWithTimeSpent(Execution.java:41)
    at kanela.agent.Kanela.start(Kanela.java:67)
    at kanela.agent.Kanela.premain(Kanela.java:47)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386)
    at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401)

Should I add another dependency ? Also I could not find any examples of this so I’m lost

3 replies
Krunoslav Uzelac
Hi guys, I'm having the Unable to start Kanela Agent. Please remove -javaagent from your startup arguments and contact Kanela support issue
javaAgents += "io.kamon" % "kanela-agent" % "1.0.5",
addSbtPlugin("io.kamon" % "sbt-kanela-runner" % "2.0.6")
addSbtPlugin("com.lightbend.sbt" % "sbt-javaagent" % "0.1.5")
any hints? :)
will try 2.1.3
Ivan Topolnjak
hey @Reeebuuk, indeed, try 2.1.3
Pavel Pliachystsik
Hello everybody! I have Play framework application (2.8.1) and subproject with Akka cluster (2.6.3). I'm trying to integrate kamon, added play agent and javaagent to pluguins, enabled it in root project and see some attach errors:
[Attach Listener] ERROR 2020-07-10 09:23:15 Logger : Error => akka.dispatch.Envelope with message Cannot resolve type description for scala.Product. Class loader: java.net.URLClassLoader@2eea9be8: java.lang.IllegalStateException: Cannot resolve type description for scala.Product
at kanela.agent.libs.net.bytebuddy.pool.TypePool$Resolution$Illegal.resolve(TypePool.java:159)
at kanela.agent.libs.net.bytebuddy.pool.TypePool$Default$LazyTypeDescription$TokenizedGenericType.toErasure(TypePool.java:6241)
at kanela.agent.libs.net.bytebuddy.pool.TypePool$Default$LazyTypeDescription$GenericTypeToken$Resolution$Raw$RawAnnotatedType.of(TypePool.java:3412)
at kanela.agent.libs.net.bytebuddy.pool.TypePool$Default$LazyTypeDescription$GenericTypeToken$Resolution$Raw$RawAnnotatedType$LazyRawAnnotatedTypeList.get(TypePool.java:3511)
at kanela.agent.libs.net.bytebuddy.pool.TypePool$Default$LazyTypeDescription$GenericTypeToken$Resolution$Raw$RawAnnotatedType$LazyRawAnnotatedTypeList.get(TypePool.java:3456)
at java.base/java.util.AbstractList$Itr.next(AbstractList.java:371)
at kanela.agent.libs.net.bytebuddy.description.type.TypeList$Generic$AbstractBase.accept(TypeList.java:275)
at kanela.agent.libs.net.bytebuddy.dynamic.scaffold.InstrumentedType$Factory$Default$1.represent(InstrumentedType.java:362)
at kanela.agent.libs.net.bytebuddy.ByteBuddy.rebase(ByteBuddy.java:851)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$TypeStrategy$Default$1.builder(AgentBuilder.java:2007)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$Transformation$Simple$Resolution.apply(AgentBuilder.java:10117)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10494)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10460)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1500(AgentBuilder.java:10223)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10907)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10845)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10413)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)
at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)
at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:167)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$RedefinitionStrategy$Dispatcher$ForJava6CapableVm.retransformClasses(AgentBuilder.java:6865)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$RedefinitionStrategy$Collector$ForRetransformation.doApply(AgentBuilder.java:7142)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$RedefinitionStrategy$Collector.apply(AgentBuilder.java:6991)
at kanela.agent.libs.net.bytebuddy.agent.bui
But after restart they always disappear, so I assume it's all because subpoject is still compiling. Running it in IntellijIdea
Hi everyone, happy Sunday :) I am going through this tutorial that involves Zipkin and Prometheus. When I go to the URL for Prometheus, I can see the text but not the UI. What am I doing wrong? Thanks in advance.
Pavel Pliachystsik
@ang-mic prometheus is standalone app, what kamon do, is export it in vew which prometheus can collect. You need to install prometheus to see ui, and grafana to ger great ui for your monitorring tool
@pavel.pliachystsik_gitlab Thank you for coming back to me. That clears the water, I wll look into it :)
  # kamon.context.codecs.string-keys {
  #   request-id = "X-Request-ID"
  # }
Didn't managed to find anywhere - is it possible to setup fallback values for those?
Ivan Topolnjak
The only way I've managed to do it - is to check the key in Kamon.currentContext and then create a new context with a key(if it's None) which then I use in a wrapper on HTTP call. Seems to me there should be an easier way?
Ivan Topolnjak
correct me if I'm wrong, but are you asking if there is a way to generate a value for "request-id" if there was none in the current context?
So the idea is I need to propagate a header "X-Request-ID" but if it's not set there should be a default value, yes.
Ivan Topolnjak
there is no built-in way to go about it
but let me ask you a few questions to understand the problem better
you mentioned wrapping an HTTP call, is that a client or server HTTP call?
So that's the way it works
val XKey = Key.broadcastString("x-key")
val context = 
    Kamon.currentContext().get(XKey) match {
      case None =>
        Kamon.currentContext().withKey(XKey, Some("XKey Value"))
      case _ => Kamon.currentContext()

Kamon.withContext(context) {
    for {
        result <- httpClient(Get(uri))
    } yield result
I just hoped there should be an easier one
I have question on kamon usage in production. Is it free to use kamon with instrumentation integrated with prometheus in production without APM? Is there any restriction to use that in terms of number of services?
Ivan Topolnjak
hey there
yeap, it is free
no restrictions at all
basically, if you can find it on our github then its free :joy:
Thanks Ivan. I was little bit confused on this https://kamon.io/apm/pricing/, Starter: up to 2 instances per service,up to 3 services, but as I understand it is if we use APM.
Ivan Topolnjak
correct, that pricing is only for APM
ok. thank you.