kamon-io/Kamon

The Open Source tool for monitoring applications running on the JVM

Ivan Topolnjak

@ivantopo

hey @Reeebuuk, indeed, try 2.1.3

Pavel Pliachystsik

@pavel.pliachystsik_gitlab

Hello everybody! I have Play framework application (2.8.1) and subproject with Akka cluster (2.6.3). I'm trying to integrate kamon, added play agent and javaagent to pluguins, enabled it in root project and see some attach errors:
[Attach Listener] ERROR 2020-07-10 09:23:15 Logger : Error => akka.dispatch.Envelope with message Cannot resolve type description for scala.Product. Class loader: java.net.URLClassLoader@2eea9be8: java.lang.IllegalStateException: Cannot resolve type description for scala.Product
at kanela.agent.libs.net.bytebuddy.pool.TypePool$Resolution$Illegal.resolve(TypePool.java:159)
at kanela.agent.libs.net.bytebuddy.pool.TypePool$Default$LazyTypeDescription$TokenizedGenericType.toErasure(TypePool.java:6241)
at kanela.agent.libs.net.bytebuddy.pool.TypePool$Default$LazyTypeDescription$GenericTypeToken$Resolution$Raw$RawAnnotatedType.of(TypePool.java:3412)
at kanela.agent.libs.net.bytebuddy.pool.TypePool$Default$LazyTypeDescription$GenericTypeToken$Resolution$Raw$RawAnnotatedType$LazyRawAnnotatedTypeList.get(TypePool.java:3511)
at kanela.agent.libs.net.bytebuddy.pool.TypePool$Default$LazyTypeDescription$GenericTypeToken$Resolution$Raw$RawAnnotatedType$LazyRawAnnotatedTypeList.get(TypePool.java:3456)
at java.base/java.util.AbstractList$Itr.next(AbstractList.java:371)
at kanela.agent.libs.net.bytebuddy.description.type.TypeList$Generic$AbstractBase.accept(TypeList.java:275)
at kanela.agent.libs.net.bytebuddy.dynamic.scaffold.InstrumentedType$Factory$Default$1.represent(InstrumentedType.java:362)
at kanela.agent.libs.net.bytebuddy.ByteBuddy.rebase(ByteBuddy.java:851)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$TypeStrategy$Default$1.builder(AgentBuilder.java:2007)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$Transformation$Simple$Resolution.apply(AgentBuilder.java:10117)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10494)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10460)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1500(AgentBuilder.java:10223)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10907)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10845)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10413)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)
at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)
at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:167)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$RedefinitionStrategy$Dispatcher$ForJava6CapableVm.retransformClasses(AgentBuilder.java:6865)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$RedefinitionStrategy$Collector$ForRetransformation.doApply(AgentBuilder.java:7142)
at kanela.agent.libs.net.bytebuddy.agent.builder.AgentBuilder$RedefinitionStrategy$Collector.apply(AgentBuilder.java:6991)
at kanela.agent.libs.net.bytebuddy.agent.bui

But after restart they always disappear, so I assume it's all because subpoject is still compiling. Running it in IntellijIdea

ang-mic

@ang-mic

Hi everyone, happy Sunday :) I am going through this tutorial that involves Zipkin and Prometheus. When I go to the URL for Prometheus, I can see the text but not the UI. What am I doing wrong? Thanks in advance.

Pavel Pliachystsik

@pavel.pliachystsik_gitlab

@ang-mic prometheus is standalone app, what kamon do, is export it in vew which prometheus can collect. You need to install prometheus to see ui, and grafana to ger great ui for your monitorring tool

ang-mic

@ang-mic

@pavel.pliachystsik_gitlab Thank you for coming back to me. That clears the water, I wll look into it :)

Alex

@forrana

Hi.

  # kamon.context.codecs.string-keys {
  #   request-id = "X-Request-ID"
  # }

Didn't managed to find anywhere - is it possible to setup fallback values for those?

Ivan Topolnjak

@ivantopo

hey

Alex

@forrana

The only way I've managed to do it - is to check the key in Kamon.currentContext and then create a new context with a key(if it's None) which then I use in a wrapper on HTTP call. Seems to me there should be an easier way?

Ivan Topolnjak

@ivantopo

correct me if I'm wrong, but are you asking if there is a way to generate a value for "request-id" if there was none in the current context?

Alex

@forrana

So the idea is I need to propagate a header "X-Request-ID" but if it's not set there should be a default value, yes.

Ivan Topolnjak

@ivantopo

there is no built-in way to go about it

but let me ask you a few questions to understand the problem better

you mentioned wrapping an HTTP call, is that a client or server HTTP call?

Alex

@forrana

So that's the way it works

val XKey = Key.broadcastString("x-key")
val context = 
    Kamon.currentContext().get(XKey) match {
      case None =>
        Kamon.currentContext().withKey(XKey, Some("XKey Value"))
      case _ => Kamon.currentContext()
    }

Kamon.withContext(context) {
    for {
        result <- httpClient(Get(uri))
    } yield result
}

I just hoped there should be an easier one

nmkarvekar

@nmkarvekar

I have question on kamon usage in production. Is it free to use kamon with instrumentation integrated with prometheus in production without APM? Is there any restriction to use that in terms of number of services?

Ivan Topolnjak

@ivantopo

hey there

yeap, it is free

no restrictions at all

basically, if you can find it on our github then its free :joy:

nmkarvekar

@nmkarvekar

Thanks Ivan. I was little bit confused on this https://kamon.io/apm/pricing/, Starter: up to 2 instances per service,up to 3 services, but as I understand it is if we use APM.

Ivan Topolnjak

@ivantopo

correct, that pricing is only for APM

nmkarvekar

@nmkarvekar

ok. thank you.

Vish Ramachandran

@vishramachandran

@ivantopo A few weeks earlier, I had asked a question about draining metrics to reporter before shutdown. The suggestion was to run Kamon.stopModules() from a shutdown hook to ensure that all metrics are drained to reporters and backends. I precisely did that and waited for the returned future to complete, but every now and then, I see missed samples. There is clear evidence from logs that the metric was published and stopModules call finished successfully, but the sample that was incremented moments before shutdown was not drained to reporters. Any other calls to be performed to drain ?

10 replies

Alexis Hernandez

@AlexITC

I have this setting for the module logging slow queries but it seems it's ignored by kamon, any ideas?

kamon {
  instrumentation.jdbc.statements.threshold = 5 seconds
}

4 replies

Challen Herzberg-Brovold

@ChallenHB

Hello everyone

I am trying to use Kamon with an http4s app to report metrics to New Relic, and I have a couple questions:
1) The documentation says to call Kamon.init() as the very first operation in your code. I am deploying my service as a war, and I'm getting an java.lang.reflect.InvocationTargetException when I call it in the Context Listener. Where should I be calling it?
2) When I'm using the Kamon new relic reporter, does it use new relic transactions at all? I.e. will my service be able to integrate with New Relic's service map?

6 replies

Thank you!

Challen Herzberg-Brovold

@ChallenHB

kamon-io/Kamon#611 makes me think not, but wondering if this has been addressed since then.

David Leonhart

@leozilla

Is there some up2date documentation howto configure kamon 2.1.x for lagom 1.6.x?
I am currently running into the same problems as described here: https://github.com/kamon-io/Kamon/issues/563#issuecomment-562867888
Kamon works when running from sbt but it doesnt work with production configuration.

1 reply

Henry

@hygt

hello, there's nothing like include-environment-tags for the Zipkin reporter?

4 replies

Franco Albornoz

@dannashirn

Hey guys, is there a way to enable the kamon-play for play 2.8 instrumentation without using kamon-bundle?, and instead using kamon-core + kamon-play?

4 replies

Daniel Leon

@leon-daniel

Hello! Has anyone needed kafka metrics so far ? I see kamon-kafka hasn't been imported into the main project

key-eugene

@key-eugene

Hi! Is there a way to specify custom bucket limits? for example i used seconds Kamon.histogram("tsr-last-trained-point", MeasurementUnit.time.seconds) and i get buckets like this

tsr_last_trained_point_seconds_bucket{le="0.005",granularity="HOUR"} 0.0
tsr_last_trained_point_seconds_bucket{le="0.01",granularity="HOUR"} 0.0
tsr_last_trained_point_seconds_bucket{le="0.025",granularity="HOUR"} 0.0
tsr_last_trained_point_seconds_bucket{le="0.05",granularity="HOUR"} 0.0
tsr_last_trained_point_seconds_bucket{le="0.075",granularity="HOUR"} 0.0
tsr_last_trained_point_seconds_bucket{le="0.1",granularity="HOUR"} 0.0
tsr_last_trained_point_seconds_bucket{le="0.25",granularity="HOUR"} 0.0
tsr_last_trained_point_seconds_bucket{le="0.5",granularity="HOUR"} 0.0
tsr_last_trained_point_seconds_bucket{le="0.75",granularity="HOUR"} 0.0
tsr_last_trained_point_seconds_bucket{le="1.0",granularity="HOUR"} 0.0
tsr_last_trained_point_seconds_bucket{le="2.5",granularity="HOUR"} 0.0
tsr_last_trained_point_seconds_bucket{le="5.0",granularity="HOUR"} 0.0
tsr_last_trained_point_seconds_bucket{le="7.5",granularity="HOUR"} 0.0
tsr_last_trained_point_seconds_bucket{le="10.0",granularity="HOUR"} 0.0
tsr_last_trained_point_seconds_bucket{le="+Inf",granularity="HOUR"} 50.0
tsr_last_trained_point_seconds_count{granularity="HOUR"} 50.0
tsr_last_trained_point_seconds_sum{granularity="HOUR"} 102740.0

But i want buckets like 3600 (hour), 86100(day), week and etc... How can i define histogram like this?

3 replies

Ivan Topolnjak

@ivantopo

dear peoplez!

Daniel Leon

@leon-daniel

@ivantopo any plans to migrate kamon-kafka into the main repository?

19 replies

David Leonhart

@leozilla

I already got kamon working using the full ?kamon-bundle but I can see that it adds a lot of dependencies which I dont need/want.
Whats the best way to pick only the things I need put still get the same auto instrumentation as if I would use the full bundle?

Lets say I only want to use: kamon-executors, kamon-scala-future, kamon-cassandra.
I think I could just exclude all other modules via libraryExclusions but imo the other way around, including only what I want, would be better.

Ivan Topolnjak

@ivantopo

in theory, kamon-cassandra should only bring up kamon-core, same with kamon-scala-future

oh, and common

are you seeing some extra dependencies being pulled?

David Leonhart

@leozilla

Oh ok, sry my bad. I was looking into the kamon build.sbt and saw that the kamon-bundle dependsOn a lot of modules and I mistakenly assumed that all those modules would be automatically pulled in if I add kamon-bundle as a dependency to my project.
I just checked the dependency tree of my project and I only see kamon-bundle and kamon-core beeing pulled in if I just specify kamon-bundle as a dependecy. So it looks fine.
I guess I need to explicitly specify all the modules I want as dependencies now. Which is exactly what I want. :-)

Ivan Topolnjak

@ivantopo

:thumbsup:

David Leonhart

@leozilla

Maybe one more question.
Whats the benefit of using the kamon metrics instead of just using the prometheus java client metrics directly?
Do u consider the kamon metrics Counter, Gauge etc more advanced or is it just a matter of additional abstraction over the actual metrics implementation which is beneficial?

Ivan Topolnjak

@ivantopo

from the pure metrics point of view, the main benefit would be that using Kamon allows you to move from Prometheus to other solutions, or use several solutions at the same time without having to change your instrumented code

and on top of that you add that Kamon also gives traces and context propagation :D

David Leonhart

@leozilla

And one more thing about my last question :-) I dont understand how the status page can work if I dont even see the kamon-status-page module beeing added?..

Ivan Topolnjak

@ivantopo

you don't need to explicitly start it, if it is in the classpath then Kamon will pick it up

chawlavishal

@chawlavishal

using kamon to report metrics from a play application (play 2.6.6)

libraryDependencies += "io.kamon" %% "kamon-bundle" % "2.1.0"
libraryDependencies += "io.kamon" %% "kamon-prometheus" % "2.1.0"

I create a custom metric that captures number of requests per customer account (there are a handful of those). I want to get the method called, and the http response status for each request. I initialize it:
code private val requestsByAccountCounter = Kamon.counter("requests_by_account")

I have an ActionBuilder where I increment the counter

requestsByAccountCounter.withTag("account", accountInHeader(request))
        .withTag("method", s"${uriBaseFromPath(request.path)}")
        .withTag("http_status", r.header.status).increment()

3 metrics are generated
requests_by_account_total{account="TestAccountId"} 0.0
requests_by_account_total{account="TestAccountId",method="/data/catalog"} 0.0
requests_by_account_total{account="TestAccountId",http_status="200",method="/data/catalog"} 34.0

I know I can increment each tag, but ideally I'd like only the 3rd metric to be generated.
How can I accomplish that?

1 reply

Where communities thrive

People

Repo info

Activity