Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Sep 15 12:49
    btomala commented #1002
  • Aug 31 12:41
    imRentable commented #1049
  • Aug 28 20:32
    haricoding commented #677
  • Aug 27 11:37
    ivantopo commented #766
  • Aug 25 08:43
    alexandru commented #766
  • Aug 25 08:37
    alexandru commented #766
  • Aug 24 12:10
    ivantopo opened #1055
  • Aug 24 09:54

    ivantopo on master

    update readme badges (compare)

  • Aug 24 09:35

    ivantopo on master

    update readme badges (compare)

  • Aug 18 14:02
    jatcwang commented #1021
  • Aug 18 12:49
    neboduus commented #1021
  • Aug 18 12:05
    jatcwang commented #1021
  • Aug 18 12:05
    jatcwang commented #1021
  • Aug 18 12:05
    jatcwang commented #1021
  • Aug 18 08:33
    neboduus commented #1021
  • Aug 17 11:09
    jatcwang commented #1021
  • Aug 17 10:49
    neboduus commented #1021
  • Aug 17 10:48
    neboduus commented #1021
  • Aug 16 07:11
    sgabalda commented #566
  • Jul 23 08:26

    SimunKaracic on v2.2.3

    (compare)

David Knapp
@Falmarri
i'm seeing some issues when having both newrelic's agent and kanela running on the same service. do you know of any quick ways to handle this? or is this just unsupported and i'm on my own?
2021-05-06_18:55:46.16634 [statsd-0] ERROR 2021-05-06 11:55:46  Logger : An error occurred while trying to apply an advisor: java.lang.ClassCastException: com.nr.agent.instrumentation.scala.WrappedTry cannot be cast to kamon.instrumentation.context.HasContext
2021-05-06_18:55:46.16637       at kamon.instrumentation.futures.scala.CallbackRunnableRunInstrumentation$.enter(FutureChainingInstrumentation.scala:90)
2021-05-06_18:55:46.16638       at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:63)
2021-05-06_18:55:46.16638       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
2021-05-06_18:55:46.16638       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
2021-05-06_18:55:46.16638       at java.lang.Thread.run(Thread.java:748)
2 replies
Pooriya-Shokri
@Pooriya-Shokri
Hi,
I've noticed three different metrics per kamon reference metrics. one with '_SUM' postfix and two others with '_BUCKET', and '_COUNT'.
is it possible to explain the differences between these metric postfixes and their meaning?
1 reply
Also it bugs me to explain some of weird metrics. For example this the "AKKA_GROUP_MEMBERS_SUM" metric for a local running project with at most 10 actors.
Can any one explain the increasing and surprisingly LARGE metric values?
Screenshot from 2021-05-08 15-57-42.png
Alexandru Nedelcu
@alexandru

Hello — we're getting an ArrayOutOfBoundsException related to tags usage, similar to this one:

kamon-io/Kamon#766

Any ideas?

1 reply
SimunKaracic
@SimunKaracic

Hey everyone, we have a new blog post :tada:
https://kamon.io/blog/monitoring-queues-and-resources-with-kamon-range-samplers/?utm_source=gitter

In this post we discuss the origin story of Kamon’s Range Samplers: what they are, how they work, and how we use them to monitor queues, connection pools and more.
Since there's been quite a bit of questions about how they work, please take a look at it!

alexander-branevskiy
@alexander-branevskiy
Can someone explain how adaptive-sampler.throughput works ?
alexander-branevskiy
@alexander-branevskiy
as I understood it's sth like maximal operations amount per minute, but I'm not 100% sure
8 replies
alexander-branevskiy
@alexander-branevskiy
guys, am I right that if spans number exceeds 4096 within tick-interval old spans will be skipped? https://github.com/kamon-io/Kamon/blob/4b397cbc242aaebbdc5dff5e28eee6cf0b5fac4d/core/kamon-core/src/main/scala/kamon/trace/Tracer.scala#L45,L46
alexander-branevskiy
@alexander-branevskiy
okay seems decreasing throughput resolves the issue
Jason Pickens
@steinybot
How do I set tags like the service and environment when using the Datadog integration?
Jason Pickens
@steinybot
Oh I think I may have found it. kamon.environment
Is that in the docs somewhere?
1 reply
Michael Morris
@micmorris
Hey all, I'm trying to get a simple Kamon setup pushing Zipkin traces on Akka-Http, but I'm running into this error: java.lang.ClassCastException: class akka.dispatch.Envelope cannot be cast to class kamon.instrumentation.context.HasContext
I've thrown so many configs at this, I'm not sure where to start
I've made sure that Kamon is the first thing loaded as far as I can see
18 replies
Jason Pickens
@steinybot
Have you tried adding the agent manually? For a while I thought I was calling Kamon.init early enough but that can be easier said than done. You could put a breakpoint in the ActorSystem initialisation somewhere and in the Kamon initialisation to double check which is hit first.
Michael Morris
@micmorris
Added some details into the reply thread above :)
I might try breakpointing and seeing if they do get called in order, sure. What do you mean by "manually" adding the agent?
Yaroslav Derman
@yarosman
@ivantopo @SimunKaracic Does kamon require additional configuration for Completable Future ? Because it seems to me that it doesn't work correct with caffeine-async-cache
3 replies
alexander-branevskiy
@alexander-branevskiy
probably it's related to this one kamon-io/Kamon#829
10 replies
Jason Pickens
@steinybot
In local development I am using Jaeger. Traces don’t always appear. I’m not sure if they never do or they just show up late. How can I figure out what the problem might be?
Jason Pickens
@steinybot
Oh I see in the thread above haha. "They're getting sampled, which means that Kamon is deciding whether to trace that operation or not. If it actually traced all your requests, there would be a significant performance penalty. It always trips people up when they try Kamon for the first time.” Yep that tripped me up
Jason Pickens
@steinybot
It’s strange how it always seems to be the first few traces upon startup that are missing. I would have expected the adaptive sampler to keep these?
2 replies
Michael Morris
@micmorris

Hey all, I'm trying to filter out some health-checks with trace groups, but everything is still getting sent regardless :/

trace {
    sampler = ${?TRACE_SAMPLING}
    random-sampler.probability = ${?TRACE_SAMPLING_RANDOM_PROBABILITY}
    adaptive-sampler {
      groups {
        health-checks {
          operations = ["GET /is_initialized"]
          rules {
            sample = never
          }
        }
        metric-checks {
          operations = ["GET /prometheus"]
          rules {
            sample = never
          }
        }
        my-endpoint {
          operations = ["GET /my-endpoint", "POST /my-endpoint"]
          rules {
            sample = always
          }
        }
      }
    }
  }

TRACE_SAMPLING = adaptive

I can't find the logic where the operations are parsed out and related to Akka-http endpoints, I bet I could find something if I knew that location!
Michael Morris
@micmorris
Found it, the docs were messing with me again...
The operations never have the verb in front operations = ["GET \/status"], it's just operations = ["\/status"]
2 replies
Jason Pickens
@steinybot
What is the best way to find out why a thread has a context which I don’t think it should?
4 replies
David Knapp
@Falmarri
for the prometheus-reporter. the default configured
information-buckets = [ 512, 1024, 2048, 4096, 16384, 65536, 524288, 1048576 ] seems really small. the top bucket is only 1 megabyte. so basically all jvm.memory.* histograms get set to +Inf. is there a reason for these buckets to be set like this? should the reporter include custom buckets for jvm.* metrics?
2 replies
alexander-branevskiy
@alexander-branevskiy
Hello guys! Does Akka instrumentation support actor Stash?
3 replies
Rajat Khandelwal
@prongs

kamon-io/sbt-kanela-runner#18

can somebody take this up? Due to bintray sunset, our project has started to fail while building.

Ben Rice
@Rendrik
Hi, I've added Kamon to our project, and things are going smoothly. However we have a couple unruly noisy processes. One out of my control makes JDBC calls quite often that I'd like to just ignore.
Following some filtering doc here https://kamon.io/docs/latest/core/utilities/ However I'm unclear on exactly what the string is I'm filtering on? Is it possible to be granular enough to filter out spans to a specific DB? Or perhaps containing an SQL match?
Ben Rice
@Rendrik
I'm looking through adaptive-sampler as well. If I interpret the operations there to match (what I see in APM) as operation name, eg '/test', then it may not work for me as the JDBC operation is simple 'update', or 'query'.
I'm not sure at this point that I can selectively narrow down JDBC sampling
2 replies
alexander-branevskiy
@alexander-branevskiy
Hello guys! We are trying to integrate Kamon metrics based on timers and faced off with problems. Right now we are observing vast performance degradation during load testing. Can you recommend what parameters we can tune to resolve this?
4 replies

we tried to tune this stuff

timer {
auto-update-interval = 10 seconds
lowest-discernible-value = 1
highest-trackable-value = 3600000000000
significant-value-digits = 2
}

alexander-branevskiy
@alexander-branevskiy
the overall picture became better but still insufficient for us
Ben Rice
@Rendrik

I have a standard play application, and the following seems to have no effect at all (even when I set sampler=never)

kamon.metric.trace {
  sampler = "adaptive"
  adaptive-sampler {
    groups {
      aws-checks {
        operations = [
          "\/api\/test",
          "\/test"
        ]
        rules {
          sample = never
        }
      }
    }
  }
}

Everything goes into APM, and I can't seem to filter things out. The config is certainly being read; at least by APM to fetch the API key. This is inside my play config conf/application.conf, and I'm using the standard runner plugin addSbtPlugin("io.kamon" % "sbt-kanela-runner-play-2.8" % "2.0.10") and the bundle dependencies. Any clues why these wouldn't take effect?

"io.kamon" %% "kamon-bundle" % "2.1.18",
 "io.kamon" %% "kamon-apm-reporter" % "2.1.18",
2 replies
Pooriya-Shokri
@Pooriya-Shokri
Hi all,
I've enabled zipkin tracing by the code snippet below in one of my actors.
The actor receives messages from Akka streams socket, so I had to create context and span by myself.
While tracing looks fine in development mode, weird traces are seen in production. As you can see in following pictures, there are traces with hundreds of spans, while my app only consists of some simple actors with no circular message passing. I guess it's a problem of trace-id generation but no evidence on that. Any help appreciated.
1 reply
Screenshot from 2021-05-23 08-54-59.png
MicrosoftTeams-image (1).png
MicrosoftTeams-image.png
alexander-branevskiy
@alexander-branevskiy
any updates here ? kamon-io/Kamon#829
@SimunKaracic you told that someone is working on it
3 replies
Srepfler Srdan
@schrepfler
I'm getting this error on kanela startup, do we need to add some dependencies next to kanela?
Please remove -javaagent from your startup arguments and contact Kanela support.: java.lang.NoClassDefFoundError: redis/clients/jedis/commands/ProtocolCommand
3 replies
schrepfler
@schrepfler:matrix.org
[m]
no worries! is setting the API key mandatory? the docs all seem to have it and I'd be happy with status-page+prometheus+zipkin/jaeger
9 replies
Diego Parra
@dpsoft
Hi all, we just published Kanela 1.0.10 with support for Java 16 and other goodies https://github.com/kamon-io/kanela/releases/tag/v1.0.10
5 replies
jamespass
@jamespass
Hello all, I am currently removing instruments from Kamon using .remove() but after some time I am still seeing the the metric on my applications metric endpoint and subsequently Prometheus. Is there a way to have the metric removed from the metrics endpoint instantly? Thanks
1 reply
Dinesh Narayanan
@ndchandar

Hello,
I am having Context propagation issues when switching between Cats Effect IO and Scala Futures with the newer version (2.2.0). I don't see issues with Cats Effect IO + Default Scala global execution context.

When using Akka Dispatchers I seem to be losing context. Sharing minimal example https://gist.github.com/ndchandar/0c54f348a72308d3abb1741f311c650c
Appreciate your help on this.

1 reply
ramyareddy
@ramyareddy:matrix.org
[m]
1 reply
Hello everyone. I'm experimenting with Kamon 2.0, akka-http/akka and integration with Prometheus. I am not sure why I am getting this error "expected equal, got "INVALID" and showing end point is down Standalone Prometheus instance scraping metrics endpoint shows tartget as down due to this error: expected equal, got "INVALID"