Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Sep 15 12:49
    btomala commented #1002
  • Aug 31 12:41
    imRentable commented #1049
  • Aug 28 20:32
    haricoding commented #677
  • Aug 27 11:37
    ivantopo commented #766
  • Aug 25 08:43
    alexandru commented #766
  • Aug 25 08:37
    alexandru commented #766
  • Aug 24 12:10
    ivantopo opened #1055
  • Aug 24 09:54

    ivantopo on master

    update readme badges (compare)

  • Aug 24 09:35

    ivantopo on master

    update readme badges (compare)

  • Aug 18 14:02
    jatcwang commented #1021
  • Aug 18 12:49
    neboduus commented #1021
  • Aug 18 12:05
    jatcwang commented #1021
  • Aug 18 12:05
    jatcwang commented #1021
  • Aug 18 12:05
    jatcwang commented #1021
  • Aug 18 08:33
    neboduus commented #1021
  • Aug 17 11:09
    jatcwang commented #1021
  • Aug 17 10:49
    neboduus commented #1021
  • Aug 17 10:48
    neboduus commented #1021
  • Aug 16 07:11
    sgabalda commented #566
  • Jul 23 08:26

    SimunKaracic on v2.2.3

    (compare)

Sherif Mohamed
@sherifkandeel
This message was deleted
Hi guys, Is there plans to support a Redis module on Kanela?
8 replies
alexander-branevskiy
@alexander-branevskiy
Hello guys! Do we have some way to provide custom span id while constructing span?
23 replies
alexander-branevskiy
@alexander-branevskiy
Hello guys! Is it safe to use Kamon.storeContext in order to resolve this issue? kamon-io/Kamon#829
58 replies
Sherif Mohamed
@sherifkandeel
Hi guys, I have a simlpe question, I do not have to re-call kamon.init() everytime I call kamon.reconfigure() or do I? In other words, is it dangerous to re-call Kamon.init() again after calling it the first thing in the application?
2 replies
Gervais Blaise
@gervaisb

Hello there, I am trying to setup Kamon on our Play 2.8 Scala application. I was able to start Kamon and add the Prometheus exporter as such as privoding a custom gauge. However, while I am executing some requests and trigger actors on the system, I do not get any metrics about Akka.
I do have the HELP and TYPE lines for akka_remote_messages_inbound_size_bytes, akka_remote_messages_outbound_size_bytes, akka_remote_serialization_time_seconds and akka_remote_deserialization_time_seconds but no values. The status page report those metrics but nothing about the AkkaModule.

I am using "sbt-kanela-runner-play-2.8" % "2.0.9"with Sbt 1.3.13 and Scala 2.13.4. We are using compile time DI with a custom loader that initialize Kamon as documented:

Kamon.loadModules()
context.lifecycle
      .addStopHook(() => Kamon.stop())

My Akka filter configuration is quite simple:
actors.track { includes = [ "**" ] excludes = [ "application/system/**" ] }

Any help would be appreciated to be able to monitor our actors.
Thanks

35 replies
David Knapp
@Falmarri
any thoughts on this? https://gitter.im/kamon-io/Kamon?at=60823ed7a2ac0d38e7d277ef not sure if it was seen since i replied to my own thread a bunch. i can't figure out how to get the tests to pass, but not sure i want to spend more time if there's a better approach
David Knapp
@Falmarri
so i'm comparing newrelic's agent with kamon, and newrelic parses out database calls to create a more meaningful operation name like "select table_name". specifically here https://github.com/newrelic/newrelic-java-agent/blob/main/newrelic-agent/src/main/java/com/newrelic/agent/database/DefaultDatabaseStatementParser.java what do you guys think of implementing this? it can be some kind of config switch so we don't break existing operation names for users. and it can either be using regexes like newrelic does, or i found this https://github.com/JSQLParser/JSqlParser i don't know the performance implications of either though. i could imagine a radical change where we could push off the parsing of the sql onto separate threads to not block client code, but that would be a large change i would think.
David Knapp
@Falmarri
here's a proof of concept of the above kamon-io/Kamon#1013
alexander-branevskiy
@alexander-branevskiy
guys, does anyone have an example of how to serialize a span?
16 replies
Harshid Dattani
@harshid86

Hi, are there any plans to publish the kamon sbt plugins on maven, now that bintray has been shut?

sbt-aspectj-runner-play-2.6 v1.1.1 previously available via https://bintray.com/kamon-io/sbt-plugins/ is not available on maven nor https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/

1 reply
David Knapp
@Falmarri
i'm seeing some issues when having both newrelic's agent and kanela running on the same service. do you know of any quick ways to handle this? or is this just unsupported and i'm on my own?
2021-05-06_18:55:46.16634 [statsd-0] ERROR 2021-05-06 11:55:46  Logger : An error occurred while trying to apply an advisor: java.lang.ClassCastException: com.nr.agent.instrumentation.scala.WrappedTry cannot be cast to kamon.instrumentation.context.HasContext
2021-05-06_18:55:46.16637       at kamon.instrumentation.futures.scala.CallbackRunnableRunInstrumentation$.enter(FutureChainingInstrumentation.scala:90)
2021-05-06_18:55:46.16638       at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:63)
2021-05-06_18:55:46.16638       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
2021-05-06_18:55:46.16638       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
2021-05-06_18:55:46.16638       at java.lang.Thread.run(Thread.java:748)
2 replies
Pooriya-Shokri
@Pooriya-Shokri
Hi,
I've noticed three different metrics per kamon reference metrics. one with '_SUM' postfix and two others with '_BUCKET', and '_COUNT'.
is it possible to explain the differences between these metric postfixes and their meaning?
1 reply
Also it bugs me to explain some of weird metrics. For example this the "AKKA_GROUP_MEMBERS_SUM" metric for a local running project with at most 10 actors.
Can any one explain the increasing and surprisingly LARGE metric values?
Screenshot from 2021-05-08 15-57-42.png
Alexandru Nedelcu
@alexandru

Hello — we're getting an ArrayOutOfBoundsException related to tags usage, similar to this one:

kamon-io/Kamon#766

Any ideas?

1 reply
SimunKaracic
@SimunKaracic

Hey everyone, we have a new blog post :tada:
https://kamon.io/blog/monitoring-queues-and-resources-with-kamon-range-samplers/?utm_source=gitter

In this post we discuss the origin story of Kamon’s Range Samplers: what they are, how they work, and how we use them to monitor queues, connection pools and more.
Since there's been quite a bit of questions about how they work, please take a look at it!

alexander-branevskiy
@alexander-branevskiy
Can someone explain how adaptive-sampler.throughput works ?
alexander-branevskiy
@alexander-branevskiy
as I understood it's sth like maximal operations amount per minute, but I'm not 100% sure
8 replies
alexander-branevskiy
@alexander-branevskiy
guys, am I right that if spans number exceeds 4096 within tick-interval old spans will be skipped? https://github.com/kamon-io/Kamon/blob/4b397cbc242aaebbdc5dff5e28eee6cf0b5fac4d/core/kamon-core/src/main/scala/kamon/trace/Tracer.scala#L45,L46
alexander-branevskiy
@alexander-branevskiy
okay seems decreasing throughput resolves the issue
Jason Pickens
@steinybot
How do I set tags like the service and environment when using the Datadog integration?
Jason Pickens
@steinybot
Oh I think I may have found it. kamon.environment
Is that in the docs somewhere?
1 reply
Michael Morris
@micmorris
Hey all, I'm trying to get a simple Kamon setup pushing Zipkin traces on Akka-Http, but I'm running into this error: java.lang.ClassCastException: class akka.dispatch.Envelope cannot be cast to class kamon.instrumentation.context.HasContext
I've thrown so many configs at this, I'm not sure where to start
I've made sure that Kamon is the first thing loaded as far as I can see
18 replies
Jason Pickens
@steinybot
Have you tried adding the agent manually? For a while I thought I was calling Kamon.init early enough but that can be easier said than done. You could put a breakpoint in the ActorSystem initialisation somewhere and in the Kamon initialisation to double check which is hit first.
Michael Morris
@micmorris
Added some details into the reply thread above :)
I might try breakpointing and seeing if they do get called in order, sure. What do you mean by "manually" adding the agent?
Yaroslav Derman
@yarosman
@ivantopo @SimunKaracic Does kamon require additional configuration for Completable Future ? Because it seems to me that it doesn't work correct with caffeine-async-cache
3 replies
alexander-branevskiy
@alexander-branevskiy
probably it's related to this one kamon-io/Kamon#829
10 replies
Jason Pickens
@steinybot
In local development I am using Jaeger. Traces don’t always appear. I’m not sure if they never do or they just show up late. How can I figure out what the problem might be?
Jason Pickens
@steinybot
Oh I see in the thread above haha. "They're getting sampled, which means that Kamon is deciding whether to trace that operation or not. If it actually traced all your requests, there would be a significant performance penalty. It always trips people up when they try Kamon for the first time.” Yep that tripped me up
Jason Pickens
@steinybot
It’s strange how it always seems to be the first few traces upon startup that are missing. I would have expected the adaptive sampler to keep these?
2 replies
Michael Morris
@micmorris

Hey all, I'm trying to filter out some health-checks with trace groups, but everything is still getting sent regardless :/

trace {
    sampler = ${?TRACE_SAMPLING}
    random-sampler.probability = ${?TRACE_SAMPLING_RANDOM_PROBABILITY}
    adaptive-sampler {
      groups {
        health-checks {
          operations = ["GET /is_initialized"]
          rules {
            sample = never
          }
        }
        metric-checks {
          operations = ["GET /prometheus"]
          rules {
            sample = never
          }
        }
        my-endpoint {
          operations = ["GET /my-endpoint", "POST /my-endpoint"]
          rules {
            sample = always
          }
        }
      }
    }
  }

TRACE_SAMPLING = adaptive

I can't find the logic where the operations are parsed out and related to Akka-http endpoints, I bet I could find something if I knew that location!
Michael Morris
@micmorris
Found it, the docs were messing with me again...
The operations never have the verb in front operations = ["GET \/status"], it's just operations = ["\/status"]
2 replies
Jason Pickens
@steinybot
What is the best way to find out why a thread has a context which I don’t think it should?
4 replies
David Knapp
@Falmarri
for the prometheus-reporter. the default configured
information-buckets = [ 512, 1024, 2048, 4096, 16384, 65536, 524288, 1048576 ] seems really small. the top bucket is only 1 megabyte. so basically all jvm.memory.* histograms get set to +Inf. is there a reason for these buckets to be set like this? should the reporter include custom buckets for jvm.* metrics?
2 replies
alexander-branevskiy
@alexander-branevskiy
Hello guys! Does Akka instrumentation support actor Stash?
3 replies
Rajat Khandelwal
@prongs

kamon-io/sbt-kanela-runner#18

can somebody take this up? Due to bintray sunset, our project has started to fail while building.

Ben Rice
@Rendrik
Hi, I've added Kamon to our project, and things are going smoothly. However we have a couple unruly noisy processes. One out of my control makes JDBC calls quite often that I'd like to just ignore.
Following some filtering doc here https://kamon.io/docs/latest/core/utilities/ However I'm unclear on exactly what the string is I'm filtering on? Is it possible to be granular enough to filter out spans to a specific DB? Or perhaps containing an SQL match?
Ben Rice
@Rendrik
I'm looking through adaptive-sampler as well. If I interpret the operations there to match (what I see in APM) as operation name, eg '/test', then it may not work for me as the JDBC operation is simple 'update', or 'query'.
I'm not sure at this point that I can selectively narrow down JDBC sampling
2 replies
alexander-branevskiy
@alexander-branevskiy
Hello guys! We are trying to integrate Kamon metrics based on timers and faced off with problems. Right now we are observing vast performance degradation during load testing. Can you recommend what parameters we can tune to resolve this?
4 replies

we tried to tune this stuff

timer {
auto-update-interval = 10 seconds
lowest-discernible-value = 1
highest-trackable-value = 3600000000000
significant-value-digits = 2
}

alexander-branevskiy
@alexander-branevskiy
the overall picture became better but still insufficient for us
Ben Rice
@Rendrik

I have a standard play application, and the following seems to have no effect at all (even when I set sampler=never)

kamon.metric.trace {
  sampler = "adaptive"
  adaptive-sampler {
    groups {
      aws-checks {
        operations = [
          "\/api\/test",
          "\/test"
        ]
        rules {
          sample = never
        }
      }
    }
  }
}

Everything goes into APM, and I can't seem to filter things out. The config is certainly being read; at least by APM to fetch the API key. This is inside my play config conf/application.conf, and I'm using the standard runner plugin addSbtPlugin("io.kamon" % "sbt-kanela-runner-play-2.8" % "2.0.10") and the bundle dependencies. Any clues why these wouldn't take effect?

"io.kamon" %% "kamon-bundle" % "2.1.18",
 "io.kamon" %% "kamon-apm-reporter" % "2.1.18",
2 replies
Pooriya-Shokri
@Pooriya-Shokri
Hi all,
I've enabled zipkin tracing by the code snippet below in one of my actors.
The actor receives messages from Akka streams socket, so I had to create context and span by myself.
While tracing looks fine in development mode, weird traces are seen in production. As you can see in following pictures, there are traces with hundreds of spans, while my app only consists of some simple actors with no circular message passing. I guess it's a problem of trace-id generation but no evidence on that. Any help appreciated.
1 reply
Screenshot from 2021-05-23 08-54-59.png