Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    λoλcat
    @catostrophe
    seems we're ready for release
    Chris Jansen
    @janstenpickle
    sorry was just walking the dog, I’ll do a release now
    λoλcat
    @catostrophe
    something happened
    Chris Jansen
    @janstenpickle
    yeah, the 2.13 build got stuck so I restarted
    Chris Jansen
    @janstenpickle
    ok 0.11.0 is out :tada:
    Chris Jansen
    @janstenpickle
    λoλcat
    @catostrophe
    janstenpickle/trace4cats#374
    now we can migrate to CE3
    Chris Jansen
    @janstenpickle
    :tada:
    λoλcat
    @catostrophe
    btw, what happens when we merge CE3 to master? github action will release something like 0.11.0+1-34ae56a5e
    maybe we should merge PR + create v0.12.0-RC1 tag
    to prevent wrong versioning of artifacts
    Chris Jansen
    @janstenpickle
    hmm yeah, that sounds like a good idea
    λoλcat
    @catostrophe
    ok. i will do it
    λoλcat
    @catostrophe
    releasing v0.12.0-RC1
    Chris Jansen
    @janstenpickle
    :+1:
    Chris Jansen
    @janstenpickle
    :tada:
    Ben Plommer
    @bplommer
    What use cases are people using the kafka exporter for? I'm wondering if it could be used to pipe traces to ad hoc Jaeger instances on dev machines until we can persuade $work to invest in something more bespoke
    Chris Jansen
    @janstenpickle
    While we haven’t fully deployed it yet, we plan on using it for pretty much what you describe; having multiple sinks for traces. But also so we can centrally configure and control rate via sampling in the collector, which is useful if you’re using a SaaS service and care about costs
    You can also have different sampling polcies depending on where you send your traces, for example I want to deploy a Jeager instance for high resolution traces, with a retention period of ~24 hours, and use Google Cloud Trace at a lower resolution with its default retention window
    This could be achieved using multiple collectors and forwarding, but it’s more fiddly than using Kafka. Especially when scaling out and retaining sample decisions
    I have some diagrams in the docs which hopefully show this in a bit more detail: https://github.com/janstenpickle/trace4cats/blob/master/docs/topologies.md
    Ben Plommer
    @bplommer
    Cool I'll take a look :)
    Chris Jansen
    @janstenpickle
    np :)
    btw, it is linked from that doc, but tail sampling may be of some interest. Because we use the trace ID as the key on spans that are added to kafka, we can guarantee they all end up on the same collector and therefore no need for some shared decision store: https://github.com/janstenpickle/trace4cats/blob/master/docs/sampling.md#tail-sampling
    Ben Plommer
    @bplommer
    looks really cool!
    Ben Plommer
    @bplommer
    ok, another question 😄 we (the fs2-kafka / vulcan devs) are going to start putting out milestones for the next major versions sometime soon, which will break bincompat - do you think you could be persuaded to release trace4cats milestones based on them? It would make dogfooding a lot easier if we ($work) adopt t4c. Obviously I'd be happy to help.
    Chris Jansen
    @janstenpickle
    sorry for the delay @bplommer, I’ve been computer detoxing! Yes, of course. Technically we don’t have a formal release of t4c for CE3 yet anyway, so depending on your schedule we can possibly roll it into that
    Ben Plommer
    @bplommer
    Great, and no worries! We don’t really have a schedule but I’d think it will be several months before the final release of the next stable versions, so I’d suggest sticking with Vulcan 1.x / fs2-kafka 2.x for t4c 0.12.
    Chris Jansen
    @janstenpickle
    ok, will do
    Chris Jansen
    @janstenpickle
    I’m just planning splitting t4c into separate repos at the moment, so we can do milestones for the kafka libraries individually
    Ben Plommer
    @bplommer
    Ah, brilliant
    Paul Snively
    @paul-snively
    Hi everyone! Is there a cross-module tracing example somewhere? e.g. http4s streaming messages from Kafka, or an http4s POST going to Kafka? It looks like there's machinery to go from one "effect that contains a Span" (Trace) to another, but it's unclear how to actually use it, especially when there may be a Stream around the effect.
    Chris Jansen
    @janstenpickle

    hey @paul-snively it really depends on how you want to operate. FS2 and a “normal” effect behave very differently, whereby the “normal” effect has some context that requires a root span to execute (this uses a Kleisli by default in the inject module). The nature of FS2 and infinite streams makes this impossible because the runtime effect may never end, so we use a writer monad to pass around a parent span like you would pass context headers between applications.

    We don’t currently have any machinery to go between the two models, as it’s never really come up, generally you’re working in one or the other, but I can see the need.

    also, but http4s streaming messages from kafka, do you mean making http requests based on a kafka message?
    Paul Snively
    @paul-snively
    Hi @janstenpickle! First, thanks so much for the reply. Second, yes, let's say I'm pulling data from Kafka and constructing a Request from that data.
    But you've hit the nail on the head: the question is really a pretty general sort of "how do we compose the different modules in Trace4Cats so Spans are passed through end-to-end?" which on one hand almost sounds like Kleisli composition of the various Kleisli[F, Span[F], *s we have, but as you say, the Stream model is clearly different, and yes, I saw your WriterT there. :-)
    Chris Jansen
    @janstenpickle

    Hi @janstenpickle! First, thanks so much for the reply. Second, yes, let's say I'm pulling data from Kafka and constructing a Request from that data.

    Oh in that case, this method may be of some use: https://github.com/trace4cats/trace4cats/blob/master/modules/fs2/src/main/scala/io/janstenpickle/trace4cats/fs2/syntax/Fs2StreamSyntax.scala#L116-L117

    it uses the span that is in the writer and injects it to the contextual effect G, where G could be a Kleisli[F, Span[F], *]
    so A => G[B] could be some message from kafka to some http4s request dispatch logic
    Ben Plommer
    @bplommer
    Is something like this built into t4c? Or is it a bad idea?
      type Traced[F[_], A] = Kleisli[F, Span[F], A]
    
      def injectK[F[_]: BracketThrow](
          ep: EntryPoint[F],
          name: String,
          spanKind: SpanKind = SpanKind.Internal
      ): Traced[F, *] ~> F = new (Traced[F, *] ~> F) {
        def apply[A](fa: Traced[F, A]): F[A] = ep.root(name, spanKind).use(fa.run)
      }
    Chris Jansen
    @janstenpickle
    sorry @bplommer, yes this is something we provide
    Ben Plommer
    @bplommer
    Thanks, I’ll take a look
    Chris Jansen
    @janstenpickle
    we tend to combine it with a Kleisli, which represents the entrypoint