Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Jan 31 2019 19:53
    wafflebot[bot] unlabeled #1280
  • Jan 31 2019 19:53
    vprithvi closed #1280
  • Jan 31 2019 19:53

    vprithvi on master

    Implement FindTraceIDs function… (compare)

  • Jan 31 2019 19:05
    codecov[bot] commented #1280
  • Jan 31 2019 19:02
    codecov[bot] commented #1280
  • Jan 31 2019 18:51
    codecov[bot] commented #1280
  • Jan 31 2019 18:51
    vprithvi synchronize #1280
  • Jan 31 2019 16:18
    pavolloffay edited #1309
  • Jan 31 2019 14:58
    wafflebot[bot] labeled #1242
  • Jan 31 2019 14:58
    wafflebot[bot] assigned #1242
  • Jan 31 2019 14:58
    wafflebot[bot] labeled #1309
  • Jan 31 2019 14:58
    wafflebot[bot] assigned #1309
  • Jan 31 2019 14:58
    pavolloffay review_requested #1309
  • Jan 31 2019 14:58
    pavolloffay review_requested #1309
  • Jan 31 2019 14:57
    pavolloffay review_requested #1309
  • Jan 31 2019 14:57
    pavolloffay review_requested #1309
  • Jan 31 2019 14:57
    pavolloffay review_requested #1309
  • Jan 31 2019 14:57
    pavolloffay review_requested #1309
  • Jan 31 2019 14:57
    pavolloffay opened #1309
  • Jan 31 2019 08:21
    vlamug commented #1280
Guillaume Dupin
@yogeek
a problem with the CI that did not push the image for the last release maybe ?
Dotan Horovits
@horovits_twitter
I'm glad to share that we at Logz.io released a managed version of Jaeger 🎉
You get Jaeger backend, with Kafka and Elasticsearch, managed for you in multi-cloud multi-region setup.
This joins our existing open source observability stack of ELK for logs and Grafana, Prometheus for metrics. Go OSS!
We work upstream, contributing our enhancements back to the open source, for everyone to benefit. 
💪
And you can also use OpenTelemetry Collector to send your traces to our backend!
https://logz.io/platform/distributed-tracing/
https://docs.logz.io/user-guide/distributed-tracing/getting-started-tracing/
Daria Plotnikova
@DariaPlotnikova_gitlab

Hi. I'm trying to implement opentelemetry tracing with jaeger+elastichsearch from python app using jaegertracing/jaeger-opentelemetry-* latest docker images. I found that if python client do nothing with correct spans (without errors) it is throwed to jaeger-agent with status.code = UNSET, this is ok according opentelemetry span's status spec. But in jaeger UI such spans are marked with tag error=true (see pictire below). At the same time this span in elastic has error tag + status.code=UNSET which is strange, cause it had OK at the python's output:

"tags": [    // this is span representation in elastic.
   {
        "key": "status.code",
        "type": "string",
        "value": "STATUS_CODE_UNSET"
      },
      {
        "key": "error",
        "type": "bool",
        "value": "true"
    },

I tried to set status.code=OK at python output and then such span is displayed correctly in UI, but in elastic it has status.code=STATUS_CODE_UNSET without error tag. Is such status swapping expected behaviour? Setting up of error tag is related to this code I guess (error tag will no exist only if status is StatusCode(0)). My questions are:

  1. Does StatusCode(0) have value UNSET ? I'm absolutely newbie in golang
  2. Is that swapping of status codes UNSET <-> OK related to that code?
1 reply
image.png
Daria Plotnikova
@DariaPlotnikova_gitlab
The second question is resolved (it's a bug in python lib - it sends 0 for OK status and 1 for UNSET).
rajm
@raz08

Hello everyone,
I'm new to Jaeger tracing. I'm using elastic to store traces. Jaeger client--> agent --> collector --> elastic . So in this stack I see only one span index created on daily basis but I have requirement to have separate index for each service. i.e something like app1-jaeger-span- , app2-jaeger-span- or with cluster level i.e. something like cluster1-jaeger-span, cluster2-jaeger-span.

Any configuration which I need to tweak to achieve this?

searching the cli there is a --es-archive.index-prefix so you should be able to use this segregate by cluster
3 replies
@@@@
@narendra0356
Hi Team,
I want to deploy collector with kafka, which one is best process to deploy the collector and ingester. Can we deploy the collector and ingester in single pod or separately. Can you give suggestion, which one is best process.
Joe Elliott
@joe-elliott
i would recommend deploying the ingester and collector separately so they can be scaled independently
tmojlupo
@tmojlupo
Hello, we are running Jaeger in K8s using the jaegertracing/all-in-one:1.16 image with badger using an emptyDir volume. Is there a way to set the retention time for how long the data is kept? I think it might be the --badger.span-store-ttl flag, but I'm not sure how to send that to the container.
Ashmita
@Ashmita152

Hello, we are running Jaeger in K8s using the jaegertracing/all-in-one:1.16 image with badger using an emptyDir volume. Is there a way to set the retention time for how long the data is kept? I think it might be the --badger.span-store-ttl flag, but I'm not sure how to send that to the container.

It does support that flag:

$ docker run -e SPAN_STORAGE_TYPE=badger jaegertracing/all-in-one:1.16 --help | grep badger.span-store-ttl
      --badger.span-store-ttl duration                           How long to store the data. Format is time.Duration (https://golang.org/pkg/time/#Duration) (default 72h0m0s)
You just to extend args with this flag in your k8s manifest.
prashantnitt07
@prashantnitt07
Need binary file for Linux VM for "jaeger-opentelemetry-collector" .
please share the link for binary file "jaeger-opentelemetry-collector"
prashantnitt07
@prashantnitt07
jaeger-opentelemetry-collector
Joe Elliott
@joe-elliott
jaeger doesn't publish otel binaries. if you pull the repo there are instructions in CONTRIBUTING.MD on how to build the project
rajm
@raz08
hi all, how do I login to jaeger collector container in kubernetes? I tried "kubectl exec -it jaeger-collector-67fb7d95b7-nb7nl /bin/sh -n jaeger" and also tried with /bin/bash, sh , bash. Didn't help. any idea how can I login.
rajm
@raz08

I set cli command --es-archive.index-prefix but it is not reflecting.

In deployment YAML I set it to

      - name: ES_ARCHIVE_INDEX_PREFIX
        value: prod

but this is not getting reflected.

Juraci Paixão Kröhling
@jpkrohling
@raz08 which version of Jaeger are you using? we were using scratch images up to 1.20, but a shell should be available in 1.21: $ podman run --entrypoint /bin/sh -it jaegertracing/all-in-one:1.21.0
about the ES index prefix, could you please share a complete YAML? note that this is only for the archive, not for the current index
rajm
@raz08
I'm using Jaeger 1.15. I need index to be created in elastic like prod-jaeger or app1-jaeger. Please do let me know which version to use and configuration ?
rajm
@raz08
How can I add prefix to current index?
Juraci Paixão Kröhling
@jpkrohling
rajm
@raz08
I checked this, I don't see any configuration to add prefix to the current index.
Juraci Paixão Kröhling
@jpkrohling
--es.index-prefix
rajm
@raz08
ok thanks @jpkrohling .
Jonathan Wong
@joncwong
From a new adopter POV, what would the main difference be if I were to instrument my app with the official jaeger-client-python vs opentelemetry-python?
7 replies
tmojlupo
@tmojlupo
@Ashmita152 is there anything else I need to do to free up disk space? I just set the argument (which I didn't realize you could set an argument without a command) to 5m for --badger.span-store-ttl. Now, the UI is showing only traces within the past 5 minutes, but the disk space usage keeps increasing. I see an issue about this, but I'm not sure what the resolution is.
jaegertracing/jaeger#1916
apoorva88
@apoorva88
Hi,
I am getting this error while sending spans
"Failed to flush spans in reporter: error sending spans over HTTP: Error: socket hang up"
this is using Jaeger-node (jaeger-client”: “^3.18.1") client in a Lambda function. Please let me know how can I fix this?
Joe Elliott
@joe-elliott
socket hang up is weird. i would double check the port i'm sending to
apoorva88
@apoorva88
Same host/port is working for all the other services though. Have you seen this issue before?
Joe Elliott
@joe-elliott
not this in particular. are all services nodejs? are they all in the same place as far as networking is concerned?
Ashmita
@Ashmita152
@tmojlupo I think you are running jaeger v1.16.0, from that ticket you mentioned its says its solved in latest v1.21.0. I would suggest to upgrade your jaeger to v1.21.0 and see if it resolves the issue, if not reopen the ticket with your findings.
Nigel Brown
@BigNigelBrown_twitter

I'm trying to use jaeger to trace messages through a kafka based batched system. I want to see the latency. The messages end up in different batches. I wondered what the best sampling system would be.

I can't use http headers because there are no http requests and there seems to be no way to propagate the tracing status per message to all the different message consumers.

What I have done is to make a number out of the message _ids (this could equally be a hash of the whole message if there were no _id field). I take a mod 1000 (if I want to trace every 1000 messages) and if this is 0 I trace the message. All the consumers know which messages to span and the trace is complete.

The question is is this a sane/useful approach and is there a better (built in) way to do this?

I would think other people have had to do something similar.

Joe Elliott
@joe-elliott
so i've never traced through a kafka queue, but i found this awhile back while thinking of tracing through queues: https://strimzi.io/blog/2019/10/08/strimzi-apache-kafka-and-tracing/
31 replies
Karim El Tel
@KarimElTel_twitter
hello, I want to make a query to the Cassandra traces table that shows each span and its parent process. Is there a way I can do a single query and get the result of the below pyton code?
traces = session.execute('SELECT trace_id,span_id,refs FROM traces')
for t in traces:
        if t['refs'] is not None:
            parentSpan=str(t['refs'][0].span_id)
            parentTrace=t['refs'][0].trace_id
            hexstr = ''.join('{:02x}'.format(x) for x in parentTrace)
            query="SELECT process FROM traces where trace_id=0x"+hexstr+ " AND span_id="+parentSpan
            result= session.execute(query)
ryo4600
@ryo4600
Can the agent send traces over authentication proxy? I have set up the collector on AWS and try to send traces from my corporate network.
Juraci Paixão Kröhling
@jpkrohling
@ryo4600 you'd need a gRPC-capable auth proxy, and I'm not aware of any at the moment. You can setup mutual TLS, though: the agent would be configured with client certs, and the server with, well, server certs :)
this is still not ideal, as the connection is long-lived -- revoked certs would only be refused when the connection is established, not for already existing connections, with the effect that agents that aren't authorized anymore are still allowed to send spans to the collector until the connection is broken
@ryo4600 if that's not sufficient for your use case, you can replace your jaeger-agent with otel-collector, and place another otel-collector in front of your jaeger collector. Something like this: https://medium.com/opentelemetry/securing-your-opentelemetry-collector-1a4f9fa5bd6f
Martijn Pieters
@mjpieters
I'm probably not going to find anything and it is not as critical, but is there a way of submitting spans to Jaeger from a shell process? A subsection of a legacy app that I'm instrumenting spawns a bash script to run further tasks, and I'd love to be able to get timings for those tasks out.
I may just have to echo time results, then create spans in the parent process after the fact.
Joe Elliott
@joe-elliott
i don't know of a good way to push thrift spans via bash, but the collector can expose a zipkin compatible endpoint on 9411
this can easily accept http/json spans
a quick snippet to curl a span into zipkin: https://github.com/grafana/tempo/issues/213#issuecomment-728889502
Martijn Pieters
@mjpieters
@joe-elliott : that is an interesting option, thanks!
Joe Elliott
@joe-elliott
np, gl!