Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
Repo info
  • 13:10
    jcchavezs assigned #1323
  • Jan 21 21:35
    Luubosi labeled #3416
  • Jan 21 21:35
    Luubosi opened #3416
  • Jan 21 10:33
    jcchavezs commented #1323
  • Jan 21 09:43
    xeraa commented #196
  • Jan 21 08:50
    jcchavezs commented #196
  • Jan 21 06:35
    ax-korneenkov labeled #196
  • Jan 21 06:35
    ax-korneenkov opened #196
  • Jan 21 05:56
    wuwen5 synchronize #1323
  • Jan 21 03:32
    shakuzen commented #1323
  • Jan 20 11:14
    wuwen5 commented #1323
  • Jan 20 10:18
    jcchavezs synchronize #205
  • Jan 20 10:18
    jcchavezs synchronize #205
  • Jan 20 06:45
    jcchavezs commented #205
  • Jan 20 02:56
    shakuzen commented #205
  • Jan 19 06:27
    jcchavezs commented #205
  • Jan 18 21:29
    evantorrie ready_for_review #205
  • Jan 18 21:29
    evantorrie edited #205
  • Jan 18 17:34
    evantorrie opened #205
  • Jan 16 22:46
    jcchavezs review_requested #1323
Adrian Cole
so either the ingress-nginx isn't sending b3 headers, or whatever your c# api is isn't reading them
based on your description of the problem anyway (start a new trace problem)
so it is important to know what c# thing you are using, which headers nginx are sending and if that thing supports reading those or not
ex there are two forms of B3 headers (multi and single header) https://github.com/openzipkin/b3-propagation
to triage you can start with this.. if the zipkin-gcp thing is receiving spans, that's good. it cannot affect propagation though
that will be the seam between your apps in this case nginx and the next thing
@nicmunroe has a nice piece on the general art of troubleshooting which applies regardless of library type https://medium.com/nikeengineering/hit-the-ground-running-with-distributed-tracing-core-concepts-ff5ad47c7058
and to answer directly, yes it is the intent to connect all spans in one http request together with the same trace ID
this is universally the case with http microservices
for our projects we usually use known working examples.. library choice matters a lot, and without knowing the library we don't know if it works.
adriancole @adriancole has to go for a bit
Yehuda Makarov
thank you for all the info. I have the answers i need. the request after it goes through the nginx ingress has b-3 headers. Maybe i will trouble shoot more within the zipkin library in c#. (which is https://github.com/openzipkin/zipkin4net)
Adrian Cole
yeah that would be odd indeed if the latter didn't work with b3 propagation. you can have a look at the examples directory there.. the project is lightly maintained, but it is used (notably by MdSol)
Adrian Cole
@yehudamakarov unsolicited 2p is to run the examples behind your nginx then compare (assuming they continue the trace)
Yehuda Makarov
@adriancole i will let you know how this goes. thanks for all the advice
Adrian Cole
Josh Young

continuing the docker/zipkin/eureka saga I switched to the other Dockerfile strategy:

# We copy files from the context into a scratch container first to avoid a problem where docker and
# docker-compose don't share layer hashes https://github.com/docker/compose/issues/883 normally.
# COPY --from= works around the issue.
FROM scratch as scratch

COPY . /code/

FROM openzipkin/zipkin-builder as built

COPY --from=scratch /code /code


RUN mvn -B --no-transfer-progress package -DskipTests=true

WORKDIR /zipkin-eureka

RUN cp /code/target/eureka-zipkin-client-*-module.jar eureka-zipkin-client-module.jar && \
  (mkdir eureka-zipkin-client-module && cd eureka-zipkin-client-module && jar -xf ../eureka-zipkin-client-module.jar) && \
  rm eureka-zipkin-client-module.jar

RUN cp /code/eureka-client.properties eureka-client.properties

FROM openzipkin/zipkin:master
MAINTAINER Zipkin "https://zipkin.io/"

COPY --from=built /zipkin-eureka/ /zipkin/

ENV MODULE_OPTS="-Dloader.path=eureka-zipkin-client-module -Dlogging.level.*=DEBUG -Dlogging.level.org=DEBUG -Dlogging.level.com=DEBUG -Dlogging.level.io=DEBUG -Dspring.application.name=zipkin"

As was suggested by @adriancole further up in the channel (Thank you!) and with that I upped the logger to DEBUG. That gave this helpful info on what spring was loading:

Positive matches:
Negative matches:
      Did not match:
         - @ConditionalOnClass did not find required class 'org.springframework.cloud.config.client.ConfigServicePropertySourceLocator' (OnClassCondition)

I tried also adding spring-cloud-starter-config-client, but that gave issues with missing dependencies. Unfortunately I'm new to spring boot (just started playing with all this yesterday) so I'm not sure where to go from here.

Currently I have eureka setup for service discover in spring boot mostly because that was the first tutorial I was looking at... would Consul be easier to setup with the zipkin docker image? Up until this Eureka has been easy to setup for the other services I was working with, but I haven't tried consul.

Adrian Cole
something that doesn't rely on java types to perform service registration and lookup is easier indeed
what you probably have not tuned right is the jar added to the loader path. it likely is missing an inner thing including the cloud-config library
the "module" jar should have packed that up in other words
that all said, yeah anything that doesn't require modifications to binaries is easier
Thierry Peng
Hi. Is there a way to secure the collector endpoint out-of-the-box in zipkin (e.g. basic auth, oauth ..)?
Adrian Cole
@thpeng if you are using kafka, sure it has some natively supported stuff, as does rabbitmq I think. http we didn't choose to bake in something as there are major issues with basic auth, credential rotation etc we didn't want to step into. Instead just use a proxy that has the features you need
you can call it mesh :)
Thierry Peng
@adriancole thanks. We're stuck with the rest endpoint because of private PaaS setup reasons. I thought about moving zipkin behind the spring-cloud-gateway, securing all non-GET endpoints (allowing the usage of the ui freely). As far as I have seen, the zipkin ui does not use other methods (upload excluded). But then we have also to modify the span reporter in sleuth (using a customized rest template).
Adrian Cole
good thing is that the ui and everything it needs is under /zipkin and only GET is needed
the upload is actually in-browser.. doesn't get to the server
so the POST thing is indeed collector-only (also under /api/v2/spans, not /zipkin) and that narrows the thing to pin
Thierry Peng
ok, thanks for the url hint. I will investigate further
Adrian Cole
and also because we chose to not proliferate BASIC, we also chose to leave open how to modify requests vs bake in basic which isn't actually supported on the backend by default
ex some SaaS offerings run zipkin clones and have different auth mecahnisms
good luck!
Jiten Vij
I have configured brave for trace id and span id in spring web-mvc project. It is showing the server call but it is not showing the call which I am making from spring mvc project to spring boot project because of which trace id is different in spring MVC and spring boot. Please advise.
There should be two span for one trace Id.. one will refer to the server call and another to the client call
chris vale
If there’s anyone else here using zipkin-finagle I’ve got a PR open #78 to update Scala to 2.13. My maven foo isn’t strong enough to cross build between scala 2.12 and 2.13 so if you know how I’d love to hear about it. Otherwise I suppose I will have to publish both with differing (?) tags.
:wave: I'm trying to set up Zipkin and am looking at storage options
which suggests my options are, Elasticsearch, X-Ray, Stackdriver, Logz.io, Scouter APM, Kafka, or Haystack.
But the default storage backend is MySQL, so I feel like I'm missing something. Architecture documentation suggests I should be able to use Cassandra as well: https://zipkin.io/pages/architecture.html#storage
So...is there somewhere I could find all the storage options and how to configure them?
Bas van Beek
https://github.com/openzipkin/zipkin#storage-component for default supported storage options.
chris vale
@here I’m going to release zipkin-finagle 2.1.5 for scala 2.12 unless someone objects
chris vale
Zipkin-finagle 2.1.5 is winging it’s way to maven central
Thanks @basvanbeek! It's too bad that googling "zipkin storage backends" gives my first link instead of yours
@adriancole .. m new to DistributedTracing. Able to use Brave and setup tracing between my services. Is there a way I can pass a custom header from frontend, which gets recorded in my spans as a tag without manually setting it in all services?
I'm able to propagate the header all the way to the backend service by .propagationFactory(ExtraFieldPropagation.newFactory(B3Propagation.FACTORY, "order-id")) in the config class. But was wondering how can I get this header as a tag in all my spans so that I can search by that in the UI
@adriancole when am trying to build custom server by using 2.12.9 its giving error Factory method 'armeriaServer' threw exception; nested exception is java.lang.NullPointerException. can you fix that issue or guide me to what to do? am in a situation where i cant use readymade server.
@adriancole i found this https://github.com/openzipkin/zipkin/issues/2796#issuecomment-574810609. if i follow ui main page is not coming up getting 404
Adrian Cole
@javabotnetflix maybe you can mention what you need to do and then we can try to see if there's an alternative to a custom built server? we are pretty clear on no custom servers as they are expensive to support and fragile. plus we change libraries in the server whenever we need to
Omid Pourhadi


I am using zipkin-autoconfigure-storage-elasticsearch-http 2.8.4 with zipkin-server 2.10.0 I can not connect to remote elasticsearch server.

I set System.setProperty("ES_HOSTS", "http://IP:9200"); in my spring boot app.

The exception is :

 Failed to connect to localhost/0:0:0:0:0:0:0:1:9200
    at zipkin2.internal.Platform$Jre8.uncheckedIOException(Platform.java:71) ~[zipkin-2.11.7.jar:na]
    at zipkin2.elasticsearch.ElasticsearchStorage.ensureIndexTemplates(ElasticsearchStorage.java:306)

in ElasticsearchStorage.java Line 67 : host is hardcoded like this


how can I change that ?