openzipkin/zipkin

Zipkin is a distributed tracing system

Adrian Cole

@adriancole

no duration can be accidental, but no timestamp is a bug

server1 being a parent of server0 that sounds odd..

can you clean PII from the trace? and/or send to me

Chris K Wensel

@cwensel

client0 is the parent of everyone

(except itself)

Adrian Cole

@adriancole

root span?

or local root in that process

Chris K Wensel

@cwensel

there is, this is just a sub-graph of the trace

not sure about local root.. i’m just looking at raw data, i dont have the code available to me

(raw data before it gets to zipkin server)

Adrian Cole

@adriancole

I mean client0 shows parentID?

Chris K Wensel

@cwensel

to summarize, i have a sub-graph of 4 spans.. client0,client1,server0,server1. client0 is the parent to the others.. client0 has no timestamp. and client1,server1,server0 have timestamps in that order

Adrian Cole

@adriancole

can you turn client0,client1,server0,server1 into (span,localEndpoint) tuples for me?

trying to understand the remote relationships if any

so far I think it looks like a retry

though I can't explain client1, we could ignore it and then it looks like a retry

instrumentation should create a new kind=client (with a new span ID) for each try

but depending on how written, or limits of the framework, you can end up in this situation. in the UI and also the dep linker, we consider the localEndpoint to differentiate when the span IDs are the same

shengjiz

@shengjiz

I read the source code and found that microseconds are calculated by the local relative time(A trace on each node has a relative time), but the relative time of nodes is different, so whether the microseconds calculated at different nodes are comparable.

Adrian Cole

@adriancole

@shengjiz no they aren't strictly comparable. hierarchy can help determine which happened before the other in absolute time. in any case, keeping server times close is helpful

Daniel Mendes

@webmutation

Hi, is there any up to date Zipkin Helm chart? I could only find the one from Financial Times, but that one seems outdated

I saw Jaeger helm in incubator and was surprised to not see one for zipkin

Adrian Cole

@adriancole

@webmutation dare you to make a pull request to the incubator with a zipkin one :D

Daniel Mendes

@webmutation

if there is not one more up to date than the financial times one, I will look at their and see if there are many changes needed or not.

Javi

@Javigomez_gitlab

@adriancole

if a span was unfinished, there would be no duration for it

[
    [
        {
            "traceId": "0823811ca25bdad3",
            "parentId": "0823811ca25bdad3",
            "id": "6439eb5a54939542",
            "kind": "CLIENT",
            "name": "get",
            "timestamp": 1579883159955171,
            "duration": 809817,
            "localEndpoint": {
                "serviceName": "myservice"
            },
            "tags": {
                "http.path": "/user",
                "http.status_code": "200"
            }
        },
        {
            "traceId": "0823811ca25bdad3",
            "parentId": "0823811ca25bdad3",
            "id": "653ef3386c7c8139",
            "kind": "CLIENT",
            "name": "get",
            "timestamp": 1579883159955740,
            "duration": 817655,
            "localEndpoint": {
                "serviceName": "myservice"
            },
            "tags": {
                "http.path": "/u/settings",
                "http.status_code": "200"
            }
        },
        {
            "traceId": "0823811ca25bdad3",
            "parentId": "0823811ca25bdad3",
            "id": "58cdf6b54f3e50ae",
            "kind": "CLIENT",
            "name": "get",
            "timestamp": 1579883159965163,
            "duration": 819692,
            "localEndpoint": {
                "serviceName": "myservice"
            },
            "tags": {
                "http.path": "/u/accounts",
                "http.status_code": "200"
            }
        },
        {
            "traceId": "0823811ca25bdad3",
            "parentId": "0823811ca25bdad3",
            "id": "427b48ddce3ccacd",
            "kind": "CLIENT",
            "name": "get",
            "timestamp": 1579883159960778,
            "duration": 900222,
            "localEndpoint": {
                "serviceName": "myservice"
            },
            "tags": {
                "http.path": "/u/feature-flags",
                "http.status_code": "200"
            }
        },
        {
            "traceId": "0823811ca25bdad3",
            "parentId": "0823811ca25bdad3",
            "id": "147230185f20c741",
            "kind": "CLIENT",
            "name": "get",
            "timestamp": 1579883159953872,
            "duration": 1239779,
            "localEndpoint": {
                "serviceName": "myservice"
            },
            "tags": {
                "http.path": "/workers/mine",
                "http.status_code": "200"
            }
        },
        {
            "traceId": "0823811ca25bdad3",
            "parentId": "0823811ca25bdad3",
            "id": "07d4dbd088e6633c",
            "kind": "CLIENT",
            "name": "get",
            "timestamp": 1579883160880901,
            "duration": 552661,
            "localEndpoint": {
                "serviceName": "myservice"
            },
            "tags": {
                "http.path": "/workers/01D/feature",
                "http.status_code": "200"
            }
        },
        {
            "traceId": "0823811ca25bdad3",
            "parentId": "0823811ca25bdad3",
            "id": "b148e6b471cd2dfa",
            "kind": "CLIENT",
            "name": "get",
            "timestamp": 1579883160880282,
            "duration": 560505,
            "localEndpoint": {
                "serviceName": "myservice"
            },
            "tags": {
                "http.path": "/current-features",
                "http.status_code": "200"
            }
        },
        {
            "traceId": "0823811ca25bdad3",
            "parentId": "0823811ca25bdad3",
            "id": "bfc4a0a54601da77",
            "kind": "CLIENT",
            "name": "get",
            "timestamp": 1579883160886515,
            "duration": 562673,
            "localEndpoint": {
                "serviceName": "myservice"
            },
            "tags": {
                "http.path": "/workers/01D/feature",
                "http.status_code": "200"
            }
        },
        {
            "traceId": "0823811ca25bdad3",
            "

we can confirm that no spans are missing duration

Javi

@Javigomez_gitlab

How could we validate and prove that the problem is not in unfinished spans but that somehow we have "leaked context" problem?

Adrian Cole

@adriancole

@Javigomez_gitlab you can hack the recorder so that it not only reports spans but also logs them. in this case you could see if something seems fishy for example things ignoring their actual parent etc https://github.com/openzipkin/zipkin-js-example/blob/master/web/recorder.js#L24

Javi

@Javigomez_gitlab

thanks

Anders Clausen

@AndersClausen

@AndersClausen definitely sounds like a job for a proxy

Thanks @adriancole , I'll tell our platform team that :-D

javabotnetflix

@javabotnetflix

@AndersClausen which version of springboot you are using for your custom zipkin server? is that working for you?

Anders Clausen

@AndersClausen

@AndersClausen which version of springboot you are using for your custom zipkin server? is that working for you?

No it wasn't working for us, so we went ahead and used the 'ready-made' Zipkin server, which we are about to test very soon. Just setting up the CI/CD pipeline for it, so we can manage our Cassandra/Elasticsearch settings with it

Adrian Cole

@adriancole

@javabotnetflix are you a bot?

also do you accept movie requests?

I appreciate we have all the lethal weapon movies now, but...

José Carlos Chávez

@jcchavezs

@javabotnetflix could you please enable Latinamerican spanish in Europe (not only European Spanish)? There are los of latinoamerican in Europe.

adriancole @adriancole chuckles

Josh Young

@mandragorn

I have a question around getting a zipkin docker container to register with eureka.
I've been trying to adapt https://github.com/openzipkin/zipkin/issues/1870#issuecomment-470068636 to a mvn bulid and Dockerfile that extends openzipkin/zipkin.
I have the mvn build working from that comment, but getting the zipkin inside the docker container to register with eureka isn't working, it isn't even trying.
This is my dockerfile:

FROM openzipkin/zipkin

ENV VERSION 0.0.1-SNAPSHOT
ENV ZIPKIN_SERVICE_NAME=zipkin

ENV EUREKA_CLIENT_JAR "eureka-zipkin-client-${VERSION}-module.jar"
ENV EUREKA_CLIENT_JAR_CONTAINER_PATH "${EUREKA_CLIENT_JAR}"
ENV EUREKA_CLIENT_PROPERTIES_FILE "eureka-client.properties"
ENV EUREKA_CLIENT_PROPERTIES_CONTAINER_PATH "${EUREKA_CLIENT_PROPERTIES_FILE}"

ENV MODULE_OPTS="-Dloader.path=${EUREKA_CLIENT_JAR_CONTAINER_PATH},${EUREKA_CLIENT_JAR_CONTAINER_PATH}!/lib -Dspring.application.name=${ZIPKIN_SERVICE_NAME}"

COPY "target/${EUREKA_CLIENT_JAR}" "${EUREKA_CLIENT_JAR_CONTAINER_PATH}"
COPY "${EUREKA_CLIENT_PROPERTIES_FILE}" "${EUREKA_CLIENT_PROPERTIES_CONTAINER_PATH}"

I added the repackaged client jar to the root of the docker container and I can confirm that it is there along with the eureka-client.properties file.
The zipkin server starts up fine, but I see no logs indicating it is trying to register with eureka.

Adrian Cole

@adriancole

@mandragorn hmm what you are doing is similar to how stackdriver is integrated.. maybe try to copy that approach instead? this might get you there quicker (in docker I mean)

https://github.com/openzipkin/zipkin-gcp/tree/master/docker

Adrian Cole

@adriancole

read all about it.. https://github.com/openzipkin/brave/releases/tag/5.9.2

Yehuda Makarov

@yehudamakarov

Hey all, just REALLY looking so hard for info on this:

I have ingress-nginx ingress on GKE. I configured the ingress with a Zipkin tracer that sends to Stackdriver. with enable-open-tracing. Then i also installed the google stackdriver tracing library in my c# api to send spans to stackdriver. But it looks like in the api it is very difficult to figure out how to CONTINUE the span FROM the trace started in the Ingress. So in Stackdriver I always start a new trace when the controller in my api is hit. as opposed to continuing from the ingress.

i want it to be ONE trace, with a span for the ingress, and a span for the api

Adrian Cole

@adriancole

I've no idea the relationship here. what has the "enable-open-tracing" option? I don't think any of our libraries do

Yehuda Makarov

@yehudamakarov

i'll upload more precise snippets. unless you're ok with a diagram

Adrian Cole

@adriancole

I think it is best to validate that this is a relevant support channel for you

I've so far not seen anything related to the code we maintain in your description, except for the word zipkin

this channel is volunteer run, so it is best to get you where you need to go. understand troubleshooting things can be frustrating, but not sure how much we can help on something we have no responsbility over either

so first what is "Zipkin tracer that sends to Stackdriver" this will start the triage

many things call themselves zipkin, and when you said that option seems like it isn't likely something we wrote

Where communities thrive

People

Repo info

Activity