Tried with adding zipkin security jar but getting
Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanDefinitionStoreException: Failed to parse configuration class [zipkin.module.security.ZipkinSecurityModule]; nested exception is java.lang.IllegalStateException: Failed to introspect annotated methods on class zipkin.module.security.ZipkinSecurityModule
27 replies
Hi all,
I got Unable to establish connection to RabbitMQ server: Connection refused when trying to run zipkin-server with rabbitmq and elasticsearch, using this docker-compose file :
version: '3.7'
services:
elasticsearch:
image: elasticsearch:7.10.1
container_name: elasticsearch
restart: on-failure
ports:
- 9200:9200
environment:
#not recommended in production
- "ES_JAVA_OPTS=-Xms750m -Xmx750m"
- discovery.type=single-node
rabbitmq:
image: rabbitmq:3.8-management-alpine
container_name: rabbitmq
restart: on-failure
ports:
- 5672:5672
- 15672:15672
environment:
RABBITMQ_DEFAULT_VHOST: ${rabbitmq_vhost}
RABBITMQ_DEFAULT_USER: ${rabbitmq_user}
RABBITMQ_DEFAULT_PASS: ${rabbitmq_pass}
RABBITMQ_VM_MEMORY_HIGH_WATERMARK: 1024MiB
zipkin-server:
image: openzipkin/zipkin:2
container_name: zipkin-server
depends_on:
- rabbitmq
- elasticsearch
restart: on-failure
ports:
- 9411:9411
environment:
RABBIT_CONCURRENCY: 1
RABBIT_CONNECTION_TIMEOUT: 60000
RABBIT_QUEUE: zipkin
RABBIT_ADDRESSES: ${rabbitmq_host}
RABBIT_PASSWORD: ${rabbitmq_pass}
RABBIT_USER: ${rabbitmq_user}
RABBIT_VIRTUAL_HOST: ${rabbitmq_vhost}
RABBIT_USE_SSL: "false"
STORAGE_TYPE: elasticsearch
ES_HOSTS: ${elasticsearch_host}
healthcheck:
test: wget --no-verbose --tries=1 --spider http://localhost:9411/health || exit 1
interval: 10s
start_period: 15s
retries: 3
zipkin-dependencies:
image: openzipkin/zipkin-dependencies
container_name: zipkin-dependencies
depends_on:
- elasticsearch
restart: on-failure
environment:
STORAGE_TYPE: elasticsearch
ES_HOSTS: ${elasticsearch_host}
volumes:
default:
external:
name: zipkin-monitoring
networks:
default:
external:
name: zipkin-monitoringknowing that rabbitmq and elastic starts normaly
2 replies
remoteEndpoint. Zipkin's documentation says it's an RPC (or messaging) span, indicating the other side of the connection, but I'm not sure of what this means. There is an ipv4 field under remoteEndpoint. Can it be an IP address of an external user?
2 replies
hello guys, another thing ... I was able to manage to run the zipkin and was able to store traces successfully in Cassandra.. but now it doesn't show any dependencies .. traces are all good, accurate and "realtime" but when I click on dependencies it's empty.. .I read I have to set the zipkin-dependencies.jar to be run periodically so I did (based on this article https://stackoverflow.com/questions/37459261/cant-view-any-dependencies-inside-zipkin-ui-dependencies-tab ) but hell - it doesn't show anything still.. I even tried to run that as a service .. but no luck either ...
my zipkin service:[Unit]Description=Managa Java serviceDocumentation=https://zipkin.io/
[Service]WorkingDirectory=/opt/zipkinEnvironment="STORAGE_TYPE=cassandra3"ExecStart=/usr/bin/java -jar zipkin.jarUser=zipkinGroup=zipkinType=simpleRestart=on-failureRestartSec=10
[Install]WantedBy=multi-user.target
my zipkin-dependency.sh (crontab - every 5 mins)STORAGE_TYPE=cassandra3 /usr/bin/java -jar /opt/zipkin/zipkin-dependencies.jar
crontab -l*/5 * * * * cd /opt/zipkin && ./zipkin-dependencies.sh
any thoughts?
4 replies
5 replies
hello guys, I have a question .... I have finally managed to make Zipkin + Cassandra work together... however, to properly display dependencies I need to periodically run zipkin-dependencies.jar (as mentioned above) ... I do this via cron in linux ... the problem is -> it takes a little over 1hr to calc the data...
Is there any way to limit the spans to automatically check for those nonprocessed only? or any other way I can tune cassandra / zipkin-dependencies.jar to make it work faster? Any tips?
I am not sure if I should focus on the cassandra tuning or the problem is somewhere else ... I tried to run twice manually the zipkin-dependencies and it says:Running Dependencies job for 2021-03-01: 1614556800000000 =< Span.timestamp 1614643199999999 for both instances ... so it obivously recalc the same data ...
2 replies
Hi,
I'm running zipkin in a container and sending a get (wrapFetch/node-fetch) trace to it (zipkin-transport-http):
const { Tracer } = require('zipkin');
const { BatchRecorder } = require('zipkin'); // format data spans
const { HttpLogger } = require('zipkin-transport-http');
const fetch = require('node-fetch');
const wrapFetch = require('zipkin-instrumentation-fetch');
const CLSContext = require('zipkin-context-cls'); // continous local storage
const ctxImpl = new CLSContext();
const localServiceName = 'xxx';
const recorder = new BatchRecorder({
logger: new HttpLogger({
endpoint: http://localhost:9411/api/v1/spans
})
});
const tracer = new Tracer({ ctxImpl, recorder, localServiceName });
const remoteServiceName = 'youtube';
const zipkinFetch = wrapFetch(fetch, { tracer, remoteServiceName });
zipkinFetch('https://www.youtube.com/').then(res => res)
How do I go about instrumenting and sending my own span?
Thanks,
3 replies
1 reply
4 replies
Seems like none of the folding or annotation stuff is available? I'm trying opentelemetry from node, there doesn't seem to be a way to override the serviceName for the http or the db? The remoteService? Is there some way to tag spans so they are remote? Change their kind? Really don't know where to look.
3 replies
4 replies
Hi all, running into a small issue connecting Zipkin to AWS Elasticsearch. Initially I was getting EmptyEndpointGroupException then I forced the health check off and realized that Zipkin couldn't find the AWS credentials so it's throwing "User: anonymous is not authorized to perform: es:ESHttpGet". This is what I'm using to execute:
STORAGE_TYPE=elasticsearch \
ES_HOSTS=https://<domain>.us-east-1.es.amazonaws.com/ \
ES_HEALTH_CHECK_ENABLED=false \
java -Dloader.path='aws.jar,aws.jar!/lib' -Dspring.profiles.active=aws \
-cp zipkin.jar org.springframework.boot.loader.PropertiesLauncher
I've even tried to forcing the environment variables but still same issue:
STORAGE_TYPE=elasticsearch \
ES_HOSTS=https://<domain>.us-east-1.es.amazonaws.com/ \
ES_HEALTH_CHECK_ENABLED=false \
AWS_ACCESS_KEY=<access> \
AWS_SECRET_KEY=<secret> \
java -Dloader.path='aws.jar,aws.jar!/lib' -Dspring.profiles.active=aws \
-cp zipkin.jar org.springframework.boot.loader.PropertiesLauncher
5 replies
1 reply
Hello folks, we are seeing some fatal error: concurrent map read and map write errors in our main application process and further debugging shows some issue with Zipkin library: github.com/openzipkin/zipkin-go
goroutine 182 [running]:
runtime.throw(0xe75af5, 0x21)
/usr/local/go/src/runtime/panic.go:774 +0x72 fp=0xc1c015f420 sp=0xc1c015f3f0 pc=0x4313f2
runtime.mapaccess2(0xcf7160, 0xc262f27e30, 0xc19f6391f0, 0xc19f6391f0, 0xc17eac231b)
/usr/local/go/src/runtime/map.go:470 +0x278 fp=0xc1c015f468 sp=0xc1c015f420 pc=0x40fc58
reflect.mapaccess(0xcf7160, 0xc262f27e30, 0xc19f6391f0, 0xe6a8ed)
/usr/local/go/src/runtime/map.go:1319 +0x3f fp=0xc1c015f4a0 sp=0xc1c015f468 pc=0x411f6f
reflect.Value.MapIndex(0xcf7160, 0xc1300544d0, 0x195, 0xcbb920, 0xc19f6391f0, 0x98, 0xcbb920, 0xc19f639250, 0x98)
/usr/local/go/src/reflect/value.go:1176 +0x16d fp=0xc1c015f518 sp=0xc1c015f4a0 pc=0x49771d
. . .
github.com/openzipkin/zipkin-go/reporter.JSONSerializer.Serialize(...)
/src/go/pkg/mod/github.com/openzipkin/zipkin-go@v0.2.2/reporter/serializer.go:36
github.com/openzipkin/zipkin-go/reporter.(*JSONSerializer).Serialize(0x164b018, 0xc1951c4480, 0x64, 0x90, 0x0, 0xc0e90ac380, 0x433160, 0xea6f90, 0xc000056a00)
<autogenerated>:1 +0x72 fp=0xc1c015fe78 sp=0xc1c015fe30 pc=0xb6f892
github.com/openzipkin/zipkin-go/reporter/http.(*httpReporter).sendBatch(0xc0000293b0, 0xc1c015ffaf, 0x1)
/src/go/pkg/mod/github.com/openzipkin/zipkin-go@v0.2.2/reporter/http/http.go:137 +0xb2 fp=0xc1c015ff90 sp=0xc1c015fe78 pc=0xb8a882
github.com/openzipkin/zipkin-go/reporter/http.(*httpReporter).sendLoop(0xc0000293b0)
/src/go/pkg/mod/github.com/openzipkin/zipkin-go@v0.2.2/reporter/http/http.go:99 +0x3f fp=0xc1c015ffd8 sp=0xc1c015ff90 pc=0xb8a49f
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0xc1c015ffe0 sp=0xc1c015ffd8 pc=0x4615c1
created by github.com/openzipkin/zipkin-go/reporter/http.NewReporter
/src/go/pkg/mod/github.com/openzipkin/zipkin-go@v0.2.2/reporter/http/http.go:254 +0x2dbThis is interesting as it is an error while serializing a Span for the Zipkin client. Upon closer inspection, there is a Tag map in the spans that seems a candidate for the concurrency issue. If the Span is being serialized and a new Tag or something is being added to it concurrently, we may run into an issue.
Does any of you have seen this issue or are aware of it? Any help would be appreciated.
Regards.
3 replies
Hello!
I wanted to open a possible bug issue but came to ask here first.
I am using Spring Boot, Sleuth and Brave. There is a weird race condition that pops up and we are stumped.
The issue investigation is described here spring-cloud/spring-cloud-sleuth#1814 where Brave throws an exception "Parent can only be null in a local root". In the last message (https://github.com/spring-cloud/spring-cloud-sleuth/issues/1814#issuecomment-767389228) is the "stumped" part. In debug another subsequent call to the same method works fine.
Any help would be much appreciated.
6 replies
Is it possible to use custom id as a trace id in zipkin ?
hello guys, I just have a question ... I have zipkin running together with cassandra as its persistent storage .. the issue I have is that with the external storage I have to recalc dependencies periodically .. it takes a bit over an hour to get that done ....
Is there the same issue with ElasticSearch ??? or is it faster? or do you have any tips how to speed it up??
I’m not sure this is the right place to talk about this. So, Looking forward to your reply. Thanks a lot!
1 reply
[{"traceId":"607e3e086301ffaf74e217d5983efbc5","parentId":"74e217d5983efbc5","id":"91d3f96d11d7cb8e","kind":"SERVER","name":"get /api/msgcenter/msg/lastest","timestamp":1618886163861028,"duration":3775,"localEndpoint":{"serviceName":"base-msg-center","ipv4":"10.20.5.160"},"remoteEndpoint":{"ipv4":"123.157.238.102"},"tags":{"http.method":"GET","http.path":"/api/msgcenter/msg/lastest","mvc.controller.class":"ClientAPIController","mvc.controller.method":"getLastestMsgRecord"},"shared":true},{"traceId":"607e3e086301ffaf74e217d5983efbc5","parentId":"74e217d5983efbc5","id":"91d3f96d11d7cb8e","kind":"CLIENT","name":"get","timestamp":1618886152337857,"duration":11526274,"localEndpoint":{"serviceName":"lingyue-gateway","ipv4":"10.20.6.52"},"tags":{"http.method":"GET","http.path":"/api/msgcenter/msg/lastest"}},{"traceId":"607e3e086301ffaf74e217d5983efbc5","parentId":"74e217d5983efbc5","id":"864bdbd5a06a0515","kind":"SERVER","name":"get /auth/check","timestamp":1618886152333018,"duration":5224,"localEndpoint":{"serviceName":"dmo-user","ipv4":"10.20.7.38"},"remoteEndpoint":{"ipv4":"10.20.6.52","port":33926},"tags":{"http.method":"GET","http.path":"/auth/check","mvc.controller.class":"AuthController","mvc.controller.method":"checkToken"},"shared":true},{"traceId":"607e3e086301ffaf74e217d5983efbc5","id":"74e217d5983efbc5","localEndpoint":{"serviceName":"lingyue-gateway","ipv4":"10.20.6.52"},"annotations":[{"timestamp":1618886174659000,"value":"brave.flush"}]},{"traceId":"607e3e086301ffaf74e217d5983efbc5","parentId":"74e217d5983efbc5","id":"864bdbd5a06a0515","kind":"CLIENT","name":"get","timestamp":1618886152331773,"duration":5668,"localEndpoint":{"serviceName":"lingyue-gateway","ipv4":"10.20.6.52"},"tags":{"http.method":"GET","http.path":"/auth/check"}},{"traceId":"607e3e086301ffaf74e217d5983efbc5","id":"74e217d5983efbc5","kind":"SERVER","name":"get","timestamp":1618886152322007,"duration":11542205,"localEndpoint":{"serviceName":"lingyue-gateway","ipv4":"10.20.6.52"},"remoteEndpoint":{"ipv4":"123.157.238.102"},"tags":{"http.method":"GET","http.path":"/base-msg-center/api/msgcenter/msg/lastest"}}]
2 replies
