Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Keshav Peswani
    @keshavpeswani
    Hi @jcchavezs : As of now, yes it trends, adaptive alerting and service graph
    Betty Zhang
    @_____Betty___twitter
    Hi, we have a Sprint Boot service that wants to integrate with Haystack, I've read the documentation but there are a bunch of repos under ExpediaDotCom and I'm confused with which one should be the right fit, can someone please advise?
    Keshav Peswani
    @keshavpeswani
    Hi @_____Betty___twitter : You can use this library : https://github.com/ExpediaDotCom/opentracing-spring-haystack-starter
    Let me know if are you facing any issues
    Betty Zhang
    @_____Betty___twitter
    @keshavpeswani Hi Keshav, our application is using Sleuth instead of opentracing, would this be a better fit? https://github.com/ExpediaDotCom/spring-cloud-sleuth-haystack-reporter
    José Carlos Chávez
    @jcchavezs
    You can also use zipkin instrumentation (https://github.com/openzipkin/sleuth-webmvc-example) and ingest data using pitchfork: https://github.com/ExpediaGroup/pitchfork
    Keshav Peswani
    @keshavpeswani
    @_____Betty___twitter : yes, then the sleuth library would be a better fit. Alternatively you can use a zipkin instrumentation and use haystack-agent as mentioned by JC above to send data in zipkin format https://github.com/ExpediaDotCom/haystack-agent#zipkin-agent-pitchfork
    Betty Zhang
    @_____Betty___twitter
    @jcchavezs I heard that Haystack supports receiving data in Zipkin format, if that is true, pitchfork might not be needed?
    Keshav Peswani
    @keshavpeswani
    Yes, Betty , haystack can ingest in zipkin format via haystack-agent.
    Betty Zhang
    @_____Betty___twitter
    @keshavpeswani What's the different between utilizing Sleuth to instrument directly to Haystack versus using a haystack-agent?
    Keshav Peswani
    @keshavpeswani
    For sending the data to haystack, one needs to send it via haystack-agent or http endpoint. HTTP endpoint can ingest the data in haystack format whereas the agent runs as a sidecar and can ingest the data in either zipkin format or haystack format and send it to haystack
    Alternatively as @jcchavezs mentioned, one can use pitchfork and send to it as well but better approach would be to use inbuilt pitchfork in haystack-agent
    José Carlos Chávez
    @jcchavezs
    Yeah, sorry I forgot to mention about the haystack-agent. Totally.
    Betty Zhang
    @_____Betty___twitter
    So to clarify, if we want to directly send to http endpoint using Sleuth, we need to convert zipkin format to Haystack format. If we use a haystack-agent we will not need to do conversion. Is that correct?
    José Carlos Chávez
    @jcchavezs
    yes. Basically you will send the data (in zipkin format) to haystack-agent and haystack agent will dispatch the data into haystack server (in haystack format).
    So no need to do convertions or whatsoever.
    Betty Zhang
    @_____Betty___twitter
    Got it, is there any guideline on which way suits for which scenarios?
    José Carlos Chávez
    @jcchavezs
    That is actually a good question. I would like to hear @keshavpeswani and @worldtiki on this one.
    Keshav Peswani
    @keshavpeswani
    So, haystack-agent works on top of pitchfork only. Performance wise they would be same. I suppose sending data to haystack kafka would be more performanent than sending it to an http endpoint to pitchfork server
    Betty Zhang
    @_____Betty___twitter
    I would also like to know if there is an existing library for haystack-agent and a sample project for reference?
    Keshav Peswani
    @keshavpeswani
    Haystack-agent is a separate side car / separate process. Refer to the architecture diagram : https://expediadotcom.github.io/haystack/docs/about/architecture.html
    App sends data to agent running as side car and then agent sends the data to haystack
    Betty Zhang
    @_____Betty___twitter
    If sending to HTTP endpoint, which component would it send to in the architecture diagram? Is it Haystack UI?
    Jason Bulicek
    @bulicekj
    HTTP collector pushes spans into Kafka, which connects to the different subsystems. Haystack UI reads from those components (traces, trends, service graph, alerting)
    Kapil Rastogi
    @kapilra
    In architecture diagram, it should fall under haystack-collectors
    Kranti Vikram A
    @vikrambe
    Hi Folks, We are trying to deploy haystack on AWS account. haystack ami's are not available on aws ami public repo.Can someone throw some light on below error Unable to discover AWS AMIs which running terraform template Error: Error refreshing state: 4 error(s) occurred: module.haystack-datastores.data.aws_route53_zone.haystack_dns_zone: 1 error(s) occurred: module.haystack-datastores.data.aws_route53_zone.haystack_dns_zone: data.aws_route53_zone.haystack_dns_zone: no matching Route53Zone found module.haystack-k8s.data.aws_route53_zone.haystack_dns_zone: 1 error(s) occurred: module.haystack-k8s.data.aws_route53_zone.haystack_dns_zone: data.aws_route53_zone.haystack_dns_zone: no matching Route53Zone found module.haystack-datastores.module.kafka.data.aws_ami.haystack-kafka-base-ami: 1 error(s) occurred: module.haystack-datastores.module.kafka.data.aws_ami.haystack-kafka-base-ami: data.aws_ami.haystack-kafka-base-ami: Your query returned no results. Please change your search criteria and try again. module.haystack-datastores.module.cassandra.data.aws_ami.haystack-cassandra-base-ami: 1 error(s) occurred: module.haystack-datastores.module.cassandra.data.aws_ami.haystack-cassandra-base-ami: data.aws_ami.haystack-cassandra-base-ami: Your query returned no results. Please change your search criteria and try again.
    Keshav Peswani
    @keshavpeswani
    @vikrambe : You can use packer to build images in your VPC. Reference : https://github.com/ExpediaDotCom/haystack/tree/master/deployment/terraform/packer. You need to call the build.sh with certain variables.json.
    Let me know if you face any issues.
    Keshav Peswani
    @keshavpeswani
    CC'ing @kapilra for helping . It would be great if we can update the documentation too.
    Kapil Rastogi
    @kapilra
    Sure, will get it done. Thanks @keshavpeswani !
    nsahai8
    @nsahai8
    @vikrambe although @keshavpeswani has already mentioned the package that can help. You will be required to fill the details according to your requirement in these variables "region": null,
    "vpc_id": null,
    "subnet_id": null
    in file: https://github.com/ExpediaDotCom/haystack/blob/master/deployment/terraform/packer/variables.json
    meanwhile let me add this detail in readme as well
    Kranti Vikram A
    @vikrambe
    @nsahai8 @keshavpeswani cool. Thanks for quick response. i will give it a try
    th100 h
    @th100_gitlab
    @keshavpeswani @nsahai8 Currently the build.sh script spins up an EC2 instance, but SSH to the instance is timing out. This could be due to our Org's restriction with allowing traffic from 0.0.0.0/0. Is it possible to whitelist specific IP instead of 0.0.0.0?
    Keshav Peswani
    @keshavpeswani
    @th100_gitlab : ssh to which type of resource is timing out ? Is it cassandra, or kafka or k8s boxes ? Can you help me with that so that I can direct you to exact change in config
    th100 h
    @th100_gitlab
    @keshavpeswani timeout while connecting to cassandra and kafka instances
    th100 h
    @th100_gitlab
    @keshavpeswani can you please help with the above timeout issue? Basically we want to configure the correct IP range to allow the script to SSH into the instance?
    Keshav Peswani
    @keshavpeswani
    These two properties need to be overriden with right CIDR's: node-elb_ingress = "0.0.0.0/0", node_ingress = "0.0.0.0/0"
    nsahai8
    @nsahai8
    @vikrambe if you are using bastion to do ssh on the ec2 this could help: https://www.packer.io/docs/builders/amazon/instance#ssh_bastion_host
    th100 h
    @th100_gitlab
    @keshavpeswani @nsahai8 I added the below snippet in deployment/terraform/cluster/aws/infrastructure/overrides.json, but it still sets 0.0.0.0 in the security group. Is 0.0.0.0 hardcoded in the binary?
    {
    "cluster": {
    "elb_ingress": <MY IP>,
    "node_ingress": <MY IP>
    }
    }
    error snippet is as below:
    ==> amazon-ebs: Waiting for SSH to become available...
    ==> amazon-ebs: Timeout waiting for SSH.
    ==> amazon-ebs: Terminating the source AWS instance...
    nsahai8
    @nsahai8
    @th100_gitlab if you are accessing the ec2 using bastion, you can checkout https://www.packer.io/docs/builders/amazon/instance#ssh_bastion_host for setting up ssh username and password
    Keshav Peswani
    @keshavpeswani
    @th100_gitlab : did it work for you ? Ideally, overrides.json should work. Can you paste the command that you running for install-all ?
    it will help in figuring out the issue in case it didnt work
    th100 h
    @th100_gitlab
    @nsahai8 we are not using a bastion to access the ec2.. we are deploying the ec2 in IngressSubnet, so we can ssh to the instance directly..
    @keshavpeswani we did not go till "install-all" step.. we are getting this failure while running "./build-images.sh"
    nsahai8
    @nsahai8
    @th100_gitlab let me explain a little bit. So when EC2 is up , packer tries to run the shell scripts etc from provisioner section. But currently the host ( from where you are running build-images.sh) is not able to do ssh on the box. Can you share how do you access boxes in that account.
    nsahai8
    @nsahai8
    You might be able to do SSH from local to those EC2 boxes in that VPC either by some ssh or via bastion.