by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Adam Janiš
    @adam-janis
    Consul have some love in here :)
    https://blog.cloudflare.com/unimog-cloudflares-edge-load-balancer/
    6 replies
    Jude DSouza
    @dsouzajude

    Hi, with regards to Consul Connect, is it possible to enable HTTP Tracing on the Ingress Gateway? It says we can enable it using Proxy.Config [1] but i get a key error on that. Any idea?

    [1] https://www.consul.io/docs/connect/gateways/ingress-gateway#ingress-gateway-configuration

    12 replies
    Jude DSouza
    @dsouzajude

    Based on this[1] i'm assuming in some maybe beta version of Consul (connect), Envoy version 1.15.0 is supported. Is there anyway I can test it on my end? Is there a docker image that I can use?

    Would appreciate any help! Thanks!

    [1]hashicorp/consul#8424

    Blake Covarrubias
    @blake
    @dsouzajude You could try https://hub.docker.com/r/hashicorpdev/consul. These are generated on every commit to the master branch. No guarantees its stable.
    Jude DSouza
    @dsouzajude
    Thanks a lot! It's not for production anyway! :)
    Blake Covarrubias
    @blake

    We recently re-organized the documentation on consul.io to hopefully make it easier to find relevant content https://www.consul.io/docs as well as introduced search functionality.

    A few folks here have previously commented about the difficulty in locating docs with the old layout. Hopefully this is an improvement. Please share any feedback you may have. Thanks!

    Michael Aldridge
    @the-maldridge
    in fewer than 3 clicks I was able to find the page listing all the command line options, major improvement in my book
    Mirza Waqas Ahmed
    @mirzawaqasahmed
    Hi All, Is there a way to have a Watcher instantiate based on specific event types?
    For instance I would like to set up a watch that may be empty in the beginning...but if it gets populated with nodes,,,I'd like to run a script...
    However, if there services is deleted I'd like to run another script?
    Any suggestions...?
    Priyanka Sengupta
    @munali
    When I run "consul debug" I see these error messages. Any ideas what they mean or how to fix?
    [ERROR] agent.dns: all resolvers failed for question from client: question="{ip-xx-xx-xx-xxx. 28 1}" client=127.0.0.1:34083 client_network=udp
    Blake Covarrubias
    @blake
    @munali I believe that indicates that your upstream resolvers are not responding to the query. What resolvers do you have defined in /etc/resolv.conf, or in Consul’s recursors setting?
    Priyanka Sengupta
    @munali

    in /etc/resolv.conf I have

    options rotate
options timeout:1
search oe.flatiron.com
nameserver 127.0.0.1
nameserver 123.45.67.89
nameserver 101.112.13.14

    and in consul's configuration, I have 

    "recursors": ["123.45.67.8", "101.112.13.14"]
    @blake I am recursing to some DNS servers I have setup from Consul and in resolv.conf, it first tries Consul running on port 53 (127.0.0.1) or otherwise, tries the same DNS servers I have setup
    Blake Covarrubias
    @blake
    Does that same query succeed if you manually run it against the resolvers?
    Priyanka Sengupta
    @munali
    yes if I do dig @123.45.67.89 www.google.com it works
    it works with all three nameservers in resolv.conf
    Priyanka Sengupta
    @munali
    the logs show the error happening intermittently so maybe the upstream dns resolvers are failing sometimes?
    Blake Covarrubias
    @blake
    That might be the issue. Do you have access to the DNS resolver’s logs in order to troubleshoot the query there?
    Priyanka Sengupta
    @munali
    yah I do, good idea, i'll try that out
    thx!
    Blake Covarrubias
    @blake
    You’re welcome. 🙂
    Jesse Quinn
    @jessequinn
    hey guys, i am having a weird issue here. I have docker-compose with a consul and consul-worker. I am getting a "No cluster leader” found in the consul-worker log. If i do a consul info on the consul container i get leader=true. And when i do a consul join ip on the consul-worker a message of success is received. However, it does not join the cluster. And the error "No cluster leader” remains. Any ideas how to fix this?
    Abhi
    @abhiiiiiiiiiii__twitter
    Hi does anyone know if there are any flags which i pass to ensure that Consul KV update/create api does not delete existing value when updating ? I have a Key which contains multiple values, is only way to update this key is to pass all the values everytime instead just add value which i want to add to already existing list
    Blake Covarrubias
    @blake
    Consul does not support a patch operation when updating keys since it isn’t aware of the data type being stored in a key. You’ll need to provide the entire payload value.
    Abhi
    @abhiiiiiiiiiii__twitter
    Thank you Blake
    Priyanka Sengupta
    @munali
    I could not tell from the documentation, but if I provide consul with a list of recursors, will it try them in a round robin order? Similar to resolv.conf 'rotate' option?
    4 replies
    Jude DSouza
    @dsouzajude

    Hi
    Is there a way to do a health check on the Consul Ingress Gateway [1] itself?

    [1] https://www.consul.io/docs/agent/config-entries/ingress-gateway

    Rodrigo Pereira
    @voiprodrigo
    Hi. Quick question.. how can I know Consul's cluster leader via DNS interface? So far only consul.service.dc.consul gives me something, but that's round-robin over all nodes
    Michael Aldridge
    @the-maldridge
    it is not possible to determine the leader over the DNS interface because consul does not set a tag for the active master
    Rodrigo Pereira
    @voiprodrigo
    Right. Found this issue on GH hashicorp/consul#4945 too.
    Michael Aldridge
    @the-maldridge
    this is almost certainly the wrong way to solve your problem though
    what are you actually trying to do?
    Rodrigo Pereira
    @voiprodrigo
    It was more out of curiosity. I guess the only advantage would be saving a forward by a follower.
    Michael Aldridge
    @the-maldridge
    but you generally shouldn't be hitting a consul server directly anyway
    Blake Covarrubias
    @blake
    @voiprodrigo Consul allows certain queries to be serviced by followers when using a stale consistency mode. It also allows configuring the TTL for DNS records so that responses can be cached for some time. What specifically are you trying to do?
    Rodrigo Pereira
    @voiprodrigo
    I have set stale DNS option. In my mind it made sense to be able to use a leader record to configure in a reverse proxy. I understand that ideally a proper load-balancer with active checks should handle this.
    Tobias Dahlberg
    @somebadcode
    I'm trying to set up a client on a machine and make it part of the cluster. The problem is that when I register a service using the local client, the servers say "no terminating-gateway or ingress-gateway associated with this gateway: gateway=simple-service". The client reports no errors and the healthcheck is fine. The service never shows up in the cluster either. I haven't found any satisfactory documentation regarding this. I often end up in the Connect documentation which is not what I'm trying to do.
    1 reply
    It's like it expects a service mesh but I only want to register a simple service.
    Rogier Wensink
    @Wensink

    Hello. We are using consul for years now and to full satisfaction. But lately we are encountering an issue which we cannot get solved.
    When changes are done to the master cluster and consul is restarted on all masters at the same time, the slaves (and other nodes too) lose the connection. In code we you a join and a rejoin (although the rejoin is completely ignored once the connection is established).

    consul:1.6.2 -join ${CONSUL_JOIN} -retry-join ${CONSUL_JOIN} -node ${LOCAL_IPV4} -advertise ${LOCAL_IPV4} -client 0.0.0.0 -data-dir /data -encrypt ${CONSUL_KEY} -log-level err

    What is the best way to get all nodes connected to the master cluster again, if these are restarted all at once?

    3 replies
    Shantanu Gadgil
    @shantanugadgil
    @Wensink afaik only retry join should suffice. I am not sure why you use both join and retry-join. That said, if you can make the agents "always restart" (with a delay) it should do the job.
    For systemd based consul agent I keep a short delay (5 or 10 seconds) between service restarts
    (i am talking about the equivalent for docker, in your case)
    If you are on a cloud, you should try out thd cloud auto join ... so the problem of server changing ips also is solved
    Rogier Wensink
    @Wensink

    We have consul running in AWS. For joining I thought it would first try the -join. And when that fails, continue with the -retry-join. But if only -retry-join will do, I'll remove the first part.
    The restart you are talking about is that for the master-cluster? To not restart all at once, but keep a delay in place for that?

    The auto-join we tried yesterday, and works nicely. But we didn't continue that, because it didn't fix our issue

    Tobias Dahlberg
    @somebadcode
    Do you have to set up a terminating or ingress gateway for service registration to fully work?
    When ever I register a service, all I get is:
    "no terminating-gateway or ingress-gateway associated with this gateway: gateway=simple-service"
    Tobias Dahlberg
    @somebadcode
    Registering a service in Consul that's running in dev mode works just fine but I can't get it to work at all when I have a cluster with ACL. The local agent client accepts it but the servers never registers it and just return the error I quoted. :(
    Tobias Dahlberg
    @somebadcode
    Is it mandatory to configure something like nginx, HAProxy or similar for Consul to even accept a service registration?
    Tobias Dahlberg
    @somebadcode
    Seems to be an ACL issue. The service shows up in the UI, the local agent that's in client mode shows it but I can't get a the expected result from asking the servers through DNS or HTTP. Trying to figure out how to make anonymous lookups work.
    Tobias Dahlberg
    @somebadcode
    Okay. It was the ACL causing trouble. I forgot the anonymous token. I've updated it and it's possible to use DNS now. Thanks for being my rubbery ducky while I debugged this.
    glokeshathena
    @glokeshathena

    Team, I have defined global-defaults.hcl with the "proxy-defaults" configuration for envoy_prometheus_bind_addr and tracing configuration.

    And I confirm that the configuration is loaded on to the consul agent with
    consul config read -kind proxy-defaults -name global

    But in the generated bootstrap json for envoy, I don't see the listener for prometheus endpoint and tracing clusters info in the generated bootstrap json.

    consul connect envoy -bootstrap -sidecar-for visit-bill

    Any hint/trouble shooting tip to figure out why consul connect command is not honoring the global configuration while generating the bootstrap config for envoy would be highly appreciated.