Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    tim
    @timhungdao
    anytime I start the consul agent (client mode) it eat all the server memory and get killed by the kernel
    Yann Huissoud
    @aiqency
    Similar to this issue: hashicorp/consul#1545 I want to add tags to services programmatically in go. /v1/catalog/register is expecting the full service definition (api.AgentServiceRegistration), but I couldn't find any API to get it in the first place (in order to modify it).
    /v1/catalog/service/ also doesn't return all required values.
    2 replies
    I forgot to specify that this must be done remotely. Basically some code on a machine would query the catalog and update the service tags.
    iluciv
    @iluciv:matrix.org
    [m]
    Hey there I needed to set up dns with iptables forwarding on the mangaement servers following here https://learn.hashicorp.com/tutorials/consul/dns-forwarding?utm_source=consul.io&utm_medium=docs#iptables-setup
    would I need to apply these nat forwarding rules on both the consul agents running in client mode and the consul server cluster agents
    at the moment I only have it set on the management server cluster
    the clients are running as non root and have the port set to 8600 in their config so I'm assuming that they will need the nat forwarding also. Management nodes are in a forward look up zone in our dns.
    iluciv
    @iluciv:matrix.org
    [m]
    oh we have fabio running on the worker nodes (those with consul running in client mode) does that constitute a relay host? (not that flash on dns)
    chrisvanmeer
    @chrisvanmeer:matrix.org
    [m]
    Depends on your DNS setup I guess...in my setup, both Consul servers and clients have dnsmasq installed so that they will default query their own DNS instance. The rest of the servers in a separate VLAN use the standard DNS servers of the environment, which has a conditional DNS forwarder for .consul that forwards to the 5 Consul servers.
    In any way, it would be beneficial when Consul clients would be able to resolve the .consul domain.
    iluciv
    @iluciv:matrix.org
    [m]
    Thanks chrisvanmeer
    Ilya Balashov
    @ibalashov24

    Hello! I have a flood of the following nasty warnings on my Consul installation (v.1.11.3):

    [WARN]  agent: Service name will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes.: service=sth-with_underlines

    Unfortunately, in my case, service renaming is not feasible. At the same time, my setup does not use the DNS interface at all, so complete DNS disabling would be an appropriate solution, I think. I've tried to set a negative DNS port as suggested here: hashicorp/consul#3135 using CLI flag "-dns-port -1" , but it seems to have no effect.
    Could you please advise if there is any way to disable DNS (or solve the warning problem)?

    1 reply
    Narendra Patel
    @narendrapatel
    Hi, we had certificates expiry for 2 of our connect enabled services ( 1 is on VMs and other is on K8s). Consul should have renewed them post the 72 hours window but it didn't.
    We had to restart the service in VM and recreate 1 pod replica for the K8s service to solve the issue. Have avoided restarting the other replica for time being on K8s to further debug the issue.
    4 replies
    Narendra Patel
    @narendrapatel
    One question on monitoring envoy sidecars. Envoy exposes a lot of metrics at the all levels like Listener, Cluster, HTTP level stats, Server level stats, etc.. What should be a good set of key metrics to monitor and perhaps alert on. Istio provides a set of precanned dashboards for use. Do we have some thing similar for Consul?
    1 reply
    Ryan Goltry
    @grogsaxle
    Is there a setting to tell consul to log DNS requests? Not that I'm sold on logging DNS requests on consul, there are other options, but wondering if dns logging in consul is an option.
    Michael Aldridge
    @the-maldridge
    this sounds like an XY problem @grogsaxle what is the problem you're trying to solve?
    5 replies
    Alex Oskotsky
    @aoskotsky-amplify
    Does terminating gateway support connecting to backends that use TLS? I am trying to put an AWS API Gateway behind a Consul terminating gateway
    1 reply
    iluciv
    @iluciv:matrix.org
    [m]
    In a consul cluster should you be setting up the worker nodes to be pointing to the consul server cluster as the dns servers? So for example in address block have the 3 consul server nodes? should the consul server node if using iptables be the only ones with recursers on them or should all node have recursers statement in the config?
    2 replies
    using port 8600 running consul as non root user in this instance
    Mauricio Dantas
    @MauricioIsARed_twitter
    image.png
    1 reply
    did anyone had problems while running consul from systemd?
    Michael Aldridge
    @the-maldridge
    that looks like a pretty standard permissions issue, can you double check that that directory is owned by the expected user?
    ShellFu
    @shellfu

    Hey all, we're running consul on kubernetes. We had to rotate our kubernetes certificates and everything came up fine Consul wise after the restart, however all of the consul-connect-inject sidecars cannot start due to x509 unknown authority "ca".

    I restarted the agents and servers again but this did nothing and am about to attempt the cert rotation process documented.

    Anyone experience this after rotating k8s certs?

    1 reply
    Michael Aldridge
    @the-maldridge
    @blake this page appears to be out of date or otherwise incorrect: https://www.consul.io/docs/connect/configuration it specifies its not required to specify connect { enabled = true } on clients, however without this specified nomad fingerprints the node as attr.consul.connect = false is this a nomad bug or a consul docs bug?
    10 replies
    Daniel Kimsey
    @dekimsey

    I have an ingress-gateway with a service-router to split the L7 traffic (following the docs for HTTP listener with Path-based Routing). But the envoy instance only ever reports "no healthy upstreams".
    Curiously, envoy /clusters shows all the configured upstream clusters (0 on all stats) and /config_dump shows all the routing config looking sane. I'm not 100% clear on what intentions should be set (ingress name -> router or ingress-name -> final destination), but I've currently got a wildcard destination and it's having no effect. And even then I'd expect a 403 response there.
    Logs clearly showing it selecting the configured final-destination cluster (the destination after the service-resolver work is done) and then complaining there are no healthy upstreams. When I look at them in /clusters, I see the correct destination IPs (mesh-gateways) are listed.

    I'm at a loss as to why envoy might be considering the clusters to have no healthy upstreams here.

    3 replies
    Alex Oskotsky
    @aoskotsky-amplify
    Does failover with service-resolver configs support failing over to the closest datacenter if i provide a list of data centers or will it randomly pick from all of them?
    1 reply
    Johan Forssell
    @johanforssell
    intention.png
    This red line - how would I go about listing this from the CLI ?
    I.e. - I would like to know which intentions I've missed without going through the web UI
    consul intention list does not show missing intentions
    iluciv
    @iluciv:matrix.org
    [m]
    I posted this in nomad meant to post it here; Hi there I've inherited some environments, for dns I'm trying to understand why the people before me set the ip in the addresses stanza for dns on the worker nodes (both windows and linux) but on the management nodes they've set is 0.0.0.0. Whats would be the purpose of putting host dns field in there instead of just leaving it as 0.0.0.0 and defaulting to local. Having it this way means for dig queries etc you need to do dig @172.x.x.x -p 8600 some.address ALL instead of just doing dig @localhost
    1 reply
    Michael Aldridge
    @the-maldridge
    if I have a repeated block in my configuration that only changes by label, can I have consul-template iterate over a static list?
    Daniel Kimsey
    @dekimsey
    We are using mesh-gateways to route traffic between our DC's and I'm noticing if there are say 3 instances of a service and 2 are marked as unhealthy (failed_eds_health), the mesh-gateway is still forwarding traffic to them resulting in 1/3 success rate for connections. Even though it's well aware they are offline/dead. Does anyone know if that is an intentional design?
    Daniel Kimsey
    @dekimsey
    Relatedly, what would cause a mesh-gateway endpoint to show up as failed_eds_health in a sidecar's clusters output? For the life of me I cannot figure out what is causing this.
    Pierre Souchay
    @pierresouchay
    @the-maldridge if you are doing complicates stuff, you might have a look at https://github.com/criteo/consul-templaterb
    Michael Aldridge
    @the-maldridge
    @pierresouchay I was doing this entirely inside consul-template because I am in a nomad template block for this particular item. I eventually solved it by making this simpler and templating from a different perspective.
    [X]
    @ennetech
    Hi to all, in a two datacenter situation in which now all servers have a dedicated public ip, how wan gossip should be configured?
    Alex Oskotsky
    @aoskotsky-amplify
    Is there a way to override the stream_idle_timeout value in envoy through consul?
    Johan Forssell
    @johanforssell
    Is there a way to check which Agent token my Consul client X is currently using?
    I'm setting my agent token with consul acl set-agent-token agent bla-bla-ba.
    2 replies
    HelloWood
    @helloworlde

    Hi, I'm new to mDNS, I found the repo https://github.com/hashicorp/mdns, and I want add a dns for a Java server with port 8080 like:

    info := []string{"Demo service"}
    service, _ := mdns.NewMDNSService("demo", "_http._tcp", "", "", 8080, nil, info)
    server, _ := mdns.NewServer(&mdns.Config{Zone: service})
    defer server.Shutdown()

    After startup mDNS service, I can't request the dns demo._http._tcp.local, how should I add correct dns for this? Thanks very much.

    ShellFu
    @shellfu

    Hey everyone,

    I have a mesh gateway federated consul setup utilizing 3 k8s clusters using consul 1.11.3

    I have setup a service (service-foo) in each datacenter. In the primary dc I have also setup a service-resolver using a crd to failover to another dc like so:

    apiVersion: consul.hashicorp.com/v1alpha1
    kind: ServiceResolver
    metadata:
      name: service-foo 
    spec:
      connectTimeout: 1s
      failover:
        '*':
          datacenters:
            - dc1
            - dc2
            - dc3
    1. Made request to service-foo in dc1, returns 200
    2. Took service-foo down in dc1
    3. Made request to service-foo in dc1, returns 503

    I'd have expected Consul to route my request to the next closest with a healthy instance of service-foo.

    I have also tried specifying a default subset in the resolver but that did not change the result. The only way I have been able to get to another datacenter is by using redirect in the service-resolver but that is not what is desired here.

    Any help would be appreciated.

    3 replies
    Alvin Lin
    @alvinlin123
    I know this is not the place for Hashicorp's Memberlist question, but I am wondering where can I get some support on https://github.com/hashicorp/memberlist ? Is there a generic Hashicorp Gitter?
    1 reply
    Saddam ZEMMALI
    @saddam.zemmali_gitlab
    HI ALL,
    Please I need help to change the datacenter name in a running consul cluster
    4 replies
    Peter Borghard
    @peterborghard
    hey folks, anyone know how to add a header in consul connect? I'm able to match and set the destination. But can't seem to figure out the config for header manipulation.
    3 replies
    Peter Borghard
    @peterborghard
    Anyone know of any good docs on setting up an http2->grpc proxy using consul connect? I can't seem to get it working, tried a few different ways.
    3 replies
    Markus Keil
    @thereapman
    Hi All,
    I'm facing failed_eds_health issues between a sidecar and a service in another data center using Meshgate WAN federation.
    the sidecar sits in a K8s deployment that has its consul servers externally (no consul srvs inside k8s).
    Accessing the service works fine from a non-k8s sidecar on a VM in the same DC as the k8s.
    The Helm chart config reference doesn't give any clues on what i'm missing.
    Is there a way to debug the health check path or any other leads i could follow?
    2 replies
    Blake Covarrubias
    @blake
    Have ideas or need some troubleshooting assistance for Consul? Meet with us and tell us all about it! Sign up at https://hashicorp.sjc1.qualtrics.com/jfe/form/SV_b7cNuNBMrPr4b8q.
    madhucs
    @madhucs:matrix.org
    [m]
    Need help , Trying to run CONSUL on EKS (Kubernetes cluster) , I see pods are not comming up the exception I see for pod is below ->

    2022-04-27T21:40:58.980112134Z
    ==> failed to parse /consul/config/..2022\_04\_27\_21\_29\_57.194001731/server.json: 1 error occurred:
    \* invalid config key auto\_reload\_config


    This is my configuration, deploying through HELM 3+
    client:
    enabled: false
    nodeSelector: |
    dev/group: tools
    tolerations: |
    - key: "tools"
    operator: "Equal"
    value: "true"
    effect: "NoSchedule"
    global:
    datacenter: adapt
    name: consul
    server:
    enabled: true
    extraConfig: |
    {
    "dns\_config": {
    "service\_ttl": {
    "\*": "15s"
    },
    "node\_ttl": "5s",
    "max\_stale": "5m"
    }
    }
    image: consul:1.10.3
    nodeSelector: |
    adapt/group: tools
    resources:
    limits:
    cpu: 300m
    memory: 300Mi
    requests:
    cpu: 300m
    memory: 300Mi
    tolerations: |
    - key: "tools"
    operator: "Equal"
    value: "true"
    effect: "NoSchedule"
    ui:
    enabled: true
    service:
    annotations: |
    service.beta.kubernetes.io/aws-load-balancer-name: mynlb
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-scheme: internal
    service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
    service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags:Environment=Dev
    type: LoadBalancer
    3 replies
    madhucs
    @madhucs:matrix.org
    [m]
    Woohoo @blake Consul 1.12.0 worked , thank you very much it saved lots of my time !! :-)
    Blake Covarrubias
    @blake
    You're welcome. :-)