Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Shantanu Gadgil
    @shantanugadgil
    Hi, I am not surs if you can install inside a chroot (exec). wouldn't package installation be more of a raw_exec task? Running as root. ???
    NickMos
    @NickMos
    Hi all, I have a quite simple job with template stanza inside with change_mode = "restart" but Nomad doesn't rerender the file inside a alloc. How could I debug this process?
    my template file is used for env and consists only of key=value pairs without {{ }} sections.
    Shantanu Gadgil
    @shantanugadgil
    @NickMos use a nice-and-unique name for the destination file. Then once Nomad report that the allocation has started, do a "find <path> -name <filename>" inside the data_dir
    you should see the file inside the "alloc id" directory somewhere
    :)
    Hi folks, has anyone faced something similar to: hashicorp/nomad#6471
    My understanding of batch tasks has always been from https://www.nomadproject.io/docs/schedulers.html#batch and that "batch tasks will not rerun is they exited 0"
    James Rasell
    @jrasell
    @shantanugadgil yep I would agree with that understanding, unless you're using the cron stanza of course in the batch job.
    NickMos
    @NickMos
    @shantanugadgil file is there
    destination = "local/file.env"
    and in alloc
    ls -la /var/nomad/alloc/cfb36a54-8f97-14f7-831e-f85bc0f5fe74/backend/local/file.env
    -rw-r--r-- 1 root root 229 Oct  9 16:38 /var/nomad/alloc/cfb36a54-8f97-14f7-831e-f85bc0f5fe74/backend/local/file.env
    Also source = "/apps/backend/env.tpl" and if I change /apps/backend/env.tpl file nothing is happened with the Job...No updated in alloc dir and no restarts. Maybe I missed something... =(
    Shantanu Gadgil
    @shantanugadgil
    hi @NickMos I think the "action" comes into play if the rendered value of the template changes. I doubt if the template pays attention to the changes in the source file (I am not sure).
    The template functionality comes from consul template, so you can check there for specifics. If you can experiment with "consul-template" and notice a change in behavior between CT and Nomad's template functionality, then I would consider this as a bug.
    @jrasell :smiley: yes, of course, cron and parameterized are special cases.
    NickMos
    @NickMos

    @shantanugadgil

    pays attention to the changes in the source file (I am not sure).

    Yes, me too. It is not clear from docs. =/

    Sergey Pashinin
    @pashinin
    Why "nomad node status" doesn't list versions? I like how "consul members" gives Consul versions on all nodes. I want to see all Nomad versions as well.
    James Rasell
    @jrasell
    @pashinin try adding -verbose on the end
    Sergey Pashinin
    @pashinin
    Thanks
    Farhad Shahbazi
    @Grauwolf_gitlab
    Hm, when running vault with consul as storage backend, would it be completly stupid to run vault via nomad/docker (in case of a Docker-infra)?
    Charlie Voiselle
    @angrycub
    If you aren't trying to leverage Nomad's Vault integration, it's not the most far fetched. I have definitely heard about other folks doing it. (it gives me the heebie-jeebies personally)
    There are some challenges around making sure that you have a means to unseal in the event that you need to migrate one of the vault allocs.
    Farhad Shahbazi
    @Grauwolf_gitlab
    @angrycub Ah true! Haven't thought about unsealing. Hm
    K. Outside of nomad and on another host(s) it is. Thanks! :)
    Michael Aldridge
    @the-maldridge
    @stevenscg I had to adjust ulimits for alpine
    @msvbhat what you're looking for is called meta-monitoring. It isn't a prometheus specific concept either, its the idea that you need some external monitor that monitors your monitoring infrastructure and screams if it goes down. Similarly the metamon should be monitored by the monitoring system to know if it goes down.
    @Grauwolf_gitlab we run vault on the metal colocated with the consul servers. Its not necessarily recommended for scaling reasons, but we closely guard what the read/write rate is and monitor to know if anything looks odd
    I think with that I'm all caught up on my scrollback
    Farhad Shahbazi
    @Grauwolf_gitlab
    @the-maldridge thanks!
    Also, I'm currently playing around with the Docker driver (having used qemu, exec and some others before). In general everything seems to work as advertised, I'm just having a problem to connect Services with dynamic ports, with each other.
    Say task A needs to talk to Task B. Task B has 3 of it's kind running (via group -> count). All of those Task B tasks have a different dynamic Port. What do I write in the config file of Task A?
    The only way I'm seeing right now is adding some reverse proxy in-between or adding consul logic to the application (which I can't do)
    James Rasell
    @jrasell
    @Grauwolf_gitlab you could use Fabio or similar as you mention to provide reverse-proxy. You need something like this, as if an allocation fails and is restarted it might come up on a different dynamic port.
    Farhad Shahbazi
    @Grauwolf_gitlab
    The solution I'm using for my qemu jobs is them all using the same ports and getting a free IP in the LAN range
    NickMos
    @NickMos
    I use Nomad+Consul as SD and Fabio as revers-proxy. Works well.
    Farhad Shahbazi
    @Grauwolf_gitlab
    @jrasell having watched your fabio talk, I figured as much :)
    NickMos
    @NickMos
    Also you can use different docker network driver
    Farhad Shahbazi
    @Grauwolf_gitlab
    Would Consul Connect help here?
    James Rasell
    @jrasell
    @Grauwolf_gitlab I hope its helpful. Consul connect for sure can help in this situation. If you eventually have access to change the codebase and are using gRPC for example, you can plug Consul dynamic discovery into the gRPC resolver map.
    Farhad Shahbazi
    @Grauwolf_gitlab
    @jrasell coming from k8s where everything is semi-automatic + complex or one has far too many options for too many things, the kind of handholding provided by your talk was very helpful, yes.
    OK, going to try out fabio
    James Rasell
    @jrasell
    Good to hear; I usually hang around in here also so will try and help out with questions you have
    Farhad Shahbazi
    @Grauwolf_gitlab
    @jrasell Is there a way for setting the fabio dest via nomad service-tags? basically the request comes in as example.com/api/application1/health, since there is not one api, but more. but the application1 mounts to /api/ so the health request should be /api/health from it's view. in haproxy i did reqirep ^([^\ :]*)\ /api/application1/(.*) \1\ /api/\2. if i understand https://fabiolb.net/cfg/ correctly, this should be possible via <dst>
    James Rasell
    @jrasell
    @Grauwolf_gitlab yes, in the service tags, if you're using the standard Fabio config you can add a tag such as tags = ["urlprefix-/api/application1/"]
    Farhad Shahbazi
    @Grauwolf_gitlab
    @jrasell this is the part i've already got working. but i need to rewrite the request. i need something like strip= but strip only seems to be able to strip from the beginning of the path, not from the middle
    James Rasell
    @jrasell
    I don't know how that would work off the top of my head I am afraid
    Farhad Shahbazi
    @Grauwolf_gitlab
    hm ok
    OK, I found this. fabiolb/fabio#679
    So fabio might now work for my use-case at all
    Daniel Santos
    @danlsgiga
    Yup and that is one of the reasons I’m moving away from Fabio
    Evaluating Caddy with consul srv dns records for service discovery atm
    Farhad Shahbazi
    @Grauwolf_gitlab
    I might even go the overlay-network route where each Container gets an IP (like k8s) and services bind to the same port, instead of a dynamic port. And let services talk directly to each other. I might even go IPv6 instead of the overlay-network. I'm not sure yet and am still in the "thinking about it" phase.
    There seems to be a weave example in the Nomad repository https://github.com/hashicorp/nomad/blob/master/scripts/example_weave.bash
    Daniel Santos
    @danlsgiga
    Cool, I’m trying to avoid complexities... we don’t have microservices so service mesh is an overkill for us... I leveraged envoy for our proxying needs but that seems to be overkill for our needs