by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Timothy Perrett
    @timperrett
    enabling people to bring their own workflows and logic, in an arbitrary way is super powerful
    Cory Parent
    @goedelsoup
    i think this paves the way to make some HA improvements on the server (non-agent) side too
    Timothy Perrett
    @timperrett
    @goedelsoup i would like to understand your comment about compliance more though. What about mTLS is not secure enough / not compliant?
    Cory Parent
    @goedelsoup
    that's mostly thinking out loud. i suppose as long as your cert supply chain isn't compromised you couldn't have a rogue agent join to intercept any secrets that may be made available.
    Timothy Perrett
    @timperrett
    right i mean, that’s outside the scope of the project. We are assuming you have an effective secret distribution method, even with the current nelson implemetnation (which requires vault privs and so forth)
    its a meta problem
    Cory Parent
    @goedelsoup
    for sure. that's totally fair.
    Timothy Perrett
    @timperrett
    awesome

    @goedelsoup about this:

    The definition of an agent implies a cluster/LAN singleton. Can we better enforce this via a semaphore built in Consul or DynamoDB? Perhaps this should also be pluggable. I'd prefer the constraint of only one can run over advising to only ever run one.

    could you say a bit more? I’m not entirely following
    Cory Parent
    @goedelsoup

    This agent is assumed to be the singular agent in charge of that DC.

    Basically, just that this can be enforced via a leader election process.

    Timothy Perrett
    @timperrett
    i see it more like a mutable cell with a mutex - only one agent can be occupying the role as arbitor for the datacenter at a time.
    Adelbert Chang
    @adelbertc
    i think hes saying instead of the CP optimistically treating "latest agent to register wins" it uses proper leader election protocol
    controlled via Consul or ZK or something
    Timothy Perrett
    @timperrett
    isnt that an implementation of the agent though? If you need HA, then sure you can do leader election
    Cory Parent
    @goedelsoup

    which gives specified behavior to this

    If another agent later tries to register for the same DC the CP will optimistically assume a new Agent has taken over.

    Timothy Perrett
    @timperrett
    i would really like to avoid agents needing to do raft. its vastly more complex to orchestrate and im not sold on the benifits
    Architecturally it seems like a chioce agent implementors could have if they wanted it
    on the CP side, you’re alway going to have one registered
    Cory Parent
    @goedelsoup
    that's totally fair, especially given there isn't a good JVM implementation
    Timothy Perrett
    @timperrett
    happy to be wrong by the way, just trying to explore the space and make sure im not missing something
    Cory Parent
    @goedelsoup
    yeah, i'd say all of my questions are of the form, can we make better guarantees on X and is it worth the cost
    Adelbert Chang
    @adelbertc
    ^ this was approximately our discussion yesterday too :laughing:
    Timothy Perrett
    @timperrett
    like all good engineering discussions “It depends”
    im paranoid about operational complexity, as what we have right now “just works”
    so i’d like something that is operationally of equal complexity (within a given approxomation)
    Cory Parent
    @goedelsoup
    for sure. i'm 100% on board and i think you've debunked my devil's advocate questions.
    Adelbert Chang
    @adelbertc
    coolio i am planning on starting work soon then
    Timothy Perrett
    @timperrett
    :fire:
    in parallel our team will be working on the branch deployments RFC
    @adelbertc how are you thinking to refactor? Ideally we would have the agent in another repo, to keep us honest about the boundaries of certain code
    perhaps we should make a repo for it then copy the code over there, then we can keep the 0.14 on the train its on, with another branch (perhaps 1.x) that has these major changes
    i’d like to have a 0.15 with the branch deployment feature
    Adelbert Chang
    @adelbertc

    @timperrett My plan is:

    1. Split the pipeline processor and cleanup pipeline into their distinctive control plane/data plane parts - e.g. "what to deploy" vs. "how to deploy" and "mark as garbage" vs. "sweep garbage"
    2. Come up with the Protobuf data models for the events
    3. Write a reference implementation of a data plane that mimics the status quo
    4. Sink the pipeline processor, cleanup pipeline, sweeper, and deployment monitor into a network port
    5. Migrate the routing cron

    I think #1 and #2 will be good refactorings anyways. I will definitely put the agent in a different repo :+1: Will be sure to give it a cool maritime or astronomy name

    agree on the splitting with 0.14.x and 1.x and stuff
    mhp.
    @miranhpark

    perhaps we should make a repo for it then copy the code over there, then we can keep the 0.14 on the train its on, with another branch (perhaps 1.x) that has these major changes

    just want to +1 that i like this approach too

    Adelbert Chang
    @adelbertc
    Timothy Perrett
    @timperrett
    yeah send your thanks toward @drewgonzales360
    Adelbert Chang
    @adelbertc
    @drewgonzales360 :fire: :fire: :fire:
    Timothy Perrett
    @timperrett
    Oh i’ve had get nelson org added to the Github Actions beta - was thinking i might use actions to trigger the buildkite part to make it auto-release
    then it would just ship itself
    Adelbert Chang
    @adelbertc
    so this fills in that awkward gap we have btwn "i merged to Nelson's branch" and "hey Tim can you make a release pls"
    Timothy Perrett
    @timperrett
    yeahh
    seems sick
    Adelbert Chang
    @adelbertc
    @timperrett did you pre-emptively create this for me https://github.com/getnelson/agent
    Timothy Perrett
    @timperrett
    @adelbertc i did :P
    and a travis-ci job
    Cory Parent
    @goedelsoup
    yooo, so stoked someone finally tackled the TF provider
    Timothy Perrett
    @timperrett
    yeah its going to be really valuable
    Adelbert Chang
    @adelbertc
    i just started learning Terraform too so this is timely :laughing: