Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • May 24 2017 15:42

    plombardi89 on experimental

    Update module config (compare)

  • May 24 2017 15:30

    plombardi89 on experimental

    Update RDS config sample (compare)

  • May 23 2017 21:06
    plombardi89 commented #24
  • May 23 2017 21:05
    plombardi89 commented #24
  • May 23 2017 20:59
    plombardi89 commented #24
  • May 23 2017 20:56
    plombardi89 opened #24
  • May 23 2017 14:59
    plombardi89 commented #23
  • May 23 2017 14:56
    plombardi89 commented #23
  • May 23 2017 14:55
    plombardi89 opened #23
  • May 23 2017 14:51
    plombardi89 commented #17
  • May 23 2017 14:45
    plombardi89 opened #22
  • May 22 2017 22:11

    plombardi89 on experimental

    VPC peering with proper securit… (compare)

  • May 22 2017 20:05

    plombardi89 on experimental

    DO NOT associate the kube secur… (compare)

  • May 19 2017 21:55

    plombardi89 on experimental

    Programmatically generate corre… (compare)

  • May 19 2017 21:43

    plombardi89 on experimental

    Wrong Terraform variable (compare)

  • May 19 2017 21:18

    plombardi89 on experimental

    set source_security_group_id (compare)

  • May 19 2017 21:03

    plombardi89 on experimental

    wrong variable (compare)

  • May 19 2017 20:57

    plombardi89 on experimental

    More stupid. (compare)

  • May 19 2017 20:55

    plombardi89 on experimental

    Cleanup stupidity (compare)

  • May 19 2017 20:49

    plombardi89 on experimental

    Use explicit counts because Ter… (compare)

Abhay Saxena
@ark3
If I want to re-spin my cluster, should I delete the fabric, or just the cluster?
Philip Lombardi
@plombardi89
Deleting the cluster would be more appropriate (and faster). If you delete the fabric then you torch everything. When we move to a model where Loom can provision databases and other backing resources behind the scenes then deleting the fabric would be inappropriate unless you really wanted to torch all that backing infrastructure
Flynn
@kflynn
So is simple-v2 the model I should use to recreate my cluster?
Philip Lombardi
@plombardi89
Yes, at Datawire it is at least ;)
Flynn
@kflynn
OK. So I POST: curl -v -X POST -H "Content-Type: application/json" -d '{"name": "flynn-k8s", "model": "simple-v2"}' localhost:7000/fabrics and after awhile I see Loom’s log say “Cluster is starting. It sholud be ready in a few minutes."
That was a few minutes ago. ;)

At this point if I do

curl --output ~/.kube/config.d/flynn-k8s localhost:7000/fabrics/flynn-k8s

I get some stuff, but the stuff I get back has no credentials.

Philip Lombardi
@plombardi89
curl --output ~/.kube/config.d/flynn-k8s localhost:7000/fabrics/flynn-k8s/cluster/config
The status of the cluster formation can be polled as well curl localhost:7000/fabrics/:name/cluster
Flynn
@kflynn
Oh FFS. Didn’t select the whole line when copying, I guess. :P Thanks!
Philip Lombardi
@plombardi89
A Loom CLI that doesn't require remembering the CURL commands is a long-term wishlist item
Flynn
@kflynn
Yup.
Flynn
@kflynn
OK. All’s well here. loom:latest Did The Right Thing For Me™.
Enrique Carbonell
@kikicarbonell
Hi...just I'm discovering loom tool but what is the main difference with others automation tools like Terraform?
I note the focus on Kubernetes but other tools work with recipies to deploy any on any hypervisor or cloud provider.
Philip Lombardi
@plombardi89

Hi @kikicarbonell

Loom is designed to be a full Kubernetes fabric management tool for ops and developers. This means that Loom is being designed to provision not only Kubernetes clusters but other infrastructure-level tech (what we call "fabric") such as backing databases, queue servers etc. for applications and services. However, what really makes Loom different from using raw Terraform, or any other infrastructure automation tool is that it is being designed to expose a very simple HTTP REST API for developers that they can script around. The operational theory behind Loom is that an Ops Engineer defines "models" which set a lot of low level parameters for how infrastructure should be provisioned and then developers use the REST API to provision on-demand what they need in their fabrics. This keeps control in the Ops engineers hands while reducing friction for developers because it avoids forcing them to learn several different automation tools such as but not limited to Terraform, Kops, AWS CLI, Ansible etc.

Does the answer your question? Please let me know if I can help you in any way.

Enrique Carbonell
@kikicarbonell
ohh yes you are answered my question.
@plombardi89 I was reviewing the Terraform because I thinks that it support API rest...but I understand your explanation. thks
Loom is designed to DevOps movement...
jeje
@plombardi89 are you interested to present the tools and your ideas on DevOpsDays events?
Philip Lombardi
@plombardi89

@kikicarbonell Terraform does not have a REST API in the public open source version. Hashicorp's private paid offering, however, does expose some kind of API. I think of Terraform as an important tool, but it is a tool that exists at a lower level than what Loom is trying to accomplish.

I'd be happy to present, could you provide me some more information?

Philip Lombardi
@plombardi89
Much nicer API validation incoming soon:
plombardi@palwork ~> curl -X POST -H "Accept: application/json" -d '{"name": "80foo", "region": "us-east-1"}' localhost:7000/api/models
[ {
  "title" : "Field Does Not Match Regex Pattern",
  "description" : "JSON value at '/name' did not match expected pattern /[a-z][a-z0-9_]{0,31}/",
  "path" : "/name"
}, {
  "title" : "Invalid Cloud Region",
  "description" : "Cloud provider region 'us-east-1' is not valid or usable",
  "path" : "/region"
}, {
  "title" : "Required Field Is Null or Absent",
  "description" : "JSON value at '/masterType' is required to be non-null but the provided value was null.",
  "path" : "/masterType"
} ]
It should really help people understand what is wrong with their requests
Abhay Saxena
@ark3
Sweet.
Philip Lombardi
@plombardi89
Need to figure out the easiest and quickest path to get Loom running for folks as a service inside of AWS - thinking about doing it as an Elastic Beanstalk App: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_image.html
Richard Li
@richarddli
why not just create a CloudFormation template?
(only reason I suggest that is when Amazon released their tracing stuff, that's what they did)
@plombardi89
Philip Lombardi
@plombardi89
@richarddli That's the plan. Cfn template to create an EBS setup running Loom. Nice and simple.
Philip Lombardi
@plombardi89
Hi folks! Just tagged and release 0.1.1 which is a tiny update to include configurable VPC CIDR blocks. More coming soon!
Flynn
@kflynn
W00t! :)
omnilinguist
@omnilinguist
hey guys, what's the approximately estimated timeframe for Loom to reach general availability (says it's currently in Alpha mode)?
Richard Li
@richarddli
hey there, we're fairly conservative on how we label our software. it's actively being used right now by a couple different orgs (and we're dogfooding it regularly internally). so for us to get to GA is mostly having it run in prod usage for awhile longer, a lot of docs, installer, etc. so i would guess it's a month or two, but it definitely works right now.
omnilinguist
@omnilinguist
what about working at scale? :) how big are the k8s clusters of the largest users?
btw @richarddli certainly wasn't expecting weekend responses, thanks for your time! I'm going to be putting a fair amount of thought on this scoping/planning later today (i guess closer to evening eastern time) as well as going forward, may ask some more questions here or in the ambassador room; again, setting low expectations on turnaround
Richard Li
@richarddli
no worries, i'm in and out, but tend to do some work on sunday mornings, lest i forget :smile: . the biggest k8s clusters are not that big at the moment (i don't know offhand) but I wouldn't really look at Loom as a set of best practices for large-scale k8s clusters ... I'd look at Loom more as "REST API for developers to do self-service provisioning" and it's not opinionated about how the cluster it deploys is configured
Philip Lombardi
@plombardi89

API stability is another big aspect of transition out of Tech Preview / Alpha to something more stable. Getting the API "right" is important because it's the main front end for developers and will likely be scripted around by folks so it cannot just break often once we decide to lock it down.

My primary focus right now is reworking the core of Loom so that it's easier to reason about and test. I wrote the initial very quickly as a prototype and to validate the idea. We're beyond that initial validation point now and into the "turn this into something usable" phase of development. That work is happening in the experimental branch right now.

Beyond that it's a combination of:

  • API stability and general program robustness improvement via usage and feedback
  • Adding some often asked features on for resource provisioning (e.g. some folks have asked for the ability to start RDS DB server and associate it with a cluster automatically)
  • Upgrade path. I want Loom to be easy to upgrade because my experiences with delivering on-prem software is that if you make it hard to upgrade or do not provide an story around it then nobody ever upgrades.
Cluster size is an interesting question. The biggest hypothetical clusters on AWS using default "kubenet" networking is about ~50 nodes due to the way routing tables and security groups work in AWS.

We could go bigger by doing a couple things:

  1. Make federation easier (run many <X> node clusters and represent them as a single one from the schedulers perspective)
  2. Allow folks to use overlay networking which circumvents the AWS networking limits

Most likely we will do both at some point. I want to understand the overlay networking stuff better before I just start letting folks spin clusters up with it because it's inevitable any issues that crop up will come back to us first :)

Philip Lombardi
@plombardi89
There's also a large architectural question looming about state storage in Loom which I need to answer soon. Right now Loom uses S3 for persistence... and it has nice properties for ease-of-setup, but its eventual consistency model and lack of fine grained query capabilities has caused some issues already that have me thinking about switching to DynamoDB (and using consistent reads) or PostgreSQL.
Philip Lombardi
@plombardi89
Oh and an auth story of some sort
plombardi89 @plombardi89 takes note to write up a TODO plan of some sort
omnilinguist
@omnilinguist
hey guys, coming back to this after a bit of a hiatus (got sidetracked by some other stuff). it looks like the loom documentation is very sparse; how can the k8s cluster that loom creates be scaled up/down, monitored, maintained, etc?
Philip Lombardi
@plombardi89
@omnilinguist Are you using Loom currently or evaluating?
omnilinguist
@omnilinguist
still evaluating, the lack of documentation is a bit concerning
Philip Lombardi
@plombardi89

@omnilinguist Understandable concern about the docs. Truthfully, Loom is in a bit of a limbo state at the moment. We received some great feedback from customers about the tool, but one of the biggest issues they expressed was that Loom is too much of a black box.

We still believe this problem needs to be solved but we're taking some time right now to rethink and redesign. If you have feedback for us about how you would want a tool like this to work we would love to get your thoughts or if you have notes from your research of other tools and what you did or did not like about them then that would be super interesting to read.