Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Jean-François Lamy
    @jflamy
    @erulabs this was the brand new agent from earlier today and the ingress yaml were looking fine.
    @erulabs given the magnitude of the ingress changes (new names and new secrets) and my brutal way of updating (delete and recreate) it's not really surprising that a restart was needed
    Seandon Mooy
    @erulabs
    Hehe - yes, I'm not surprised either, but our agent code listens to changes to the ingress system and should react properly - that might have been a little more than it can handle but either way I'd like it to handle it eventually - can be non-obvious that it just needs a restart to pick up the changes. Glad you got it sorted though! Going to add a test for that situation to my list either way <3
    @RyzeNGrind, You mention "without relying on an external website" - can you explain more what you mean there? KubeSail will give you a domain name you can use to communicate with your clusters, but certainly you can point a domain at your cluster yourself as well. Do you mean a custom domain for apps hosted on your cluster? If so, you should be able to configure that on our site as well. Let me know if I'm misunderstanding - would love to help get you setup and working properly!
    1 reply
    jflamy-dev
    @jflamy-dev
    @erulabs am trying to setup things so Lens uses the kubesail config to get to my cluster. When giving Lens the config exported from kubesail, I get
    2021/01/28 17:11:32 http: proxy error: x509: certificate is valid for docker-for-desktop, kubernetes, kubernetes.default, kubernetes.default.svc, kubernetes.default.svc.cluster.local, 
    kubernetes
    .docker.internal, vm.docker.internal, localhost, not owlcms.jflamy.usw1.k8g8.com
    So there is some magic missing - my cluster refuses its proxied identity
    Seandon Mooy
    @erulabs
    Hey @jflamy-dev - I believe you can add insecure-skip-tls-verify: true to your KubeConfig and lens should respect that. The alternative option is to boot your kube cluster in such a way that it has owlcms.jflamy.usw1.k8g8.com as its "advertise address" - the first option should be a lot easier to get going, but is a bit less secure
    EG: edit the kubeconfig like so:
    clusters:
    - cluster:
        server: your-address
        insecure-skip-tls-verify: true
    jflamy-dev
    @jflamy-dev
    Ok, will give it a try.
    Seandon Mooy
    @erulabs
    (I've just added that to the kubeconfig thats provided for BYOC clusters to kubesail, been leaning towards doing that for a while (the docs already mention the commandline version of that flag, but it's a new feature in the config itself))
    In about 10 minutes the /config page will have that pre-filled for you :dancer:
    jflamy-dev
    @jflamy-dev
    (did not work at first attempt; will try after getting dinner started)
    Seandon Mooy
    @erulabs
    okay, I'll do a test as well - I believe support for that was only recently added to Lens, so might need the newest version
    jflamy-dev
    @jflamy-dev
    I have latest.
    jflamy-dev
    @jflamy-dev
    Lens crashes after switching false to true.
    Seandon Mooy
    @erulabs
    Hey @jflamy-dev - I believe you may need to remove the ca-data portion of the kube config - can you try refreshing your browser and using the config from https://kubesail.com/config ?
    (We've removed the ca-data and automatically added the verify: false flag for now)
    jflamy-dev
    @jflamy-dev
    Works. Thks much!
    Seandon Mooy
    @erulabs
    No problemo - as always, your feedback drove an improvement for everyone! :heart: thank you!
    jflamy-dev
    @jflamy-dev
    Next up: k3s running on WSL2. Runs fine normally. Trying to add kubesail-agent yields 0/1 nodes are available: 1 node(s) were unschedulable. Was trying k3s as a dry run prior to tackling Digital Oceab because the memory footprint is much smaller than running a kubernetes cluster on Digital Ocean (they actually suggest running the baby versions for single-node clusters).
    jflamy-dev
    @jflamy-dev
    Nothing special in the logs. Comes with Traefik. Since I cannot connect, can't do the kubesail ingress magic.
    Retch
    @Retch
    Hello, i wanted to add my microk8s, and i get this warning: ClusterRoleBinding is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRoleBinding Is this important?
    jflamy-dev
    @jflamy-dev
    @erulabs regarding the node unschedulable issue on k3s: I am running a single node, just running the k3s server. Maybe there is a taint or something similar that is preventing the agent from running ? My intent is to have something as simple as possible to run on a a small DigitalOcean node. I want to be able to publish a recipe whereby a moderately technical person can open a DO account, install k3s, expose it via Kubesail, and either deploy a template. You know, sort of like the shared clusters (sad face). Cheap is the operative word, the target is penny-pinching amateur sport federations and clubs.
    @erulabs I would try microk8s instead but I was so happy to ditch VirtualBox that I won't reinstall it just for that. WSL2 doesn't support snap.
    Seandon Mooy
    @erulabs
    @Retch that warning can be ignored - yoyre just using a very shiney new version of Kubernetes and its warning about some upcoming changes for us to deal with. I did see some errors last night from your agent registration process tho - did you manage to get things working?
    @jflamy-dev hrmm - you might try “wsl —shutdown” and then restarting k3s - i typically recommend docker-desktop on windows for now though. K3s on windows is pretty experimental currently
    DavidCamelo
    @DavidCamelo
    hi @erulabs my cluster went down due to an OOM problem, I killed all the pods from the remote console but my node has not yet recovered.
    image.png
    image.png
    Seandon Mooy
    @erulabs
    Hey @DavidCamelo - Just me just a few moments and I'll look into that for you. Looks like a java process is the culprit :(
    DavidCamelo
    @DavidCamelo
    Yes, I increased the memory usage of my java deployment and ran a job on it and that caused the memory error.
    My fault :(
    Seandon Mooy
    @erulabs
    Ah yes, looks like around 1pm the memory usage spiked up a bit. No worries, we're here to help :)
    DavidCamelo
    @DavidCamelo
    Thanks!
    Seandon Mooy
    @erulabs
    Hey @DavidCamelo - should be back online now - I see the agent is connected. Let me know if you spot any other issues :heart:
    By the way, if we dont reply quickly here, feel free to email support@kubesail.com - we monitor that more closely than this chat
    DavidCamelo
    @DavidCamelo
    Thank you very much, I see that everything is up
    i will decrease the memory usage of my java deployment
    jflamy-dev
    @jflamy-dev
    @erulabs trying to run microk8s on a digital ocean ubuntu. I added the agent, but the connection somehow broke. I currently get
    root@owlcms-tor1-01:~# microk8s kubectl logs  kubesail-agent-758c4485f7-4rp7j -n kubesail-agent
    (2021-01-29T22:52:03.384Z) info: kubesail-agent starting! { "version": "0.23.3" }
    (2021-01-29T22:52:03.786Z) warn: We'll route traffic to this Node's HostPort 80 & 443, because it appears you have a hostPort nginx ingress controller enabled!
    (2021-01-29T22:52:04.198Z) error: KubeSail agentKey and agentSecret rejected! Please re-install this agent at https://kubesail.com/clusters { "status": 404 }
    I had deleted the cluster, expecting that if the agent came knocking back it would reset things and re-register the cluster.
    How does one "reinstall the agent". If I re-run the yaml, I get all sorts of errors about secrets being already there and suchlike.
    Seandon Mooy
    @erulabs
    Ah, if you "force" delete the cluster from the UI, it does break the agent a bit. Go ahead and do a kubectl delete namespace kubesail-agent and then try re-adding it.
    Ideally the UI tells you to delete the agent installation first, but we've made it easier to "force delete" now
    jflamy-dev
    @jflamy-dev
    That did it. Now tackling the connection to my custom domain.
    jflamy-dev
    @jflamy-dev
    @erulabs gave up on microk8s on a 2GB Digital Ocean basic droplet. k3s seems less invasive. I installed k3s without traefik, and then used the kubesail interface to install nginx and cert-manager.
    I get some strange 404 errors after adding my two ingresses (I have two webapps in the cluster).
    pod.go:58] cert-manager/controller/challenges/http01/selfCheck/http01/ensurePod "msg"="found one existing HTTP01 solver pod" "dnsName"="officials.jflamy.dev" "related_resource_kind"="Pod" "related_resource_name"="cm-acme-http-solver-6j7r9" "related_resource_namespace"="default" "resource_kind"="Challenge" "resource_name"="owlcms-ingress-1581768128-2354231461-1002489940" "resource_namespace"="default" "type"="http-01" 
    service.go:43] cert-manager/controller/challenges/http01/selfCheck/http01/ensureService "msg"="found one existing HTTP01 solver Service for challenge resource" "dnsName"="officials.jflamy.dev" "related_resource_kind"="Service" "related_resource_name"="cm-acme-http-solver-57tw9" "related_resource_namespace"="default" "resource_kind"="Challenge" "resource_name"="owlcms-ingress-1581768128-2354231461-1002489940" "resource_namespace"="default" "type"="http-01" 
    ingress.go:91] cert-manager/controller/challenges/http01/selfCheck/http01/ensureIngress "msg"="found one existing HTTP01 solver ingress" "dnsName"="officials.jflamy.dev" "related_resource_kind"="Ingress" "related_resource_name"="cm-acme-http-solver-sm8gv" "related_resource_namespace"="default" "resource_kind"="Challenge" "resource_name"="owlcms-ingress-1581768128-2354231461-1002489940" "resource_namespace"="default" "type"="http-01" 
    sync.go:185] cert-manager/controller/challenges "msg"="propagation check failed" "error"="wrong status code '404', expected '200'" "dnsName"="officials.jflamy.dev" "resource_kind"="Challenge" "resource_name"="owlcms-ingress-1581768128-2354231461-1002489940" "resource_namespace"="default" "type"="http-01"
    Jean-François Lamy
    @jflamy
    @erulabs after fixing the cname which was broken and not pointing to the correct cluster, I now get 503 errors instead of 404 when a challenge is made.
    Seandon Mooy
    @erulabs
    Ack :/ I wonder if restarting the agent again does the trick... If you can browse to the domain and get your app (and not a 503) thats a good indication its not the agent. If you get a 503 then the cert will not be able to work. The certs use the same domain name as your app but with a /.well-known sort of URL path.
    If restarting the agent -does- work, thats a sort of urgent issue I'll try to fix asap... We actually have a couple patches in the pipeline for the agent that might improve that.
    Carl J. Mosca
    @carljmosca
    how can we clean up what appear to be “stale” domains? I have created/deleted/recreated (with a new name) a cluster and noticed that the old dynamic domain is hanging around - is there a way for me to delete it?
    specifically: k3s1.carljmosca.dns.k8g8.com