kubesail-agent deployment YAML? I don't see any new info in the docs, https://docs.kubesail.com/byoc/ . thanks for your help!
Hey @jaytula - you'll need to use the image version kubesail/agent:v0.20.1 and add the following section to your env: on the kubesail agent:
- name: NODE_IP
valueFrom:
fieldRef:
fieldPath: status.hostIPYou can copy-paste it from https://byoc.kubesail.com/USERNAME.yaml - and you'll see a warning message in the logs about using HostPort routing :)
@erulabs I kubectl apply -f https://byoc.kubesail.com/USERNAME.yaml and it appears to work better. But I ran into a problem. I tried spinning up another set of deployment/service/ingress with same image (but different name) and it appears the kubesail-agent did not do its magic for it. So I spun up a third deployment but had this subdomain point to my IP address and this worked. The yaml all looks like this:
apiVersion: apps/v1
kind: Deployment
metadata:
name: demo
labels:
app: demo
spec:
selector:
matchLabels:
app: demo
template:
metadata:
labels:
app: demo
spec:
containers:
- name: demo
image: ghcr.io/jaytula/nginx-1:latest
ports:
- containerPort: 80
imagePullSecrets:
- name: regcred
---
apiVersion: v1
kind: Service
metadata:
name: demo
spec:
type: NodePort
selector:
app: demo
ports:
- name: demo
protocol: TCP
port: 8100
targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: demo
annotations:
cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
tls:
- hosts:
- demo.codepasta.io
secretName: demo-tls-secret
rules:
- host: demo.codepasta.io
http:
paths:
- path: /
pathType: ImplementationSpecific
backend:
service:
name: demo
port:
number: 8100The domain demo.codepasta.io points to my ip address. kanboard.codepasta.io has A record that points to kubesail gateway (not working, kubesail invalid cert.). pork.codepasta.io has A record also pointing to kubesail (working).
Hi @erulabs , happy new year! After some weeks of inactivity I re-joined my cluster to kubesail. Looks like a few new glitches have appeared. First, after verifying the cluster , the ui throws this error:
TypeError: Cannot read property 'verified' of undefined
at https://kubesail.com/static/js/14.1c471194.chunk.js:2:958416
at Array.map (<anonymous>)
at r.value (https://kubesail.com/static/js/14.1c471194.chunk.js:2:958202)
at qa (https://kubesail.com/static/js/15.c1ca2479.chunk.js:2:331079)
at za (https://kubesail.com/static/js/15.c1ca2479.chunk.js:2:330872)
at _s (https://kubesail.com/static/js/15.c1ca2479.chunk.js:2:366611)
at gu (https://kubesail.com/static/js/15.c1ca2479.chunk.js:2:358020)
at mu (https://kubesail.com/static/js/15.c1ca2479.chunk.js:2:357945)
at su (https://kubesail.com/static/js/15.c1ca2479.chunk.js:2:354954)
at https://kubesail.com/static/js/15.c1ca2479.chunk.js:2:306325If I refresh the ui the error goes away and the new cluster looks and works fine.
After I install the ingress-nginx and cert-manager via the ui, I try and deploy an nginx app from the template and I also create a service and ingress from the nginx App -> Network tab. Watching the agent log the agent doesn't appear to notice the new ingress. Next I restart the agent and now the agent picks up the ingress, but then there is also a warning in the log that repeats every 10 seconds or so:
(2021-01-12T05:50:36.314Z) info: kubesail-agent starting! { "version": "0.20.1" }
(2021-01-12T05:50:36.351Z) info: Connecting to Kubernetes API...
(2021-01-12T05:50:37.026Z) warn: Unable to determine Ingress Controller! You can install an ingress controller via the KubeSail.com interface!
(2021-01-12T05:50:37.980Z) info: Registering with KubeSail...
(2021-01-12T05:50:38.574Z) info: Connected to gateway socket! { "KUBESAIL_AGENT_GATEWAY_TARGET": "https://usw1.k8g8.com" }
(2021-01-12T05:50:38.645Z) info: Agent recieved new agent-data from gateway! {
"clusterAddress": "pi43x7k3s.dklinkman.usw1.k8g8.com",
"firewall": {
"pi43x7k3s.dklinkman.usw1.k8g8.com": 1
}
}
(2021-01-12T05:50:38.661Z) info: KubeSail Agent registered and ready! KubeSail support information: {
"clusterAddress": "pi43x7k3s.dklinkman.usw1.k8g8.com",
"agentKey": "a0737f613f7f19fd7bac865bb35fd8d7",
"version": "0.20.1"
}
(2021-01-12T05:50:38.983Z) info: Agent recieved new agent-data from gateway! {
"clusterAddress": "pi43x7k3s.dklinkman.usw1.k8g8.com",
"firewall": {
"pi43x7k3s.dklinkman.usw1.k8g8.com": 1,
"nginx.pi43x7k3s.dklinkman.usw1.k8g8.com": "0.0.0.0/0"
}
}
(2021-01-12T05:50:46.303Z) warn: Received request, but no Ingress controller was found! You can install one on KubeSail.com in your cluster settings page. { "host": "nginx.pi43x7k3s.dklinkman.usw1.k8g8.com" }
(2021-01-12T05:50:47.983Z) warn: Received request, but no Ingress controller was found! You can install one on KubeSail.com in your cluster settings page. { "host": "nginx.pi43x7k3s.dklinkman.usw1.k8g8.com" }
(2021-01-12T05:50:56.634Z) warn: Received request, but no Ingress controller was found! You can install one on KubeSail.com in your cluster settings page. { "host": "nginx.pi43x7k3s.dklinkman.usw1.k8g8.com" }
(2021-01-12T05:51:06.931Z) warn: Received request, but no Ingress controller was found! You can install one on KubeSail.com in your cluster settings page. { "host": "nginx.pi43x7k3s.dklinkman.usw1.k8g8.com" }and also the TLS certificate doesn't get finalized because the cert manager is stuck in a http 503 error loop. Restarting the agent again doesn't help.
At this point I downgrade the 0.20.1 agent to 0.19.0 and then everything starts to work normally. The pending cert is finalized and I can access the application via the gateway kgk8.com url. Still with the 0.19.0 agent I can deploy other apps and expose them to the Internet. With new apps the agent picks up the ingress right away and the TLS certificate is completed in about 30 seconds.
Looks like a lot of the error have gone away. I still get this in the agent log right after joining the cluster, but the error goes away once the cert-manager is installed
(2021-01-12T15:07:29.017Z) info: kubesail-agent starting! { "version": "0.21.0" }
(2021-01-12T15:07:29.099Z) info: Connecting to Kubernetes API...
(2021-01-12T15:07:30.657Z) warn: Unable to create ClustuerIssuer, cert-manager is probably still starting up. Retrying in 30 seconds. conversion webhook for cert-manager.io/v1alpha2, Kind=ClusterIssuer failed: Post "https://cert-manager-webhook.cert-manager.svc:443/convert?timeout=30s": service "cert-manager-webhook" not found {
"code": 500,
"stack": "Error: conversion webhook for cert-manager.io/v1alpha2, Kind=ClusterIssuer failed: Post \"https://cert-manager-webhook.cert-manager.svc:443/convert?timeout=30s\": service \"cert-manager-webhook\" not found\n at /home/node/app/node_modules/kubernetes-client/backends/request/client.js:231:25\n at Request._callback (/home/node/app/node_modules/kubernetes-client/backends/request/client.js:168:14)\n at Request.self.callback (/home/node/app/node_modules/request/request.js:185:22)\n at Request.emit (events.js:315:20)\n at Request.<anonymous> (/home/node/app/node_modules/request/request.js:1154:10)\n at Request.emit (events.js:315:20)\n at IncomingMessage.<anonymous> (/home/node/app/node_modules/request/request.js:1076:12)\n at Object.onceWrapper (events.js:421:28)\n at IncomingMessage.emit (events.js:327:22)\n at endReadableNT (internal/streams/readable.js:1327:12)"
}
(2021-01-12T15:07:30.661Z) info: Registering with KubeSail...
(2021-01-12T15:07:31.422Z) info: Connected to gateway socket! { "KUBESAIL_AGENT_GATEWAY_TARGET": "https://usw1.k8g8.com" }
(2021-01-12T15:08:13.521Z) info: Agent recieved new agent-data from gateway! {
"clusterAddress": "rpi43x7k3s.dklinkman.usw1.k8g8.com",
"firewall": {
"rpi43x7k3s.dklinkman.usw1.k8g8.com": 1
}
}
(2021-01-12T15:08:13.533Z) info: KubeSail Agent registered and ready! KubeSail support information: {
"clusterAddress": "rpi43x7k3s.dklinkman.usw1.k8g8.com",
"agentKey": "a5878b6019bb003adde6ef2864d8c397",
"version": "0.21.0"
}
(2021-01-12T15:08:31.606Z) warn: Unable to create ClustuerIssuer, cert-manager is probably still starting up. Retrying in 30 seconds. conversion webhook for cert-manager.io/v1alpha2, Kind=ClusterIssuer failed: Post "https://cert-manager-webhook.cert-manager.svc:443/convert?timeout=30s": service "cert-manager-webhook" not found {
"code": 500,
"stack": "Error: conversion webhook for cert-manager.io/v1alpha2, Kind=ClusterIssuer failed: Post \"https://cert-manager-webhook.cert-manager.svc:443/convert?timeout=30s\": service \"cert-manager-webhook\" not found\n at /home/node/app/node_modules/kubernetes-client/backends/request/client.js:231:25\n at Request._callback (/home/node/app/node_modules/kubernetes-client/backends/request/client.js:168:14)\n at Request.self.callback (/home/node/app/node_modules/request/request.js:185:22)\n at Request.emit (events.js:315:20)\n at Request.<anonymous> (/home/node/app/node_modules/request/request.js:1154:10)\n at Request.emit (events.js:315:20)\n at IncomingMessage.<anonymous> (/home/node/app/node_modules/request/request.js:1076:12)\n at Object.onceWrapper (events.js:421:28)\n at IncomingMessage.emit (events.js:327:22)\n at endReadableNT (internal/streams/readable.js:1327:12)"I still have to restart the agent to pick up a new ingress, but now I have to restart for every new ingress. With 0.19.0 I only had to restart the agent once for the first ingress, and the rest would be picked up automatically
The connection to the server localhost:8080 was refused - did you specify the right host or port?
KUBECONFIG environment variable should point to the configuration file for your kubernetes setup. The configuration file location will vary depending on what kubernetes you are using. The docs should be able to tell you. On my k3s system the file I need is /etc/rancher/k3s/k3s.yaml. Mind the file permissions also. I usually copy that file to ${HOME}/.kube/config. Don't know why--habit I guess.
kubesail/agent:v0.22.0 if you'd like to give that a shot (it's not yet published as stable on the kubesail website). It should resolve all the needing-to-restart issues for ingresses :)
So it seems to be working ok. The ingress is created and the TLS certificate is finalized and the k8g8.com URL works. But there doesn't seem to be any logging now. At least not like before. With 0.19.0 when a new ingress is created the agent would log and new firewall listing with the new host added. Also when the agent was restarted it would log the current firewall list of hosts. but now just:
(2021-01-12T22:11:44.529Z) info: kubesail-agent starting! { "version": "0.22.0" }
(2021-01-12T22:11:46.403Z) info: KubeSail Agent registered and ready! KubeSail support information: {
"clusterAddress": "n40x7k8s.dklinkman.usw1.k8g8.com",
"agentKey": "6e69954ddcd8d642be27b08405e2aabd",
"version": "0.22.0"
}Functionally it seems to be working fine, just the logging looks different
I was planning to try kubesail. But it is showing me this.
kubectl create...). Let me know if you have any questions!
dns.k8g8.com - this is a dynamic DNS address that will point at your machines public IP address! Very useful for hosting games, SSH servers, and anything non-http that the Gateway tunneling service doesn't address for you. Let me know if you have any questions! Thanks everyone!