Hi, I've been playing around with gvisor
on an EKS cluster and have the following setup:
1) An EKS cluster with gvisor
installed on every node.
2) A runtime class with the following definition:
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
name: gvisor
handler: runsc
Using that setup I can create deployments running with gvisor
by specifying the runtime class in the spec like: runtimeClassName: gvisor
.
This all works fine and I am able to get inside the pods and confirm that they are running with gvisor
correctly.
The problem, however, is that if I specify resource limits for the containers, they are never enforced.
They work fine with the default runtime class and when a pod exceeds them it gets terminated, but with gvisor
the pods exceed them and can even crash the whole node.
I've read that all containers should have limits specified and I am running a single container, but the limits still don't get enforced.
Any suggestions?
kubectl top pod
and kubectl describe node/{NODE_WHERE_POD _IS_RUNNING}
say?
@zkoopmanskubectl top pod
:
NAME CPU(cores) MEMORY(bytes)
traffic-5879846794-kgb2m 382m 353Mi
kubectl describe node/...
:
Non-terminated Pods: (4 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age
--------- ---- ------------ ---------- --------------- ------------- ---
dmarinov traffic-5879846794-kgb2m 10m (0%) 10m (0%) 10Mi (0%) 10Mi (0%) 2m6s
kube-system aws-node-btl8z 25m (1%) 0 (0%) 0 (0%) 0 (0%) 2d19h
kube-system kube-proxy-c9nl2 100m (6%) 0 (0%) 0 (0%) 0 (0%) 2d19h
kube-system metrics-server-847dcc659d-mb59d 100m (6%) 0 (0%) 200Mi (17%) 0 (0%) 5m21s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 235m (14%) 10m (0%)
memory 210Mi (18%) 10Mi (0%)
ephemeral-storage 0 (0%) 0 (0%)
hugepages-1Gi 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
attachable-volumes-aws-ebs 0 0
Heya! A bit of a newbie question: In "The true cost of containing" (https://www.usenix.org/system/files/hotcloud19-paper-young.pdf) and the Performance guide (https://gvisor.dev/docs/architecture_guide/performance/) there are references to "internal" or "sandbox-internal" tmpfs filesystems. If I mount a tmpfs through docker's tmpfs option, would that be internal or external?
In that case, it'll be handled like bind mount, will go into the heavy path (sync with kernel through gofer).
Runtimes: runc runsc io.containerd.runc.v2 io.containerd.runtime.v1.linux
daemon.json
is correct. But when I start a container with runtime: runsc
it seems to loop with no log. Any clue ?
runsc do echo "hello world"
show anything at all?
creating container: cannot create sandbox: cannot read client sync file: waiting for sandbox to start: EOF
runsc --debug --debug-log=/tmp/runsc/ do echo "hello world"
doesn't create logs in /tmp/runsc/
?eric@vault:~/hauk $ uname -a
Linux vault 5.15.61-v8+ #1579 SMP PREEMPT Fri Aug 26 11:16:44 BST 2022 aarch64 GNU/Linux
Heya! A bit of a newbie question: In "The true cost of containing" (https://www.usenix.org/system/files/hotcloud19-paper-young.pdf) and the Performance guide (https://gvisor.dev/docs/architecture_guide/performance/) there are references to "internal" or "sandbox-internal" tmpfs filesystems. If I mount a tmpfs through docker's tmpfs option, would that be internal or external?
Good question - I'd like to know this too. Seeing less than ideal tmpfs performance when using /tmp to build Go binaries on an otherwise read-only filesystem