Enable gVisor based container sandboxing in a Tanzu Kubernetes Grid Cluster
Before i talk about the procedure to enable gVisor in a TKG cluster worker nodes, let us understand what is container sandboxing.
Traditional Linux containers are not sandboxes, What does that mean?
So, this means that the containers running on a worker nodes can directly make a call to linux kernel.
Sandboxed containers with gVisor
So here, container running application are making system calls to gVisor which is a linux kernel running on userspace.
There is a very good article that i found, go through once to understand more.
Steps to enable gVisor sandboxing in a TKG cluster worker node
- By default, TKG cluster uses a containerd container runtime and that internally uses runc as default runtime. You can validate by looking at containerd configuration file located on a node.
$ cat cat /etc/containerd/config.toml
Now, we have to configure containerd to use “containerd-shim-runsc-v1”, but before we really configure, we need to install “runsc” and “containerd-shim-runsc-v1”.
Run the following commands on one of the worker node
$ set -e
$ ARCH=$(uname -m)
$ URL=https://storage.googleapis.com/gvisor/releases/release/latest/${ARCH}
$ wget ${URL}/runsc ${URL}/runsc.sha512 \
${URL}/containerd-shim-runsc-v1 ${URL}/containerd-shim-runsc-v1.sha512
sha512sum -c runsc.sha512 \
-c containerd-shim-runsc-v1.sha512
$ rm -f *.sha512
$ chmod a+rx runsc containerd-shim-runsc-v1
$ sudo mv runsc containerd-shim-runsc-v1 /usr/local/bin
Now, runsc is installed on a worker node. Verify this by running the below command and it will give you a help options
$ runsc --help
2. Now, lets update containerd configuration file to use “runsc” runtime.
Update /etc/containerd/config.toml
$ cat <<EOF | sudo tee /etc/containerd/config.toml
version = 2
[plugins."io.containerd.runtime.v1.linux"]
shim_debug = true
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
runtime_type = "io.containerd.runc.v2"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc]
runtime_type = "io.containerd.runsc.v1"
EOF
3. Restart containerd
$ sudo systemctl restart containerd
4. Lets come back to Kubernetes side now. Come out from worker node and now we will be creating some Kubernetes objects.
5. Create a RuntimeClass, Save the below yaml file e.g. gvisor.yml
apiVersion: node.k8s.io/v1beta1
kind: RuntimeClass
metadata:
name: gvisor
handler: runsc
6. Lets apply this using kubectl, This is not a namespace scoped resource, so no need to specify the namespace.
$ kubectl create -f gvisor.yml
runtimeclass.node.k8s.io/gvisor created$ k get runtimeclass
NAME HANDLER AGE
gvisor runsc 5s
7. Now, its time to deploy a pod with newly created runtime class and also since we have installed runsc on a one worker node, so we will specify the node name too. save the file e.g. pod.yml
apiVersion: v1
kind: Pod
metadata:
name: nginx-gvisor
spec:
nodeName: <nodename here>
runtimeClassName: gvisor
containers:
- name: nginx
image: nginx
8. Lets create the pod
$ k create -f pod.yml
9. in default namespace, you will see the pod running now.
$ k get pods
NAME READY STATUS RESTARTS AGE
nginx-gvisor 1/1 Running 0 47m
10. You can validate on worker node too and will find that there are containers using runsc
And we are done here:)…
Also i would like to mention that, this is just my test and i was curious to see if this works with TKG or not.. You should check with VMware before doing any of this in production environment.
You can find some useful resources here
gVisor quick start
Install runsc