So long dockershim

Hey as you know the latest versions of kubernetes have gotten rid of dockershim, so for example in GKE docker is not the default runtime anymore … which is a bit of a pain in a way cause is kind of handy to ssh into gke node and:

docker exec --id 0 --privileged -it container bash

Sometimes when contianers are run unprivielged and as non-root inside kubernetes , for example when you need to strace tcpdump etc.

Tools inside GKE nodes

When you ssh into a node there’s a bunch of tools that will let you exec into a containerd container eg:

  • ctr
  • crictl
  • nsenter

crictl

I think using crictl ps is kind of handy to see what runnning inside your node , obvs you can also use ps and that makes it clear enough but:

$ crictl ps
CONTAINER           IMAGE               CREATED             STATE               NAME                                        ATTEMPT             POD ID
e997360c64249       43c27288df5fd       4 days ago          Running             fluentbit-gke                               0                   234ac625fdd0e
843a9a99e825a       1f080b7e64a89       4 days ago          Running             gce-pd-driver                               0                   d76d77945d683
ba81ad7c3e701       26e198c160890       4 days ago          Running             gke-metrics-agent                           0                   3c313e66b0c60
9f45cb147a5c0       5dbad453ee5df       4 days ago          Running             fluentbit                                   0                   234ac625fdd0e
b67043ddccd8e       ea55835fd1e56       4 days ago          Running             kube-proxy                                  0                   2272003e214d3

You can also exec and other things with crictl … but as far as i know you can’t run –privileged or –id which is a bit of a dissapointment

ctr

I like ctr it kind of has more control over everthing , but Disclaimer Containerd has its own namespaces so don’t get confused with kubernetes namespaces to list them you can run:

$ ctr ns ls
NAME   LABELS
k8s.io
moby

ctr uses a unix socket to communicate to containerd that by default works without you doing much but if for whatever reason you need to specify it:

$> netstat --unix -l | grep containerd.sock
unix  2      [ ACC ]     STREAM     LISTENING     26227    /run/containerd/containerd.sock.ttrpc
unix  2      [ ACC ]     STREAM     LISTENING     26229    /run/containerd/containerd.sock

and then you can simply call it like:

$> ctr -a /run/containerd/containerd.sock ns ls
NAME   LABELS
k8s.io
moby

Now k8s.io is obviously the gke namespace for containerd , to list all the containers you could do

$> ctr --namespace k8s.io c ls
CONTAINER                                                           IMAGE                                                                                    RUNTIME
06703b498c93acbeaa9033a93d76ac0c6fbca1be72103e35cea1d9436b11bbd9    sha256:80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c                  io.containerd.runc.v2
2176efe0b0b7741396898d3f0d6105c17dac3dd2fe804c7d7e5f6094d1ea0086    sha256:80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c                  io.containerd.runc.v2
2272003e214d3f0780c71ad70dec6cf675d7d74d42d3efc12e2759cd1018d8fb    sha256:80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c                  io.containerd.runc.v2
234ac625fdd0e5a80ceb46e7dab628d7a3f68765c481d1a78cc7f260702b1db9    sha256:80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c                  io.containerd.runc.v2
2e038a9b95a52503fe694c6d15f1c7cf9559c653566d3b9d221929486c1d2863    sha256:8ee6ce05080eca1cdf1cff0bd9216e66cba03b674dd3735718ebbd45d99075b0                  io.containerd.runc.v2
3c313e66b0c6021c6cb2504edf7a71528bf264a7cea36c1f2f38d07a68b92bd6    sha256:80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c                  io.containerd.runc.v2
843a9a99e825a1b8a7a27364fa0285c8893428ec0bfb68d131a4f7f8cc0fd416    gke.gcr.io/gcp-compute-persistent-disk-csi-driver:v1.2.4-gke.0                           io.containerd.runc.v2
9f45cb147a5c0d2b88bbbb9cd60cf819b88e657b4db58fb338cd47b58cf92c2d    sha256:5dbad453ee5df1ffc945e2a7cb31cd62217f1486ee5eea3436a478566148e75c                  io.containerd.runc.v2
aa6e25d2862888a32bedb556bb03d532d95039eb0749e279fa034cbff9d55bc8    sha256:265255432d759ef8436a0a0011a1a578c9d4b57855ad903a1928314f698571b0                  io.containerd.runc.v2
b67043ddccd8eaefe4813c2b5421118c86555bdaf2d8e7399d251bb004a8a6e9    gke.gcr.io/kube-proxy-amd64:v1.20.11-gke.1300                                            io.containerd.runc.v2
ba81ad7c3e701891f4825274c2102b879a26caf8247e960e2e5c633a024bef96    sha256:26e198c160890aedde5129d267bd4a8df0a72a56df212d9411c54cfb3765c18f                  io.containerd.runc.v2
c64db11edc065e522b745092608ee2e281594cc01a6c24470e8f9e970095bb74    sha256:80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c                  io.containerd.runc.v2
d35741ea9541640169b0c4e0f52019552fcb456d9bbe4d9879a427c8f1b9720c    sha256:80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c                  io.containerd.runc.v2
d76d77945d6833d3bb5c0a907bd1f81aa869cf91463ccf1e1cdaac266b3c7b8a    sha256:80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c                  io.containerd.runc.v2
e997360c64249c3582125b41e521224fdb6a46f68f6041034c811078026b0edc    gke.gcr.io/fluent-bit-gke-exporter:v0.16.2-gke.0                                         io.containerd.runc.v2

Keep in mind that k8s.io is not the default ns so if you don’t pass –namespace you get nothing:

$> ctr c ls
CONTAINER    IMAGE    RUNTIME
$>

Let’s exec with ctr

It’s quite simple and i don’t want to bore you with the details:

$> ctr --namespace k8s.io task exec -t --exec-id demo b36015a17bf0b3b9ff16f06109c5d0c0b1e83dc74241039fd278896973180f62  bash
root@gateway-7f8c9c6f78-kqtfk:/app# uname -a
Linux gateway-7f8c9c6f78-kqtfk 5.4.144+ #1 SMP Sat Sep 25 09:56:01 PDT 2021 x86_64 x86_64 x86_64 GNU/Linux

So basicaly is task exec instead of just exec , and it takes an –exec-id which is the id of the task

Let’s exec with crictl

Same story with crictl very simple to do and very similar to docker exec

$> crictl exec -it b36015a17bf0b bash
root@gateway-7f8c9c6f78-kqtfk:/app# uname -a
Linux gateway-7f8c9c6f78-kqtfk 5.4.144+ #1 SMP Sat Sep 25 09:56:01 PDT 2021 x86_64 x86_64 x86_64 GNU/Linux

Right that was all hope this comes handy