RUNC CONTAINERD CRI DOCKERSHIM

$ runc help
NAME:
   runc - Open Container Initiative runtime

runc is a command line client for running applications packaged according to
the Open Container Initiative (OCI) format and is a compliant implementation of the
Open Container Initiative specification.

runc integrates well with existing process supervisors to provide a production
container runtime environment for applications. It can be used with your
existing process monitoring tools and the container will be spawned as a
direct child of the process supervisor.

Containers are configured using bundles. A bundle for a container is a directory
that includes a specification file named "config.json" and a root filesystem.
The root filesystem contains the contents of the container.

To start a new instance of a container:

    # runc run [ -b bundle ] <container-id>

Where "<container-id>" is your name for the instance of the container that you
are starting. The name you provide for the container instance must be unique on
your host. Providing the bundle directory using "-b" is optional. The default
value for "bundle" is the current directory.

USAGE:
   runc [global options] command [command options] [arguments...]

VERSION:
   1.0.2
commit: v1.0.2-0-g52b36a2
spec: 1.0.2-dev
go: go1.16.8
libseccomp: 2.5.1

COMMANDS:
   checkpoint  checkpoint a running container
   create      create a container
   delete      delete any resources held by the container often used with detached container
   events      display container events such as OOM notifications, cpu, memory, and IO usage statistics
   exec        execute new process inside the container
   init        initialize the namespaces and launch the process (do not call it outside of runc)
   kill        kill sends the specified signal (default: SIGTERM) to the container's init process
   list        lists containers started by runc with the given root
   pause       pause suspends all processes inside the container
   ps          ps displays the processes running inside a container
   restore     restore a container from a previous checkpoint
   resume      resumes all processes that have been previously paused
   run         create and run a container
   spec        create a new specification file
   start       executes the user defined process in a created container
   state       output the state of a container
   update      update container resource constraints
   help, h     Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --debug             enable debug output for logging
   --log value         set the log file path where internal debug information is written
   --log-format value  set the format used by logs ('text' (default), or 'json') (default: "text")
   --root value        root directory for storage of container state (this should be located in tmpfs) (default: "/run/user/1000/runc")
   --criu value        path to the criu binary used for checkpoint and restore (default: "criu")
   --systemd-cgroup    enable systemd cgroup support, expects cgroupsPath to be of form "slice:prefix:name" for e.g. "system.slice:runc:434234"
   --rootless value    ignore cgroup permission errors ('true', 'false', or 'auto') (default: "auto")
   --help, -h          show help
   --version, -v       print the version

$ dockerd --help | grep root
      --data-root string                        Root directory of persistent Docker state (default "/var/lib/docker")
      --exec-root string                        Root directory for execution state files (default "/var/run/docker")
      --rootless                                Enable rootless mode; typically used with RootlessKit

$ sudo runc --root /var/run/docker/runtime-runc/moby list
ID                                                                 PID         STATUS      BUNDLE                                                                                                                CREATED                          OWNER
011d6fd316e09fc8a5ff3b226d35b9394cd93c57604c91aa52573559368a822c   940971      running     /run/containerd/io.containerd.runtime.v2.task/moby/011d6fd316e09fc8a5ff3b226d35b9394cd93c57604c91aa52573559368a822c   2021-11-25T04:10:25.216394136Z   root
...

$ sudo runc --root /var/run/containerd/runc/k8s.io list
ID                                                                 PID         STATUS      BUNDLE                                                                                                                  CREATED                          OWNER
14542e5b60446f87af20e200e019a15d1ad1509eb506a2068266b9f98694c704   1191        running     /run/containerd/io.containerd.runtime.v2.task/k8s.io/14542e5b60446f87af20e200e019a15d1ad1509eb506a2068266b9f98694c704   2021-12-08T12:55:21.616268255Z   root
...

$ sudo runc --root /var/run/containerd/runc/k8s.io state 14542e5b60446f87af20e200e019a15d1ad1509eb506a2068266b9f98694c704
{
  "ociVersion": "1.0.2-dev",
  "id": "14542e5b60446f87af20e200e019a15d1ad1509eb506a2068266b9f98694c704",
  "pid": 1191,
  "status": "running",
  "bundle": "/run/containerd/io.containerd.runtime.v2.task/k8s.io/14542e5b60446f87af20e200e019a15d1ad1509eb506a2068266b9f98694c704",
  "rootfs": "/run/containerd/io.containerd.runtime.v2.task/k8s.io/14542e5b60446f87af20e200e019a15d1ad1509eb506a2068266b9f98694c704/rootfs",
  "created": "2021-12-08T12:55:21.616268255Z",
  "annotations": {
    "io.kubernetes.cri.container-name": "kube-flannel",
    "io.kubernetes.cri.container-type": "container",
    "io.kubernetes.cri.image-name": "quay.io/coreos/flannel:v0.15.0",
    "io.kubernetes.cri.sandbox-id": "f0c7bf11fac17f29a8df40f1d937ec35df81e202e42ea8a604e37806aebbb662",
    "io.kubernetes.cri.sandbox-name": "kube-flannel-ds-6xpbj",
    "io.kubernetes.cri.sandbox-namespace": "kube-system"
  },
  "owner": ""
}

$ sudo runc --root /var/run/containerd/runc/k8s.io ps 14542e5b60446f87af20e200e019a15d1ad1509eb506a2068266b9f98694c704
UID         PID   PPID  C STIME TTY          TIME CMD
root       1191    806  0 Dec08 ?        00:00:25 /opt/bin/flanneld --ip-masq --kube-subnet-mgr

$ ctr help
NAME:
   ctr -
        __
  _____/ /______
 / ___/ __/ ___/
/ /__/ /_/ /
\___/\__/_/

containerd CLI


USAGE:
   ctr [global options] command [command options] [arguments...]

VERSION:
   1.4.11

DESCRIPTION:

ctr is an unsupported debug and administrative client for interacting
with the containerd daemon. Because it is unsupported, the commands,
options, and operations are not guaranteed to be backward compatible or
stable from release to release of the containerd project.

COMMANDS:
   plugins, plugin            provides information about containerd plugins
   version                    print the client and server versions
   containers, c, container   manage containers
   content                    manage content
   events, event              display containerd events
   images, image, i           manage images
   leases                     manage leases
   namespaces, namespace, ns  manage namespaces
   pprof                      provide golang pprof outputs for containerd
   run                        run a container
   snapshots, snapshot        manage snapshots
   tasks, t, task             manage tasks
   install                    install a new package
   oci                        OCI tools
   shim                       interact with a shim directly
   help, h                    Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --debug                      enable debug output in logs
   --address value, -a value    address for containerd's GRPC server (default: "/run/containerd/containerd.sock") [$CONTAINERD_ADDRESS]
   --timeout value              total timeout for ctr commands (default: 0s)
   --connect-timeout value      timeout for connecting to containerd (default: 0s)
   --namespace value, -n value  namespace to use with commands (default: "default") [$CONTAINERD_NAMESPACE]
   --help, -h                   show help
   --version, -v                print the version

$ sudo ctr -n k8s.io i ls
REF                                                                                               TYPE                                                      DIGEST                                                                  SIZE      PLATFORMS                                                                                                                          LABELS
docker.io/library/busybox:latest                                                                  application/vnd.docker.distribution.manifest.list.v2+json sha256:e7157b6d7ebbe2cce5eaa8cfe8aa4fa82d173999b9f90a9ec42e57323546c353 758.9 KiB linux/386,linux/amd64,linux/arm/v5,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/mips64le,linux/ppc64le,linux/riscv64,linux/s390x io.cri-containerd.image=managed
docker.io/library/busybox@sha256:e7157b6d7ebbe2cce5eaa8cfe8aa4fa82d173999b9f90a9ec42e57323546c353 application/vnd.docker.distribution.manifest.list.v2+json sha256:e7157b6d7ebbe2cce5eaa8cfe8aa4fa82d173999b9f90a9ec42e57323546c353 758.9 KiB linux/386,linux/amd64,linux/arm/v5,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/mips64le,linux/ppc64le,linux/riscv64,linux/s390x io.cri-containerd.image=managed
k8s.gcr.io/pause:3.2                                                                              application/vnd.docker.distribution.manifest.v2+json      sha256:2a7b365f500c323286ac47e9e32af9bd50ee65de7fe2a27355eb5987c8df9ad8 669.7 KiB linux/amd64                                                                                                                        io.cri-containerd.image=managed
sha256:7138284460ffa3bb6ee087344f5b051468b3f8697e2d1427bac1a20c8d168b14                           application/vnd.docker.distribution.manifest.list.v2+json sha256:e7157b6d7ebbe2cce5eaa8cfe8aa4fa82d173999b9f90a9ec42e57323546c353 758.9 KiB linux/386,linux/amd64,linux/arm/v5,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/mips64le,linux/ppc64le,linux/riscv64,linux/s390x io.cri-containerd.image=managed
sha256:80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c                           application/vnd.docker.distribution.manifest.v2+json      sha256:61e45779fc594fcc1062bb9ed2cf5745b19c7ba70f0c93eceae04ffb5e402269 669.7 KiB linux/amd64                                                                                                                        io.cri-containerd.image=managed

$ sudo crictl image ls
IMAGE                       TAG                 IMAGE ID            SIZE
docker.io/library/busybox   latest              7138284460ffa       1.46MB
k8s.gcr.io/pause            3.2                 80d28bedfe5de       686kB

{
  "metadata": {
    "name": "busybox"
  },
  "image":{
    "image": "busybox"
  },
  "command": [
    "top"
  ],
  "log_path":"busybox.0.log",
  "linux": {
  }
}

{
  "metadata": {
    "name": "nginx-sandbox",
    "namespace": "default",
    "attempt": 1,
    "uid": "hdishd83djaidwnduwk28bcsb"
  },
  "log_directory": "/tmp",
  "linux": {
  }
}

$ sudo crictl run container-config.json pod-config.json
b08ad7b8517d0e37853f3a7211fbc7ba283a7b34cff5bd0ae108e9d956034a24

$ sudo crictl pods
POD ID              CREATED             STATE               NAME                NAMESPACE           ATTEMPT             RUNTIME
91ff0a7d5e81a       15 seconds ago      Ready               nginx-sandbox       default             1                   (default)

$ sudo crictl ps
CONTAINER           IMAGE               CREATED             STATE               NAME                ATTEMPT             POD ID
b08ad7b8517d0       busybox             15 seconds ago      Running             busybox             0                   91ff0a7d5e81a

$ sudo crictl stopp 91ff0a7d5e81a
Stopped sandbox 91ff0a7d5e81a

$ sudo crictl rmp 91ff0a7d5e81a
Removed sandbox 91ff0a7d5e81a

$ dockerd --help | grep containerd
      --containerd string                       containerd grpc address
      --containerd-namespace string             Containerd namespace to use (default "moby")
      --containerd-plugins-namespace string     Containerd namespace to use for plugins (default "plugins.moby")
      --cri-containerd                          start containerd with cri

$ grep cri /etc/containerd/config.toml
disabled_plugins = ["cri"]

$ sudo ctr plugin ls | grep cri

[plugins]
  # ...
  [plugins."io.containerd.grpc.v1.cri"]
    # ...
    sandbox_image = "k8s.gcr.io/pause:3.5"

$ sudo ctr plugin ls | grep cri
io.containerd.grpc.v1           cri                      linux/amd64    ok

$ sudo kubeadm init --cri-socket /run/containerd/containerd.sock --ignore-preflight-errors NumCPU --kubernetes-version v1.22.3
[init] Using Kubernetes version: v1.22.3
[preflight] Running pre-flight checks

<other outputs>

Your Kubernetes control-plane has initialized successfully!

$ sudo kubectl get node -owide --kubeconfig /etc/kubernetes/admin.conf
NAME     STATUS   ROLES                  AGE     VERSION   INTERNAL-IP      EXTERNAL-IP   OS-IMAGE                       KERNEL-VERSION    CONTAINER-RUNTIME
node-1   Ready    control-plane,master   6m55s   v1.22.4   192.168.91.137   <none>        Debian GNU/Linux 10 (buster)   4.19.0-17-amd64   containerd://1.4.8

$ sudo kubectl get no node-1 -ogo-template='{{.status.nodeInfo.containerRuntimeVersion}}' --kubeconfig /etc/kubernetes/admin.conf
containerd://1.4.8

$ systemctl status kubelet.service --no-page --full
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/lib/systemd/system/kubelet.service; disabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: active (running) since Thu 2021-11-25 17:29:02 CST; 21min ago
     Docs: https://kubernetes.io/docs/home/
 Main PID: 38090 (kubelet)
    Tasks: 13 (limit: 2330)
   Memory: 54.7M
   CGroup: /system.slice/kubelet.service
           └─38090 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=/run/containerd/containerd.sock --pod-infra-container-image=k8s.gcr.io/pause:3.5