Got error logs in kubelet while running k8s on RancherOS

[niusmallnan]

RancherOS Version: (ros os version)
v1.4.0

Built a k8s cluster by RKE, and got some error logs in kubelet:

E0625 06:32:40.427190    2190 summary.go:102] Failed to get system container stats for "/docker/7fe4e42aee56f227ae165205dd1409936000f6cf06c3e79af13b41f5aed2244b/docker/b04074c807422b5f8efc050b0cba975f20cdacb39a1e28197cfb4e1465190759": failed to get cgroup stats for "/docker/7fe4e42aee56f227ae165205dd1409936000f6cf06c3e79af13b41f5aed2244b/docker/b04074c807422b5f8efc050b0cba975f20cdacb39a1e28197cfb4e1465190759": failed to get container info for "/docker/7fe4e42aee56f227ae165205dd1409936000f6cf06c3e79af13b41f5aed2244b/docker/b04074c807422b5f8efc050b0cba975f20cdacb39a1e28197cfb4e1465190759": unknown container "/docker/7fe4e42aee56f227ae165205dd1409936000f6cf06c3e79af13b41f5aed2244b/docker/b04074c807422b5f8efc050b0cba975f20cdacb39a1e28197cfb4e1465190759"
E0625 06:32:40.427208    2190 summary.go:102] Failed to get system container stats for "/docker/7fe4e42aee56f227ae165205dd1409936000f6cf06c3e79af13b41f5aed2244b": failed to get cgroup stats for "/docker/7fe4e42aee56f227ae165205dd1409936000f6cf06c3e79af13b41f5aed2244b": failed to get container info for "/docker/7fe4e42aee56f227ae165205dd1409936000f6cf06c3e79af13b41f5aed2244b": unknown container "/docker/7fe4e42aee56f227ae165205dd1409936000f6cf06c3e79af13b41f5aed2244b"
E0625 06:32:50.438692    2190 summary.go:102] Failed to get system container stats for "/docker/7fe4e42aee56f227ae165205dd1409936000f6cf06c3e79af13b41f5aed2244b/docker/b04074c807422b5f8efc050b0cba975f20cdacb39a1e28197cfb4e1465190759": failed to get cgroup stats for "/docker/7fe4e42aee56f227ae165205dd1409936000f6cf06c3e79af13b41f5aed2244b/docker/b04074c807422b5f8efc050b0cba975f20cdacb39a1e28197cfb4e1465190759": failed to get container info for "/docker/7fe4e42aee56f227ae165205dd1409936000f6cf06c3e79af13b41f5aed2244b/docker/b04074c807422b5f8efc050b0cba975f20cdacb39a1e28197cfb4e1465190759": unknown container "/docker/7fe4e42aee56f227ae165205dd1409936000f6cf06c3e79af13b41f5aed2244b/docker/b04074c807422b5f8efc050b0cba975f20cdacb39a1e28197cfb4e1465190759"
E0625 06:32:50.438710    2190 summary.go:102] Failed to get system container stats for "/docker/7fe4e42aee56f227ae165205dd1409936000f6cf06c3e79af13b41f5aed2244b": failed to get cgroup stats for "/docker/7fe4e42aee56f227ae165205dd1409936000f6cf06c3e79af13b41f5aed2244b": failed to get container info for "/docker/7fe4e42aee56f227ae165205dd1409936000f6cf06c3e79af13b41f5aed2244b": unknown container "/docker/7fe4e42aee56f227ae165205dd1409936000f6cf06c3e79af13b41f5aed2244b"

Tried the same steps on Ubuntu, cannot see these error logs.

[Jason-ZW]

After mount the systemd cgroup, the error disappeared

mkdir -p /sys/fs/cgroup/systemd
mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd

[niusmallnan]

The cgroup driver has been set to cgroupfs on kubelet, but it still depends on /sys/fs/cgroup/systemd. This is very strange

$ rke -v
rke version v0.1.7
$ curl -sSL http://localhost:8001/api/v1/nodes/172.31.12.239/proxy/configz | jq . |  grep cgroup
    "cgroupsPerQOS": true,
    "cgroupDriver": "cgroupfs",

[rootwuj]

Tested with K8s cluster(deployed by RKE v0.1.7) with RancherOS v1.4.1-rc2.
This version of got the logs without the above error message.