stat /var/lib/calico/nodename: no such file or directory:

[Liu-fu-gui]

环境华数物理机 + pve + ubt22.04

`apiVersion: kubekey.kubesphere.io/v1alpha2
kind: Cluster
metadata:
name: sample
spec:
hosts:

• {name: master01, address: 10.100.23.51, internalAddress: 10.100.23.51, user: root, password: "root"}
  roleGroups:
  etcd:

  • master01
    control-plane:
  • master01
    worker:
  • master01
    controlPlaneEndpoint:

  Internal loadbalancer for apiservers

  internalLoadbalancer: haproxy

  domain: lb.kubesphere.local
  address: ""
  port: 6443
  kubernetes:
  version: v1.32.0
  clusterName: cluster.local
  autoRenewCerts: true
  containerManager: containerd
  etcd:
  type: kubekey
  network:
  plugin: calico
  kubePodsCIDR: 172.16.0.0/13
  kubeServiceCIDR: 172.24.0.0/13

  multus support. https://github.com/k8snetworkplumbingwg/multus-cni

  multusCNI:
  enabled: false
  registry:
  privateRegistry: ""
  namespaceOverride: ""
  registryMirrors: []
  insecureRegistries: []
  addons: []

`

报错信息：

Mar 10 11:23:20 master01 kubelet[30360]: E0310 11:23:20.101790 30360 log.go:32] "StopPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to destroy network for sandbox "e0f6665744e6f48928559cbfb3662c9d21a009331801da410c8152b2d37cdd0e": plugin type="calico" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" podSandboxID="e0f6665744e6f48928559cbfb3662c9d21a009331801da410c8152b2d37cdd0e"

Mar 10 11:23:20 master01 kubelet[30360]: E0310 11:23:20.101887 30360 kuberuntime_manager.go:1479] "Failed to stop sandbox" podSandboxID={"Type":"containerd","ID":"e0f6665744e6f48928559cbfb3662c9d21a009331801da410c8152b2d37cdd0e"}

Mar 10 11:23:20 master01 kubelet[30360]: E0310 11:23:20.101939 30360 kuberuntime_manager.go:1079] "killPodWithSyncResult failed" err="failed to "KillPodSandbox" for "96eaf81f-6e4f-4b5a-aacd-bfd065845a6f" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"e0f6665744e6f48928559cbfb3662c9d21a009331801da410c8152b2d37cdd0e\": plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/""

Mar 10 11:23:20 master01 kubelet[30360]: E0310 11:23:20.101973 30360 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to "KillPodSandbox" for "96eaf81f-6e4f-4b5a-aacd-bfd065845a6f" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"e0f6665744e6f48928559cbfb3662c9d21a009331801da410c8152b2d37cdd0e\": plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/"" pod="kube-system/coredns-576f59f987-nvstr" podUID="96eaf81f-6e4f-4b5a-aacd-bfd065845a6f"

Mar 10 11:23:20 master01 kubelet[30360]: I0310 11:23:20.369207 30360 scope.go:117] "RemoveContainer" containerID="eb96abc8f724a699b33d98fd36ac0d26f9d1a8496f45b7197693e0029fa0d1ab"

Mar 10 11:23:21 master01 kubelet[30360]: I0310 11:23:21.374692 30360 scope.go:117] "RemoveContainer" containerID="eb96abc8f724a699b33d98fd36ac0d26f9d1a8496f45b7197693e0029fa0d1ab"

Mar 10 11:23:21 master01 kubelet[30360]: I0310 11:23:21.375222 30360 scope.go:117] "RemoveContainer" containerID="fc52f2ad6f4d7d614edfb87bc0f4b37fadc4263137de057f04c3435583d58cff"

Mar 10 11:23:21 master01 kubelet[30360]: E0310 11:23:21.375453 30360 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to "StartContainer" for "calico-node" with CrashLoopBackOff: "back-off 10s restarting failed container=calico-node pod=calico-node-fp44f_kube-system(45208a7b-4b49-4eec-a09a-f14f31087119)"" pod="kube-system/calico-node-fp44f" podUID="45208a7b-4b49-4eec-a09a-f14f31087119"

Mar 10 11:23:25 master01 kubelet[30360]: E0310 11:23:25.118604 30360 log.go:32] "StopPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to destroy network for sandbox "e7ca2caab400695302490760c6f4a69348788886f7330240da1ea7e567563d9a": plugin type="calico" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" podSandboxID="e7ca2caab400695302490760c6f4a69348788886f7330240da1ea7e567563d9a"

Mar 10 11:23:25 master01 kubelet[30360]: E0310 11:23:25.118689 30360 kuberuntime_manager.go:1479] "Failed to stop sandbox" podSandboxID={"Type":"containerd","ID":"e7ca2caab400695302490760c6f4a69348788886f7330240da1ea7e567563d9a"}

Mar 10 11:23:25 master01 kubelet[30360]: E0310 11:23:25.118741 30360 kuberuntime_manager.go:1079] "killPodWithSyncResult failed" err="failed to "KillPodSandbox" for "df5850ac-71ea-4222-a991-e6f83020cac6" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"e7ca2caab400695302490760c6f4a69348788886f7330240da1ea7e567563d9a\": plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/""

Mar 10 11:23:25 master01 kubelet[30360]: E0310 11:23:25.118774 30360 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to "KillPodSandbox" for "df5850ac-71ea-4222-a991-e6f83020cac6" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"e7ca2caab400695302490760c6f4a69348788886f7330240da1ea7e567563d9a\": plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/"" pod="kube-system/calico-kube-controllers-75c587cc8d-jf892" podUID="df5850ac-71ea-4222-a991-e6f83020cac6"

Mar 10 11:23:26 master01 kubelet[30360]: I0310 11:23:26.740841 30360 scope.go:117] "RemoveContainer" containerID="fc52f2ad6f4d7d614edfb87bc0f4b37fadc4263137de057f04c3435583d58cff"

Mar 10 11:23:26 master01 kubelet[30360]: E0310 11:23:26.741035 30360 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to "StartContainer" for "calico-node" with CrashLoopBackOff: "back-off 10s restarting failed container=calico-node pod=calico-node-fp44f_kube-system(45208a7b-4b49-4eec-a09a-f14f31087119)"" pod="kube-system/calico-node-fp44f" podUID="45208a7b-4b49-4eec-a09a-f14f31087119"

Mar 10 11:23:27 master01 kubelet[30360]: E0310 11:23:27.099796 30360 log.go:32] "StopPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to destroy network for sandbox "27ebb31735da264f97c20058195ae9e6c36af5f07922660b3b1f7cfd499d69ed": plugin type="calico" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" podSandboxID="27ebb31735da264f97c20058195ae9e6c36af5f07922660b3b1f7cfd499d69ed"

Mar 10 11:23:27 master01 kubelet[30360]: E0310 11:23:27.099869 30360 kuberuntime_manager.go:1479] "Failed to stop sandbox" podSandboxID={"Type":"containerd","ID":"27ebb31735da264f97c20058195ae9e6c36af5f07922660b3b1f7cfd499d69ed"}

Mar 10 11:23:27 master01 kubelet[30360]: E0310 11:23:27.099921 30360 kuberuntime_manager.go:1079] "killPodWithSyncResult failed" err="failed to "KillPodSandbox" for "1275b4d6-e67a-463c-ba5d-cb0ebf64ce19" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"27ebb31735da264f97c20058195ae9e6c36af5f07922660b3b1f7cfd499d69ed\": plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/""

Mar 10 11:23:27 master01 kubelet[30360]: E0310 11:23:27.099954 30360 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to "KillPodSandbox" for "1275b4d6-e67a-463c-ba5d-cb0ebf64ce19" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"27ebb31735da264f97c20058195ae9e6c36af5f07922660b3b1f7cfd499d69ed\": plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/"" pod="kube-system/coredns-576f59f987-lmjqq" podUID="1275b4d6-e67a-463c-ba5d-cb0ebf64ce19"

Mar 10 11:23:27 master01 kubelet[30360]: I0310 11:23:27.390913 30360 scope.go:117] "RemoveContainer" containerID="fc52f2ad6f4d7d614edfb87bc0f4b37fadc4263137de057f04c3435583d58cff"

Mar 10 11:23:27 master01 kubelet[30360]: E0310 11:23:27.391109 30360 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to "StartContainer" for "calico-node" with CrashLoopBackOff: "back-off 10s restarting failed container=calico-node pod=calico-node-fp44f_kube-system(45208a7b-4b49-4eec-a09a-f14f31087119)"" pod="kube-system/calico-node-fp44f" podUID="45208a7b-4b49-4eec-a09a-f14f31087119"

Mar 10 11:23:29 master01 kubelet[30360]: E0310 11:23:29.100601 30360 log.go:32] "StopPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to destroy network for sandbox "8a67ca2ee7441687c4bbfb429c9f4e9c53705aad6a24feaeae90c3505dbfba56": plugin type="calico" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" podSandboxID="8a67ca2ee7441687c4bbfb429c9f4e9c53705aad6a24feaeae90c3505dbfba56"

Mar 10 11:23:29 master01 kubelet[30360]: E0310 11:23:29.100690 30360 kuberuntime_manager.go:1479] "Failed to stop sandbox" podSandboxID={"Type":"containerd","ID":"8a67ca2ee7441687c4bbfb429c9f4e9c53705aad6a24feaeae90c3505dbfba56"}

Mar 10 11:23:29 master01 kubelet[30360]: E0310 11:23:29.100744 30360 kuberuntime_manager.go:1079] "killPodWithSyncResult failed" err="failed to "KillPodSandbox" for "0b8f3737-979d-4005-aa98-0d9fca07fdc6" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"8a67ca2ee7441687c4bbfb429c9f4e9c53705aad6a24feaeae90c3505dbfba56\": plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/""

Mar 10 11:23:29 master01 kubelet[30360]: E0310 11:23:29.100781 30360 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to "KillPodSandbox" for "0b8f3737-979d-4005-aa98-0d9fca07fdc6" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"8a67ca2ee7441687c4bbfb429c9f4e9c53705aad6a24feaeae90c3505dbfba56\": plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/"" pod="kube-system/openebs-localpv-provisioner-69b4dfd865-z6nvq" podUID="0b8f3737-979d-4005-aa98-0d9fca07fdc6"

Mar 10 11:23:33 master01 kubelet[30360]: E0310 11:23:33.100579 30360 log.go:32] "StopPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to destroy network for sandbox "e0f6665744e6f48928559cbfb3662c9d21a009331801da410c8152b2d37cdd0e": plugin type="calico" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" podSandboxID="e0f6665744e6f48928559cbfb3662c9d21a009331801da410c8152b2d37cdd0e"

Mar 10 11:23:33 master01 kubelet[30360]: E0310 11:23:33.100653 30360 kuberuntime_manager.go:1479] "Failed to stop sandbox" podSandboxID={"Type":"containerd","ID":"e0f6665744e6f48928559cbfb3662c9d21a009331801da410c8152b2d37cdd0e"}

Mar 10 11:23:33 master01 kubelet[30360]: E0310 11:23:33.100701 30360 kuberuntime_manager.go:1079] "killPodWithSyncResult failed" err="failed to "KillPodSandbox" for "96eaf81f-6e4f-4b5a-aacd-bfd065845a6f" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"e0f6665744e6f48928559cbfb3662c9d21a009331801da410c8152b2d37cdd0e\": plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/""

Mar 10 11:23:33 master01 kubelet[30360]: E0310 11:23:33.100737 30360 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to "KillPodSandbox" for "96eaf81f-6e4f-4b5a-aacd-bfd065845a6f" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"e0f6665744e6f48928559cbfb3662c9d21a009331801da410c8152b2d37cdd0e\": plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/"" pod="kube-system/coredns-576f59f987-nvstr" podUID="96eaf81f-6e4f-4b5a-aacd-bfd065845a6f"