RBAC issues with addon-resizer since the upgrade to Kubernetes 1.30

[cesarca]

Which component are you using?:

/area addon-resizer

What version of the component are you using?:

Component version: addon-resizer 2.3

What k8s version are you using (kubectl version)?:

kubectl 1.31.0

kubectl version Output

$ kubectl version

What environment is this in?:

Azure Kubernetes Services

What did you expect to happen?:
With the RBAC configuration that worked in AKS 1.29 I expected that it would work similarly for 1.30

What happened instead?:
Now the addon-resizer logs show these errors in AKS 1.30:

E0212 09:48:15.109843       1 reflector.go:205] k8s.io/autoscaler/addon-resizer/nanny/kubernetes_client.go:147: Failed to list *v1.Deployment: Unauthorized
E0212 09:48:15.260151       1 reflector.go:205] k8s.io/autoscaler/addon-resizer/nanny/kubernetes_client.go:124: Failed to list *v1.Node: Unauthorized
E0212 09:48:16.296158       1 reflector.go:205] k8s.io/autoscaler/addon-resizer/nanny/kubernetes_client.go:124: Failed to list *v1.Node: Unauthorized
E0212 09:48:17.148749       1 reflector.go:205] k8s.io/autoscaler/addon-resizer/nanny/kubernetes_client.go:147: Failed to list *v1.Deployment: Unauthorized
E0212 09:48:17.315418       1 reflector.go:205] k8s.io/autoscaler/addon-resizer/nanny/kubernetes_client.go:124: Failed to list *v1.Node: Unauthorized
E0212 09:48:18.012364       1 reflector.go:205] k8s.io/autoscaler/addon-resizer/nanny/kubernetes_client.go:135: Failed to list *v1.Pod: Unauthorized
E0212 09:48:18.181736       1 reflector.go:205] k8s.io/autoscaler/addon-resizer/nanny/kubernetes_client.go:147: Failed to list *v1.Deployment: Unauthorized
E0212 09:48:18.336854       1 reflector.go:205] k8s.io/autoscaler/addon-resizer/nanny/kubernetes_client.go:124: Failed to list *v1.Node: Unauthorized
E0212 09:48:19.031161       1 reflector.go:205] k8s.io/autoscaler/addon-resizer/nanny/kubernetes_client.go:135: Failed to list *v1.Pod: Unauthorized
E0212 09:48:19.205592       1 reflector.go:205] k8s.io/autoscaler/addon-resizer/nanny/kubernetes_client.go:147: Failed to list *v1.Deployment: Unauthorized

How to reproduce it (as minimally and precisely as possible):

Here is my current RBAC configuration for kube-state-metrics deployment

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kube-state-metrics
  labels:
    app: kube-state-metrics
    stack: management
    namespace: {{ kubernetes_namespace }}
rules:
- apiGroups: [""]
  resources:
  - nodes
  - pods
  - services
  - resourcequotas
  - replicationcontrollers
  - limitranges
  - persistentvolumeclaims
  - persistentvolumes
  - namespaces
  - endpoints
  verbs: ["list", "watch"]
- apiGroups: ["extensions", "apps"]
  resources:
  - daemonsets
  - deployments
  - replicasets
  - ingresses
  verbs: ["list", "watch"]
- apiGroups: ["apps"]
  resources:
  - statefulsets
  verbs: ["list", "watch"]
- apiGroups: ["batch"]
  resources:
  - cronjobs
  - jobs
  verbs: ["list", "watch"]
- apiGroups: ["autoscaling"]
  resources:
  - horizontalpodautoscalers
  verbs: ["list", "watch"]
- apiGroups: ["storage.k8s.io"]
  resources:
  - storageclasses
  verbs: ["list", "watch"]
- apiGroups: ["policy"]
  resources:
  - poddisruptionbudgets
  verbs: ["list", "watch"]
- apiGroups: ["certificates.k8s.io"]
  resources:
  - certificatesigningrequests
  verbs: ["list", "watch"]
- apiGroups: ["networking.k8s.io"]
  resources: ["ingresses"]
  verbs: ["list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kube-state-metrics
  labels:
    app: kube-state-metrics
    stack: management
    namespace: {{ kubernetes_namespace }}
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kube-state-metrics
subjects:
- kind: ServiceAccount
  name: kube-state-metrics
  namespace: {{ kubernetes_namespace }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: kube-state-metrics
  namespace: {{ kubernetes_namespace }}
  labels:
    app: kube-state-metrics
    stack: management
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: kube-state-metrics-resizer
subjects:
- kind: ServiceAccount
  name: kube-state-metrics
  namespace: {{ kubernetes_namespace }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: kube-state-metrics-resizer
  namespace: {{ kubernetes_namespace }}
  labels:
    app: kube-state-metrics
    stack: management
rules:
- apiGroups: [apps]
  resources: [deployments]
  verbs:
    - list
    - watch
- apiGroups: [apps]
  resources: [deployments]
  resourceNames: [kube-state-metrics]
  verbs:
    - get
    - update
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kube-state-metrics
  namespace: {{ kubernetes_namespace }}
  labels:
    app: kube-state-metrics
    stack: management

Anything else we need to know?:

I have tried different RBAC configurations but I cannot avoid the errors

[Shubham82]

/area addon-resizer

[raywainman]

Thanks for opening this issue!

I'm a bit confused here... Are you sharing a service account for the addon-resizer with kube-state-metrics?

Can you share your addon-resizer deployment spec?

[cesarca]

Thanks for opening this issue!

I'm a bit confused here... Are you sharing a service account for the addon-resizer with kube-state-metrics?

Can you share your addon-resizer deployment spec?

Hi Raywainman, yes, in this version, addon-resizer is a container within the kube-state-metrics deployment. I already solved the issue by upgrading to version 1.8.11 of addon-resizer and changing the architecture to deploy it as a standalone deployment separate from kube-state-metrics. Thanks

[raywainman]

Sounds good, yeah running it as a standalone deployment would have been my recommendation as well.

Will close this issue. Please re-open if I'm missing something.