A new parameter for the management of the limits that the VPA assigns to the new Pod

[FabrizioCafolla]

Which component are you using?:

/area vertical-pod-autoscaler

Is your feature request designed to solve a problem? If so describe the problem this feature should solve.:

I am having problems with the resources.limit described in issue #6996

• I expect the pod to be restarted with the new resources and limit
• I expect the required resources to be in a range for CPU > minAllowed && CPU < maxAllowed and Memory > minAllowed && Memory < maxAllowed
• I expect the new resource limits to always be CPU <= maxAllowed and Memory <= maxAllowed

I also expected the maxAllowed option to set the resources.limit, but it seems to only apply to resources.requests. This causes the pod to potentially use too many resources (which I don't want). See more details described in the issue

Describe the solution you'd like.:

I want to be able to handle new limits on the Pod.

Because with the controlledValues: RequestsAndLimits' option makes the VPA set limits beyond the maxAllowed', while this is expected behavior, it is too risky and uncontrollable because the limits the VPA sets run the risk of causing the pod to use many more resources than expected by saturating the nodes.

Describe any alternative solutions you've considered.:

Can we think of a new option for the ContainerResourcePolicy configuration, such as adding maxLimitAllowed? This would allow setting a ceiling limit on the pod's resources.

Additional context.:

Hi @voelzmo. I am testing VPA and I am experiencing the same problem. I don't know if I am missing something but the pod limits are not correct ( or rather not as I expect)

System

• MacOs Darwin Kernel Version 24.2.0
• kind version 0.22.0

VPA PoC

apiVersion: apps/v1
kind: Deployment
metadata:
  name: high-cpu-utilization-deployment
spec:
  replicas: 2
  selector:
    matchLabels:
      app: cpu-utilization-app
  template:
    metadata:
      labels:
        app: cpu-utilization-app
    spec:
      containers:
        - name: cpu-utilization-container
          image: ubuntu
          command:
            [
              "/bin/sh",
              "-c",
              "apt-get update && apt-get install -y stress-ng && while true; do stress-ng --cpu 1; done",
            ]
          resources:
            limits:
              cpu: "20m"
              memory: "50Mi"
            requests:
              cpu: "1m"
              memory: "10Mi"
---
apiVersion: "autoscaling.k8s.io/v1"
kind: VerticalPodAutoscaler
metadata:
  name: stress-vpa
spec:
  targetRef:
    apiVersion: "apps/v1"
    kind: Deployment
    name: high-cpu-utilization-deployment
  updatePolicy:
    updateMode: Auto
  resourcePolicy:
    containerPolicies:
      - containerName: "*"
        minAllowed:
          cpu: 1m
          memory: 10Mi
        maxAllowed:
          cpu: 200m
          memory: 100Mi
        controlledResources: ["cpu", "memory"]
        controlledValues: RequestsAndLimits

Desired status

• I expect the high-cpu-utilization-deployment pod to be restarted with the new resources and limit
• I expect the required resources to be in a range for CPU > 1m && CPU < 200m and Memory > 10Mi && CPU < 100Mi
• I expect the new resource limits to always be CPU <= 200m and Memory <= 100Mi

I also expected the maxAllowed option to set the resources.limit, but it seems to only apply to resources.requests. This causes the pod to potentially use too many resources (which I don't want), as in the example above where I switched to 4 CPUs. I used the controlledValues: RequestsOnly configuration to prevent the VPA from touching the limits.

What happens

I created locally in my cluster kind the Deployment and VPA described above

➜  test-vpa git:(main) ✗ kubectl get pods                                                        
NAME                                              READY   STATUS    RESTARTS   AGE
high-cpu-utilization-deployment-fc98c7d68-5vwts   1/1     Running   0          36s
high-cpu-utilization-deployment-fc98c7d68-q2ztv   1/1     Running   0          36s

I retrieve the resources of the created pods (which are those indicated in the Deployment)

kubectl get pods high-cpu-utilization-deployment-fc98c7d68-5vwts -o yaml

...
    name: cpu-utilization-container
    resources:
      limits:
        cpu: 20m
        memory: 50Mi
      requests:
        cpu: 1m
        memory: 10Mi
...

The VPA creates a new pod with new resources

➜  test-vpa git:(main) ✗ kubectl get pods                                                        
NAME                                              READY   STATUS        RESTARTS   AGE
high-cpu-utilization-deployment-fc98c7d68-5vwts   1/1     Terminating   0          56s
high-cpu-utilization-deployment-fc98c7d68-8tk45   1/1     Running       0          6s
high-cpu-utilization-deployment-fc98c7d68-q2ztv   1/1     Running       0          56s

I retrieve the new resources of the pod and it turns out that the limits are incorrect.

kubectl get pods high-cpu-utilization-deployment-fc98c7d68-8tk45  -o yaml

...
    resources:
      limits:
        cpu: "4" <--------- Why is there this new limit?
        memory: 500Mi <---- Why is there this new limit?
      requests:
        cpu: 200m
        memory: 100Mi
...

test-vpa git:(main) ✗ kubectl get vpa stress-vpa -o yaml

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"autoscaling.k8s.io/v1","kind":"VerticalPodAutoscaler","metadata":{"annotations":{},"name":"stress-vpa","namespace":"default"},"spec":{"resourcePolicy":{"containerPolicies":[{"containerName":"*","controlledResources":["cpu","memory"],"controlledValues":"RequestsAndLimits","maxAllowed":{"cpu":"200m","memory":"100Mi"},"minAllowed":{"cpu":"20m","memory":"50Mi"}}]},"targetRef":{"apiVersion":"apps/v1","kind":"Deployment","name":"high-cpu-utilization-deployment"},"updatePolicy":{"updateMode":"Auto"}}}
  creationTimestamp: "2025-01-30T11:49:39Z"
  generation: 1
  name: stress-vpa
  namespace: default
  resourceVersion: "33688"
  uid: 6d5c5758-84b8-4507-bbcd-02f035d8694c
spec:
  resourcePolicy:
    containerPolicies:
    - containerName: '*'
      controlledResources:
      - cpu
      - memory
      controlledValues: RequestsAndLimits
      maxAllowed:
        cpu: 200m
        memory: 100Mi
      minAllowed:
        cpu: 20m
        memory: 50Mi
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: high-cpu-utilization-deployment
  updatePolicy:
    updateMode: Auto
status:
  conditions:
  - lastTransitionTime: "2025-01-30T11:50:29Z"
    status: "True"
    type: RecommendationProvided
  recommendation:
    containerRecommendations:
    - containerName: cpu-utilization-container
      lowerBound:
        cpu: 32m
        memory: 100Mi
      target:
        cpu: 200m
        memory: 100Mi
      uncappedTarget:
        cpu: 1168m
        memory: 262144k
      upperBound:
        cpu: 200m
        memory: 100Mi

Questions

1. How do I manage the new limits?
2. Can we think of a new option for the ContainerResourcePolicy configuration, such as adding maxLimitAllowed? This would allow setting a ceiling limit on the pod's resources.

Originally posted by @FabrizioCafolla in #6996