Skip to content

feat: add ModelServing eviction protection - #1485

Open
VanderChen wants to merge 1 commit into
volcano-sh:mainfrom
VanderChen:feat/006-modelserving-eviction-budget-rework-upstream
Open

feat: add ModelServing eviction protection#1485
VanderChen wants to merge 1 commit into
volcano-sh:mainfrom
VanderChen:feat/006-modelserving-eviction-budget-rework-upstream

Conversation

@VanderChen

@VanderChen VanderChen commented Aug 3, 2026

Copy link
Copy Markdown
Member

What type of PR is this?

/kind feature

What this PR does / why we need it:

This PR adds opt-in eviction protection for ModelServing workloads.

During voluntary node maintenance, such as node draining for OS upgrades, hardware repair, or other operational work, multiple ModelServing Pods may be evicted concurrently. Without workload-aware protection, these evictions can reduce serving capacity below the availability required by the workload.

This PR introduces eviction budgets at two protection levels:

  • ServingGroup: limits how many ServingGroups may be disrupted at the same time;
  • Role: limits disruptions independently for each Role within a ServingGroup.

The strategy is configured through spec.rolloutStrategy.evictionStrategy:

  • protectionLevel selects ServingGroup or Role protection;
  • minAvailable defines the minimum available ServingGroups;
  • roleMinAvailable defines the minimum available Role instances for each Role.

The eviction webhook:

  • allows evictions while the configured availability budget is satisfied;
  • denies evictions that would reduce availability below the budget;
  • treats multiple Pods from the same disrupted ServingGroup or Role instance as one disruption unit;
  • coordinates concurrent eviction requests;
  • refreshes readiness from the Kubernetes API when informer state may be stale;
  • allows an already NotReady target Pod to be evicted without expanding the existing disruption;
  • cleans up recovered or expired disruption records so rolling maintenance can continue.

The PR also adds:

  • validation for invalid or incomplete eviction strategies;
  • Helm configuration for enabling the eviction webhook;
  • the required webhook registration and RBAC permissions;
  • generated CRD and client-go updates;
  • unit tests and documentation.

Which issue(s) this PR fixes:

Fixes #1481

Bug evidence (required for bug-related PRs):

N/A. This is an enhancement.

The implementation was verified in a local Kind cluster with real policy/v1
Eviction requests.

Observed behavior:

  • a ServingGroup eviction within the budget was allowed;
  • another Pod from the same disrupted ServingGroup was allowed;
  • an eviction from a different ServingGroup was denied after reaching
    minAvailable;
  • a Role eviction within its budget was allowed;
  • another instance of the same Role was denied after reaching
    roleMinAvailable;
  • an eviction for a different Role was evaluated against its independent budget;
  • NotReady target Pods were allowed to be evicted, while Ready Pods belonging to
    an already unavailable protection unit remained protected;
  • invalid strategies were rejected and valid strategies were accepted.

Special notes for your reviewer:

  • Eviction protection is opt-in and disabled by default.
  • The feature only applies to Pods owned by a ModelServing with an eviction
    strategy configured.
  • ServingGroup protection uses minAvailable.
  • Role protection uses roleMinAvailable, scoped by ServingGroup and Role.
  • Concurrent webhook requests share disruption state to prevent multiple
    requests from independently consuming the same availability budget.
  • NotReady target Pods do not consume an additional disruption budget.
  • The PR is limited to ModelServing eviction protection and does not modify the
    ModelServing controller reconciliation or recovery behavior.
  • Review feedback is especially welcome on the API shape, budget semantics,
    concurrency handling, and NotReady target behavior.

Does this PR introduce a user-facing change?:

Add optional ServingGroup- and Role-level eviction protection for ModelServing workloads during voluntary node maintenance.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@volcano-sh-bot

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign yaozengzeng for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@katara-Jayprakash

Copy link
Copy Markdown
Member

@VanderChen make sure ci is happy sir!
/cc @LiZhenCheng9527

Add opt-in eviction budgets for ModelServing workloads so voluntary node maintenance can preserve serving availability at ServingGroup or Role granularity.

Track in-flight disruptions across webhook replicas, handle informer cache gaps and concurrent eviction requests, allow safe eviction of unhealthy targets, and validate strategy settings.

Expose the webhook through the Helm chart and document the API and deployment configuration.

Signed-off-by: Min Chen <vanderchen@outlook.com>
@VanderChen
VanderChen force-pushed the feat/006-modelserving-eviction-budget-rework-upstream branch from ce35051 to ccd20c7 Compare August 5, 2026 01:25
@aeron-gh aeron-gh mentioned this pull request Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add ModelServing-aware eviction protection for safe node drain

3 participants