feat: surface generated workload failures in ModelBooster/ModelServing status

[vanshika2720]

What type of PR is this?

/kind enhancement

What this PR does / why we need it:

ModelServing's Progressing/UpdateInProgress conditions previously exposed only generic progress messages even when a generated Pod was stuck due to a concrete failure such as image pull failure, container crash, or scheduling failure.

This PR surfaces actionable generated Pod failure information through the existing ModelServing and ModelBooster status conditions.

The change:

• Extracts failure details from structured Pod status fields, including:
  • PodScheduled condition;
  • container/init-container Waiting reasons;
  • container/init-container Terminated reasons and exit codes.
• Avoids relying on Kubernetes Event text for programmatic classification.
• Records failure details per Role in the model-serving-controller's in-memory store.
• Clears recorded failures when the corresponding Pod becomes ready, preventing stale failure information.
• Reuses the existing Progressing and UpdateInProgress conditions instead of introducing new condition types.
• Replaces the generic condition reason/message with actionable failure information when a ServingGroup is blocked by a generated workload failure.
• Propagates actionable ModelServing failure information to ModelBooster's Active condition when the generated ModelServing is not available.
• Updates condition Reason/Message even when the condition Status itself has not changed, so newly surfaced failure details are not silently discarded.

This makes failures such as unschedulable Pods, image-pull failures, init-container failures, and runtime crashes visible directly from the parent resource status instead of requiring users to inspect generated Pods manually.

Which issue(s) this PR fixes:

Fixes #1503

Bug evidence (required for bug-related PRs):

The issue is observed along the generated workload production path:

ModelBooster → ModelServing → generated Pod → Pod scheduling/container status

Generated Pods can fail or remain unavailable while the parent ModelServing continues to report only generic Progressing/UpdateInProgress information. Previously, Pod failure information was used only for the existing recovery flow and the actionable reason was not propagated to the parent status.

Examples of failure information that can be surfaced include:

• Pod scheduling failures such as a missing PVC;
• image-pull failures;
• init-container/download failures;
• runtime container crashes.

The implementation reads the structured Pod status populated by the Kubernetes scheduler/kubelet and propagates the failure through:

Pod → Role failure state → ModelServing condition → ModelBooster Active condition

The failure is cleared when the affected Pod recovers, preventing stale status from remaining after recovery.

Tests cover scheduling failure, image-pull failure, init-container failure, runtime/container failure, recovery and stale-failure clearing, multiple failure cases, and end-to-end status propagation.

Special notes for your reviewer:

• Existing ModelServing condition types are reused; no new API condition type was introduced.
• ModelServingStatus and ModelBoosterStatus API structs are unchanged.
• Failure classification uses structured Pod status fields rather than Kubernetes Event message text.
• The first actionable failure is selected deterministically according to the existing Role/ServingGroup ordinal ordering.
• Failure details are cleared when the affected Role's Pod becomes ready.
• No controller behaviour unrelated to status propagation was changed.
• No CRD changes are required.

Validation performed:

• go build ./... — passed
• go vet ./... — passed
• gofmt — clean
• Targeted ModelServing/ModelBooster tests — passed
• Race tests for affected packages — new tests passed
• Full go test ./... — unit/controller packages passed; E2E tests require a live Kubernetes cluster

Does this PR introduce a user-facing change?:

Yes. Parent ModelBooster and ModelServing status conditions now expose actionable generated workload failure information.

Surface generated Pod/workload failure details in ModelBooster and ModelServing status conditions.

[volcano-sh-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign lizhencheng9527 for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• pkg/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[vanshika2720]

@YaoZengzeng @LiZhenCheng9527 @hzxuzhonghu

[LiZhenCheng9527]

Can you show the status of model serving in your cluster?
Verify the effectiveness of your PR