[BUG] H100 can be misclassified as A100-80G and receive A100 runtime overrides

[YouNeedCryDear]

What happened?

An H100 GPU workload used the A100-80G accelerator-specific ServingRuntime configuration. The generated runtime arguments used the A100 tensor parallelism override (nvidia-a100-80g-4) instead of the H100 override (nvidia-h100-1).

Root cause analysis process:

1. Traced accelerator selection from the InferenceService controller to AcceleratorClassSelector.GetAcceleratorClass.
2. Found that the selected accelerator class name is passed through the workload request into the engine/decoder component builder.
3. Confirmed that component construction stores the selected class name in BaseComponentFields.AcceleratorClassName.
4. Found that runtime argument overrides are chosen by calling SupportedModelFormat.GetAcceleratorConfig(AcceleratorClassName).
5. Confirmed GetAcceleratorConfig is a direct map lookup into supportedModelFormats[].acceleratorConfig.
6. Therefore, once the selected class name is nvidia-a100-80g-4, OME applies the A100-specific runtime args and tensor parallelism override.
7. Traced why the selected class name can be wrong: policy-based selection builds candidates from runtime.spec.acceleratorRequirements.acceleratorClasses, preserving runtime list order. Under FirstAvailable, the first valid class in that list is selected.
8. If a runtime lists nvidia-a100-80g-4 before nvidia-h100-1, and the InferenceService does not explicitly pin H100 or provide constraints that exclude A100, the selected class name can be A100 even for an H100 deployment path.

The core issue is identity confusion: the selected AcceleratorClassName is treated as the hardware identity for runtime override selection, but in this path it is derived from runtime class ordering rather than from a validated match to the actual accelerator hardware. Class ordering can therefore cause OME to apply the wrong accelerator-specific config.

What did you expect to happen?

OME should only apply accelerator-specific ServingRuntime overrides for the accelerator class that matches the intended or actual accelerator target. If the workload is targeting H100, runtime argument merging should use the H100 accelerator config and not an earlier A100 entry from the runtime's accelerator class list.

How can we reproduce it (as minimally and precisely as possible)?

Create a ServingRuntime with accelerator-specific model format overrides keyed by A100 and H100 classes, with A100 listed before H100 in acceleratorRequirements.acceleratorClasses. Reconcile an InferenceService that selects accelerators by FirstAvailable policy, or otherwise does not explicitly pin H100.

apiVersion: ome.io/v1beta1
kind: ServingRuntime
metadata:
  name: example-runtime
spec:
  acceleratorRequirements:
    acceleratorClasses:
      - nvidia-a100-80g-4
      - nvidia-h100-1
  supportedModelFormats:
    - modelFormat:
        name: safetensors
      autoSelect: true
      acceleratorConfig:
        nvidia-a100-80g-4:
          tensorParallelismOverride:
            tensorParallelSize: 4
        nvidia-h100-1:
          tensorParallelismOverride:
            tensorParallelSize: 1
---
apiVersion: ome.io/v1beta1
kind: InferenceService
metadata:
  name: example-isvc
spec:
  runtime:
    name: example-runtime
  acceleratorSelector:
    policy: FirstAvailable

With the current selection flow, FirstAvailable can select nvidia-a100-80g-4 because it is the first valid candidate from the runtime list. The component then uses that selected class name to fetch supportedModelFormats[].acceleratorConfig["nvidia-a100-80g-4"], applying the A100 tensor parallelism override.

Anything else we need to know?

Suggested implementation direction:

• Add regression coverage for the full value flow: selected class name -> component AcceleratorClassName -> GetAcceleratorConfig -> TensorParallelismOverride.
• Ensure policy-based accelerator selection does not treat runtime list order alone as proof of hardware identity when accelerator-specific runtime overrides are applied.
• Consider requiring an explicit accelerator selector, stronger constraints, or an availability/match signal before applying class-specific acceleratorConfig overrides.
• Make the selection behavior deterministic and observable when multiple accelerator classes are valid for a runtime, especially when those classes have different tensor parallelism overrides.

Environment

• OME version: current main branch / local checkout
• Kubernetes version (use kubectl version): not captured
• Cloud provider or hardware configuration: heterogeneous NVIDIA GPU cluster with H100 and A100-80G accelerator classes
• OS (e.g., from /etc/os-release): not captured
• Runtime (SGLang, vLLM, etc.) and version: runtime using supportedModelFormats[].acceleratorConfig tensor parallelism overrides
• Model being served (if applicable): not captured
• Install method (Helm, kubectl, etc.): not captured

Acceptance criteria

• FirstAvailable or other policy-based accelerator selection cannot silently apply an A100 accelerator config for an H100-targeted workload solely because A100 appears earlier in runtime.spec.acceleratorRequirements.acceleratorClasses.
• Runtime argument merging uses an accelerator class identity that is explicitly requested or validated against the selected/target accelerator path before applying class-specific tensorParallelismOverride.
• Regression tests cover runtime ordering where A100 appears before H100 and verify that the H100 tensor parallelism override is used for an H100-targeted workload.
• Logs, events, or status make it clear which accelerator class was selected and why when accelerator-specific runtime overrides are applied.

Blocked by

None - can start immediately.