Skip to content

Conversation

Aryanag2
Copy link
Member

Update ADS to extend the CONTAINER_FAMILY_COMPATIBILITY variable to include GPT-OSS models.

@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Aug 21, 2025
@Aryanag2 Aryanag2 requested a review from elizjo August 21, 2025 22:48
Copy link

📌 Cov diff with main:

No lines with coverage information in this diff.

📌 Overall coverage:

Coverage-18.49%

@@ -126,6 +126,7 @@ class Platform(ExtendedEnum):
InferenceContainerTypeFamily.AQUA_VLLM_V1_CONTAINER_FAMILY: [
InferenceContainerTypeFamily.AQUA_VLLM_V1_CONTAINER_FAMILY,
InferenceContainerTypeFamily.AQUA_VLLM_CONTAINER_FAMILY,
InferenceContainerTypeFamily.AQUA_VLLM_OPENAI_CONTAINER_FAMILY,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CONTAINER_FAMILY_COMPATIBILITY map is used in the get_preferred_compatible_family method, which determines the preferred container family when multiple (but potentially compatible) container families are selected in a multi-model deployment.

With the current implementation, if a GPT model is selected along with another model (for example, Llama 3), the algorithm checks compatibility and may incorrectly select AQUA_VLLM_V1_CONTAINER_FAMILY. Since AQUA_VLLM_V1_CONTAINER_FAMILY does not support GPT models, the deployment will fail.

To avoid this issue, we should introduce a new compatibility group in the dictionary that explicitly includes GPT alongside the relevant container families. For example:

CONTAINER_FAMILY_COMPATIBILITY: Dict[str, List[str]] = {
    InferenceContainerTypeFamily.AQUA_VLLM_OPENAI_CONTAINER_FAMILY: [
        InferenceContainerTypeFamily.AQUA_VLLM_OPENAI_CONTAINER_FAMILY,
        InferenceContainerTypeFamily.AQUA_VLLM_LLAMA4_CONTAINER_FAMILY,
        InferenceContainerTypeFamily.AQUA_VLLM_V1_CONTAINER_FAMILY,
        InferenceContainerTypeFamily.AQUA_VLLM_CONTAINER_FAMILY,
    ],
}

This way, when GPT models are included in a deployment, the system will correctly resolve to the AQUA_VLLM_OPENAI_CONTAINER_FAMILY group, ensuring proper compatibility and avoiding deployment errors.

@mrDzurb mrDzurb changed the title Update ADS to extend the CONTAINER_FAMILY_COMPATIBILITY variable to i… [AQUA] Extend the CONTAINER_FAMILY_COMPATIBILITY to include GPT-OSS models. Aug 26, 2025
Copy link

📌 Cov diff with main:

No lines with coverage information in this diff.

📌 Overall coverage:

Coverage-58.25%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OCA Verified All contributors have signed the Oracle Contributor Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants