Skip to content

Conversation

Aryanag2
Copy link
Member

Update ADS to extend the CONTAINER_FAMILY_COMPATIBILITY variable to include GPT-OSS models.

@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Aug 21, 2025
@Aryanag2 Aryanag2 requested a review from elizjo August 21, 2025 22:48
elizjo
elizjo previously approved these changes Aug 21, 2025
Copy link

📌 Cov diff with main:

No lines with coverage information in this diff.

📌 Overall coverage:

Coverage-18.49%

darenr
darenr previously approved these changes Aug 22, 2025
InferenceContainerTypeFamily.AQUA_VLLM_V1_CONTAINER_FAMILY: [
InferenceContainerTypeFamily.AQUA_VLLM_V1_CONTAINER_FAMILY,
InferenceContainerTypeFamily.AQUA_VLLM_CONTAINER_FAMILY,
InferenceContainerTypeFamily.AQUA_VLLM_OPENAI_CONTAINER_FAMILY,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CONTAINER_FAMILY_COMPATIBILITY map is used in the get_preferred_compatible_family method, which determines the preferred container family when multiple (but potentially compatible) container families are selected in a multi-model deployment.

With the current implementation, if a GPT model is selected along with another model (for example, Llama 3), the algorithm checks compatibility and may incorrectly select AQUA_VLLM_V1_CONTAINER_FAMILY. Since AQUA_VLLM_V1_CONTAINER_FAMILY does not support GPT models, the deployment will fail.

To avoid this issue, we should introduce a new compatibility group in the dictionary that explicitly includes GPT alongside the relevant container families. For example:

CONTAINER_FAMILY_COMPATIBILITY: Dict[str, List[str]] = {
    InferenceContainerTypeFamily.AQUA_VLLM_OPENAI_CONTAINER_FAMILY: [
        InferenceContainerTypeFamily.AQUA_VLLM_OPENAI_CONTAINER_FAMILY,
        InferenceContainerTypeFamily.AQUA_VLLM_LLAMA4_CONTAINER_FAMILY,
        InferenceContainerTypeFamily.AQUA_VLLM_V1_CONTAINER_FAMILY,
        InferenceContainerTypeFamily.AQUA_VLLM_CONTAINER_FAMILY,
    ],
}

This way, when GPT models are included in a deployment, the system will correctly resolve to the AQUA_VLLM_OPENAI_CONTAINER_FAMILY group, ensuring proper compatibility and avoiding deployment errors.

@mrDzurb mrDzurb changed the title Update ADS to extend the CONTAINER_FAMILY_COMPATIBILITY variable to i… [AQUA] Extend the CONTAINER_FAMILY_COMPATIBILITY to include GPT-OSS models. Aug 26, 2025
Copy link

📌 Cov diff with main:

No lines with coverage information in this diff.

📌 Overall coverage:

Coverage-58.25%

@Aryanag2 Aryanag2 dismissed stale reviews from darenr and elizjo via 8abdd00 August 29, 2025 20:29
Copy link

📌 Cov diff with main:

No lines with coverage information in this diff.

📌 Overall coverage:

Coverage-18.29%

@mrDzurb mrDzurb enabled auto-merge (squash) September 4, 2025 00:53
@mrDzurb mrDzurb merged commit 5cde238 into main Sep 4, 2025
13 of 17 checks passed
Copy link

github-actions bot commented Sep 4, 2025

📌 Cov diff with main:

No lines with coverage information in this diff.

📌 Overall coverage:

Coverage-18.29%

1 similar comment
Copy link

github-actions bot commented Sep 4, 2025

📌 Cov diff with main:

No lines with coverage information in this diff.

📌 Overall coverage:

Coverage-18.29%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OCA Verified All contributors have signed the Oracle Contributor Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants