Skip to content

chore: upgrade vLLM to 0.18.0 and torch/unsloth to equivalent versions#342

Open
mckornfield wants to merge 1 commit intomainfrom
all-the-upgrades/mck
Open

chore: upgrade vLLM to 0.18.0 and torch/unsloth to equivalent versions#342
mckornfield wants to merge 1 commit intomainfrom
all-the-upgrades/mck

Conversation

@mckornfield
Copy link
Copy Markdown
Collaborator

  • Upgrade various versions to address CVEs
  • Remove deprecated parameters

@mckornfield mckornfield requested review from a team as code owners April 2, 2026 16:13
Copilot AI review requested due to automatic review settings April 2, 2026 16:13
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR upgrades the vLLM/Torch/Unsloth dependency stack to newer versions to address CVEs and adjusts the vLLM backend implementation/tests to align with removed/deprecated vLLM parameters and behaviors.

Changes:

  • Upgraded vLLM to 0.18.0 and updated Torch/TorchVision/TorchAO/Unsloth/XFormers pins accordingly.
  • Removed per-request logits processor usage from the vLLM backend (including removing the TypicalLogitsWarper wrapper) and updated unit tests to reflect that typical_p is excluded.
  • Increased an end-to-end test timeout to accommodate longer runtimes with the upgraded stack.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.

File Description
tests/generation/test_vllm_backend.py Updates mapping/unit tests to assert typical_p is excluded instead of creating a logits processor.
tests/e2e/test_safe_synthesizer.py Extends e2e timeout to reduce flakiness under slower runs.
src/nemo_safe_synthesizer/generation/vllm_backend.py Removes deprecated/unsupported vLLM request-time logits processing and deprecated structured-output args.
pyproject.toml Bumps dependency pins for torch/torchvision/torchao/unsloth/vllm/xformers.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mckornfield mckornfield force-pushed the all-the-upgrades/mck branch from a87679c to 01acad9 Compare April 2, 2026 17:15
Copilot AI review requested due to automatic review settings April 2, 2026 18:51
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@kendrickb-nvidia
Copy link
Copy Markdown
Collaborator

/sync

Copilot AI review requested due to automatic review settings April 2, 2026 20:24
@mckornfield mckornfield force-pushed the all-the-upgrades/mck branch from 4efd951 to 45b6003 Compare April 2, 2026 20:24
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mckornfield mckornfield force-pushed the all-the-upgrades/mck branch from 45b6003 to 514c635 Compare April 2, 2026 21:48
nina-xu
nina-xu previously approved these changes Apr 3, 2026
Copy link
Copy Markdown
Contributor

@nina-xu nina-xu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this could work as is. approving pending a full slurm sweep

Signed-off-by: Matt Kornfield <mkornfield@nvidia.com>
Copilot AI review requested due to automatic review settings April 3, 2026 14:55
@mckornfield mckornfield force-pushed the all-the-upgrades/mck branch from 514c635 to 6573f17 Compare April 3, 2026 14:55
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants