test: move backend unit tests #3958

pvijayakrish · 2025-10-29T17:07:11Z

Overview:

Move the backend unit tests for vLLM, SGLang, and TRTLLM into their respective backend subdirectories to align with the test directory structure.

Summary by CodeRabbit

Tests
- Reorganized Prometheus utility tests into backend-specific test modules for improved maintainability.
- Added unit tests for Prometheus utilities across multiple backends with comprehensive filtering and error-handling validation.
- Enhanced test configuration with improved fixture management and conditional test collection based on installed dependencies.
Chores
- Updated test path resolution to use repository-root anchoring for consistency across test suites.

copy-pr-bot · 2025-10-30T17:27:55Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

coderabbitai · 2025-10-30T17:32:01Z

Walkthrough

Tests for Prometheus utilities were moved from a centralized tests/unit/ module into backend-specific test directories; new backend conftest fixtures were added and unit tests updated to use repository-root-based path resolution.

Changes

Cohort / File(s)	Summary
Backend conftest factories & hooks `components/src/dynamo/vllm/tests/conftest.py`, `components/src/dynamo/trtllm/tests/conftest.py`	Add `pytest_ignore_collect` to skip backend tests when framework modules are absent and add `make_cli_args_fixture(module_name: str)` to produce a `mock_cli_args` fixture for mocking CLI argv (positional and kwargs styles).
Prometheus utils tests (per-backend) `components/src/dynamo/vllm/tests/test_vllm_prometheus_utils.py`, `components/src/dynamo/sglang/tests/test_sglang_prometheus_utils.py`, `components/src/dynamo/trtllm/tests/test_trtllm_prometheus_utils.py`	New backend-specific tests for `get_prometheus_expfmt`: fixtures mocking `generate_latest`, tests for metric prefix filtering/exclusion, prefix addition, HELP/TYPE preservation, label/value checks, empty-result handling, and error handling.
Backend unit test path & fixture updates `components/src/dynamo/vllm/tests/test_vllm_unit.py`, `components/src/dynamo/sglang/tests/test_sglang_unit.py`, `components/src/dynamo/trtllm/tests/test_trtllm_unit.py`	Update imports to use backend conftest `make_cli_args_fixture`; introduce `REPO_ROOT` and compute `TEST_DIR`/`JINJA_TEMPLATE_PATH` from repository root instead of relative file paths.
Removed centralized tests `tests/unit/test_prometheus_utils.py`	Remove the centralized Prometheus utilities test module (its fixtures and `TestGetPrometheusExpfmt` class were deleted in favor of backend-specific copies).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Areas to review closely:
- correctness of pytest_ignore_collect patterns and their mapping to required module import checks;
- parity between removed centralized tests and the new backend-specific test implementations (coverage and assertions);
- REPO_ROOT calculation and resulting JINJA_TEMPLATE_PATH resolution across environments.

Poem

🐇 I hopped through tests and moved each little file,

Prometheus metrics found a cozy backend style.
Conftests stitched the argv, paths now rooted true,
Tiny rabbit cheers — the tests ran anew! 🥕

Pre-merge checks

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description Check	⚠️ Warning	The pull request description provides only the Overview section with a brief statement about moving backend unit tests into their respective subdirectories. However, the required template includes four key sections: Overview, Details, Where should the reviewer start, and Related Issues. The description lacks the Details section describing the specific changes made, guidance on where reviewers should focus, and any issue references. This represents incomplete coverage of the template structure, providing only approximately 25% of the expected content sections.
Docstring Coverage	⚠️ Warning	Docstring coverage is 73.91% which is insufficient. The required threshold is 80.00%.	You can run `@coderabbitai generate docstrings` to improve docstring coverage.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Title Check	✅ Passed	The title "test: move backend unit tests" directly and concisely describes the primary change in the pull request. The changeset moves unit tests for vLLM, SGLang, and TRTLLM backends into their respective backend subdirectories, which is accurately captured by the title. The title uses conventional commit formatting with a "test:" prefix and is specific and clear enough for someone scanning history to understand the primary change without ambiguity.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 1

🧹 Nitpick comments (1)

components/src/dynamo/trtllm/tests/conftest.py (1)

16-77: Consider consolidating duplicate conftest code across backends.

The pytest_ignore_collect and make_cli_args_fixture implementations are identical across all three backend conftest files (vllm, sglang, trtllm). Consider extracting these into a shared test utilities module to reduce duplication.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1da9d70 and 38f618a.

📒 Files selected for processing (9)

components/src/dynamo/sglang/tests/test_sglang_prometheus_utils.py (1 hunks)
components/src/dynamo/sglang/tests/test_sglang_unit.py (1 hunks)
components/src/dynamo/trtllm/tests/conftest.py (1 hunks)
components/src/dynamo/trtllm/tests/test_trtllm_prometheus_utils.py (1 hunks)
components/src/dynamo/trtllm/tests/test_trtllm_unit.py (1 hunks)
components/src/dynamo/vllm/tests/conftest.py (1 hunks)
components/src/dynamo/vllm/tests/test_vllm_prometheus_utils.py (1 hunks)
components/src/dynamo/vllm/tests/test_vllm_unit.py (1 hunks)
tests/unit/test_prometheus_utils.py (0 hunks)

💤 Files with no reviewable changes (1)

tests/unit/test_prometheus_utils.py

🧰 Additional context used

🧠 Learnings (2)

📓 Common learnings

Learnt from: nnshah1
PR: ai-dynamo/dynamo#1444
File: tests/fault_tolerance/scenarios.py:57-57
Timestamp: 2025-07-01T15:39:56.789Z
Learning: The fault tolerance tests in tests/fault_tolerance/ are designed to run only in the mounted container environment, so hardcoded absolute paths with `/workspace/` prefix are intentional and should not be changed to relative paths.

📚 Learning: 2025-07-01T15:39:56.789Z

Learnt from: nnshah1
PR: ai-dynamo/dynamo#1444
File: tests/fault_tolerance/scenarios.py:57-57
Timestamp: 2025-07-01T15:39:56.789Z
Learning: The fault tolerance tests in tests/fault_tolerance/ are designed to run only in the mounted container environment, so hardcoded absolute paths with `/workspace/` prefix are intentional and should not be changed to relative paths.

Applied to files:

components/src/dynamo/trtllm/tests/test_trtllm_unit.py

🧬 Code graph analysis (6)

components/src/dynamo/vllm/tests/test_vllm_prometheus_utils.py (2)

components/src/dynamo/common/utils/prometheus.py (1)

get_prometheus_expfmt (100-197)

components/src/dynamo/sglang/tests/test_sglang_prometheus_utils.py (4)

TestGetPrometheusExpfmt (17-98)

mock_generate_latest (42-43)

mock_generate_latest (54-55)

test_error_handling (89-98)

components/src/dynamo/sglang/tests/test_sglang_prometheus_utils.py (1)

components/src/dynamo/common/utils/prometheus.py (1)

get_prometheus_expfmt (100-197)

components/src/dynamo/trtllm/tests/test_trtllm_prometheus_utils.py (1)

components/src/dynamo/common/utils/prometheus.py (1)

get_prometheus_expfmt (100-197)

components/src/dynamo/trtllm/tests/test_trtllm_unit.py (1)

components/src/dynamo/trtllm/tests/conftest.py (1)

make_cli_args_fixture (38-77)

components/src/dynamo/vllm/tests/test_vllm_unit.py (1)

components/src/dynamo/vllm/tests/conftest.py (1)

make_cli_args_fixture (38-77)

components/src/dynamo/sglang/tests/test_sglang_unit.py (1)

components/src/dynamo/sglang/tests/conftest.py (1)

make_cli_args_fixture (38-77)

🪛 Ruff (0.14.2)

components/src/dynamo/vllm/tests/test_vllm_prometheus_utils.py

40-40: Unused function argument: reg

(ARG001)

components/src/dynamo/sglang/tests/test_sglang_prometheus_utils.py

42-42: Unused function argument: reg

(ARG001)

54-54: Unused function argument: reg

(ARG001)

components/src/dynamo/trtllm/tests/test_trtllm_prometheus_utils.py

43-43: Unused function argument: reg

(ARG001)

components/src/dynamo/vllm/tests/conftest.py

16-16: Unused function argument: config

(ARG001)

components/src/dynamo/trtllm/tests/conftest.py

16-16: Unused function argument: config

(ARG001)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

GitHub Check: Build and Test - dynamo

🔇 Additional comments (8)

components/src/dynamo/sglang/tests/test_sglang_unit.py (1)

13-21: LGTM! Test infrastructure updated correctly for new location.

The import path change and REPO_ROOT-based path resolution properly support the test file's new location in the backend subdirectory.

components/src/dynamo/trtllm/tests/conftest.py (1)

16-35: LGTM! The pytest hook signature is correct.

The config parameter is part of the standard pytest hook signature for pytest_ignore_collect and must be present even when unused.

components/src/dynamo/trtllm/tests/test_trtllm_unit.py (1)

11-20: LGTM! Test infrastructure correctly updated.

The import path and REPO_ROOT-based path resolution are properly configured for the test file's new backend-specific location.

components/src/dynamo/vllm/tests/conftest.py (1)

16-77: LGTM! Conftest utilities correctly implemented.

The conditional test collection hook and CLI args fixture factory provide proper test infrastructure for the vLLM backend.

components/src/dynamo/sglang/tests/test_sglang_prometheus_utils.py (1)

66-98: LGTM! Test cases properly validate Prometheus filtering.

The tests correctly verify metric filtering, exclusion, label preservation, and error handling for the SGLang backend.

components/src/dynamo/vllm/tests/test_vllm_prometheus_utils.py (1)

20-83: LGTM! Prometheus utility tests correctly implemented.

The fixture and test cases properly validate metric filtering, exclusion, and error handling for the vLLM backend.

components/src/dynamo/trtllm/tests/test_trtllm_prometheus_utils.py (1)

20-111: LGTM! Comprehensive Prometheus utility test coverage.

The fixture and test cases thoroughly validate metric filtering, prefix addition, default behavior, edge cases, and error handling for the TensorRT-LLM backend.

components/src/dynamo/vllm/tests/test_vllm_unit.py (1)

12-20: LGTM! Test infrastructure properly updated.

The import path and REPO_ROOT-based path resolution correctly support the test file's relocation to the vLLM backend subdirectory.

components/src/dynamo/sglang/tests/test_sglang_prometheus_utils.py

pvijayakrish · 2025-10-30T17:43:10Z

/ok to test 38f618a

Signed-off-by: pvijayakrish <[email protected]>

components/src/dynamo/trtllm/tests/conftest.py

KrishnanPrash

Left a few comments about cleaning up conftest.py

…ach backend Signed-off-by: pvijayakrish <[email protected]>

pvijayakrish · 2025-10-30T21:45:13Z

/ok to test a0c6bbb

pull-request-size bot added the size/XL label Oct 29, 2025

copy-pr-bot bot temporarily deployed to GITLAB October 29, 2025 18:08 Inactive

copy-pr-bot bot temporarily deployed to GITLAB October 29, 2025 18:09 Inactive

copy-pr-bot bot temporarily deployed to GITLAB October 29, 2025 23:59 Inactive

copy-pr-bot bot temporarily deployed to GITLAB October 30, 2025 00:00 Inactive

pvijayakrish changed the title ~~Pvijayakrish/move backend unit tests~~ tests: move backend unit tests Oct 30, 2025

copy-pr-bot bot temporarily deployed to GITLAB October 30, 2025 01:03 Inactive

copy-pr-bot bot temporarily deployed to GITLAB October 30, 2025 01:04 Inactive

pvijayakrish changed the title ~~tests: move backend unit tests~~ test: move backend unit tests Oct 30, 2025

github-actions bot added the test label Oct 30, 2025

copy-pr-bot bot temporarily deployed to GITLAB October 30, 2025 01:21 Inactive

copy-pr-bot bot temporarily deployed to GITLAB October 30, 2025 17:23 Inactive

copy-pr-bot bot temporarily deployed to GITLAB October 30, 2025 17:24 Inactive

pvijayakrish requested review from KrishnanPrash and alec-flowers October 30, 2025 17:25

pvijayakrish force-pushed the pvijayakrish/move-backend-unit-tests branch from d4f9edf to fde944b Compare October 30, 2025 17:27

pvijayakrish marked this pull request as ready for review October 30, 2025 17:28

pvijayakrish requested review from a team as code owners October 30, 2025 17:28

pvijayakrish added 8 commits October 30, 2025 10:30

Move vllm unit tests to under backend directory

bb54dd4

REVERT this changes

7f4e9d6

Moving the backend unit tests to the respective backend directory

a8b6ac7

Renamed the prometheus test file

0f31b5c

Update the test directory path

be9d950

Fixes

160790b

Removing blankspace

d0ae85a

remove whitespace

38f618a

pvijayakrish force-pushed the pvijayakrish/move-backend-unit-tests branch from fde944b to 38f618a Compare October 30, 2025 17:30

coderabbitai bot reviewed Oct 30, 2025

View reviewed changes

components/src/dynamo/sglang/tests/test_sglang_prometheus_utils.py Outdated Show resolved Hide resolved

copy-pr-bot bot temporarily deployed to GITLAB October 30, 2025 17:43 Inactive

copy-pr-bot bot temporarily deployed to GITLAB October 30, 2025 17:51 Inactive

Removing code block

0d5fdb1

Signed-off-by: pvijayakrish <[email protected]>

KrishnanPrash reviewed Oct 30, 2025

View reviewed changes

components/src/dynamo/trtllm/tests/conftest.py Outdated Show resolved Hide resolved

KrishnanPrash reviewed Oct 30, 2025

View reviewed changes

components/src/dynamo/trtllm/tests/conftest.py Show resolved Hide resolved

KrishnanPrash requested changes Oct 30, 2025

View reviewed changes

Removing backend_requirements and having separate conftest file for e…

a0c6bbb

…ach backend Signed-off-by: pvijayakrish <[email protected]>

copy-pr-bot bot temporarily deployed to GITLAB October 30, 2025 21:45 Inactive

pvijayakrish requested a review from KrishnanPrash October 30, 2025 22:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

test: move backend unit tests #3958

test: move backend unit tests #3958

pvijayakrish commented Oct 29, 2025 •

edited by coderabbitai bot

Loading

Uh oh!

copy-pr-bot bot commented Oct 30, 2025

Uh oh!

coderabbitai bot commented Oct 30, 2025 •

edited

Loading

Uh oh!

coderabbitai bot left a comment

Uh oh!

Uh oh!

pvijayakrish commented Oct 30, 2025

Uh oh!

Uh oh!

Uh oh!

KrishnanPrash left a comment

Uh oh!

pvijayakrish commented Oct 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

test: move backend unit tests #3958

Are you sure you want to change the base?

test: move backend unit tests #3958

Conversation

pvijayakrish commented Oct 29, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview:

Summary by CodeRabbit

Uh oh!

copy-pr-bot bot commented Oct 30, 2025

Uh oh!

coderabbitai bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Poem

Pre-merge checks

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pvijayakrish commented Oct 30, 2025

Uh oh!

Uh oh!

Uh oh!

KrishnanPrash left a comment

Choose a reason for hiding this comment

Uh oh!

pvijayakrish commented Oct 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pvijayakrish commented Oct 29, 2025 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Oct 30, 2025 •

edited

Loading