Skip to content

chore: Exporters refactor#742

Open
marta-sd wants to merge 59 commits intomainfrom
martas/exporters-refactor
Open

chore: Exporters refactor#742
marta-sd wants to merge 59 commits intomainfrom
martas/exporters-refactor

Conversation

@marta-sd
Copy link
Contributor

No description provided.

@marta-sd marta-sd self-assigned this Feb 19, 2026
@copy-pr-bot
Copy link

copy-pr-bot bot commented Feb 19, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions bot added documentation Improvements or additions to documentation nemo-evaluator-launcher tests labels Feb 19, 2026
@marta-sd marta-sd force-pushed the martas/exporters-refactor branch 5 times, most recently from 4e7a050 to 5e33063 Compare February 27, 2026 09:54
@github-actions github-actions bot removed the documentation Improvements or additions to documentation label Feb 27, 2026
@marta-sd marta-sd force-pushed the martas/exporters-refactor branch from 31f5251 to c7b2292 Compare February 27, 2026 10:19
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
… output format

Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
…ith info cmd)

Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
…lflow and wandb

Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
…nel installation)

Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
@marta-sd marta-sd force-pushed the martas/exporters-refactor branch from c7b2292 to 65f0925 Compare February 27, 2026 10:20
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
@marta-sd marta-sd marked this pull request as ready for review February 27, 2026 10:38
@marta-sd marta-sd requested review from a team and agronskiy as code owners February 27, 2026 10:38
Copy link
Collaborator

@pribalta pribalta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Power Review

Reviewed 30 files, left 21 suggestions: 2 errors, 13 warnings, 6 suggestions.

Dimensions Analyzed

Dimension 🔴 ⚠️ 💡
anti_patterns 0 4 1
architecture 1 3 1
patterns 1 1 1
security-risks 0 3 0
testing-quality 0 2 3
Total 2 13 6

A pull request for exporters refactor arrives,
Our review engine carefully contrives.
Still, only you can see the bigger view,
So weigh these findings before you approve.


Automated review by Power Review — findings are advisory.

_, job_dir = prepare_local_job(jd, with_required=True, with_optional=True)
artifacts_dir = job_dir / "artifacts"
# Add required files
(artifacts_dir / "run_config.yml").write_text(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Power Review] 💡 Identical artifact file setup code duplicated across all TestLocalExporterManualScenarios methods · 88% confidence

Every test in TestLocalExporterManualScenarios (test_successful_export_multiple_jobs, test_partial_failure_missing_artifacts, test_non_existing_job_id, test_remote_job_export_with_copy, test_json_format_updates_existing_file) copies the same two write_text calls verbatim for run_config.yml and results.yml. This is the Duplicated Test Setup anti-pattern: shared setup should be extracted into a fixture or helper to reduce maintenance burden and the risk of individual tests drifting out of sync.

💡 Suggestion: Extract the artifact-file creation into a local helper, e.g.:

def _write_standard_artifacts(artifacts_dir: Path) -> None:
    (artifacts_dir / 'run_config.yml').write_text(
        'framework_name: test-harness\nconfig:\n  type: test_task\n'
        'target:\n  api_endpoint:\n    model_id: test-model\n'
    )
    (artifacts_dir / 'results.yml').write_text(
        'results:\n  tasks:\n    test_task:\n      metrics:\n'
        '        accuracy:\n          scores:\n            accuracy:\n              value: 0.9\nconfig: {}\n'
    )

Or add a pytest fixture to conftest.py.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO

inv = "test001"
artifacts_dir = tmp_path / inv / f"{inv}.0" / "artifacts"
artifacts_dir.mkdir(parents=True)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Power Review] 💡 Identical artifact directory setup duplicated across all MLflow exporter test methods · 85% confidence

Five tests (test_export_job_ok, test_export_with_skip_existing, test_export_with_update_existing, test_log_config_params_flattens_config, test_log_config_params_with_max_depth) each copy the same three-step pattern: mkdir the artifacts directory, write run_config.yml, write results.yml with identical content. This is textbook Duplicated Test Setup. Extracting it into a fixture would shorten each test and ensure the artifact format is consistent.

💡 Suggestion: Add a pytest fixture (possibly extending the existing prepare_local_job fixture in conftest.py) that creates and returns a pre-populated artifacts directory. Each test can then request this fixture instead of repeating the setup inline.

marta-sd added 3 commits March 4, 2026 12:24
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Signed-off-by: Marta Stepniewska-Dziubinska <martas@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants