Skip to content

Test cleanup: consolidate Clojure comparison tests - #2400

Closed
jucor wants to merge 2 commits into
compdemocracy:jc/pca_sklearnfrom
jucor:pca_test_cleanup
Closed

jucor wants to merge 2 commits into
compdemocracy:jc/pca_sklearnfrom
jucor:pca_test_cleanup

Conversation

@jucor

@jucor jucor commented Feb 25, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Stacked on #2399 (sklearn PCA). Please review and merge #2396 → #2397 → #2398 → #2399 first.

To see only the diff added by this PR (excluding previous commits): compare pca_sklearn...pca_test_cleanup

Stack: #2396 (NaN fix) → #2397 (test infra) → #2398 (comparer) → #2399 (sklearn PCA) → This PR

Post-sklearn cleanup:

  • Replace PCA Clojure comparison test with pass (now covered by regression tests with projection metrics)
  • Consolidate Clojure comparison tests into `test_legacy_clojure_regression.py`, delete redundant `test_golden_data.py`
  • Change `@skip` markers to `@xfail` for tests with known differences

Test plan

  • 198 passed, 3 skipped, 4 xfailed

🤖 Generated with Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR consolidates Clojure comparison tests following the sklearn PCA implementation from PR #2399. The main goal is to clean up test infrastructure after validating that sklearn PCA matches the Clojure implementation.

Changes:

  • Replace PCA Clojure comparison test with pass (validation now covered by regression tests)
  • Consolidate Clojure comparison tests into test_legacy_clojure_regression.py and delete redundant test_golden_data.py
  • Change @skip markers to @xfail for tests with known differences
  • Add parallel test execution support with pytest-xdist grouping markers
  • Add --datasets CLI option for selective test execution
  • Standardize imports across test files to use conftest helpers

Reviewed changes

Copilot reviewed 21 out of 23 changed files in this pull request and generated no comments.

Show a summary per file
File Description
test_legacy_clojure_regression.py Add deprecated PCA comparison test (pass), convert skip to xfail markers, restructure as class-based tests
test_golden_data.py Deleted - redundant with test_legacy_clojure_regression.py
test_pca_unit.py Remove power iteration tests, keep only sklearn PCA tests for NaN handling and edge cases
test_pca_edge_cases.py Remove power iteration edge case tests, keep pca_project_dataframe edge cases
test_repness_smoke.py Standardize imports to use get_available_dataset_params from conftest
test_pca_smoke.py Standardize imports to use get_available_dataset_params from conftest
test_legacy_repness_comparison.py Standardize imports to use get_available_dataset_params from conftest
test_conversation_smoke.py Standardize imports to use get_available_dataset_params from conftest
test_regression.py Add ignore_pca_sign_flip=True to ConversationComparer initialization, add extensive tests for sign flip and scaling detection
test_conversation.py Add recompute=False flags to tests that only validate sorting/moderation (skip unnecessary PCA), add warning filters for sklearn PCA with minimal data
test_pipeline_integrity.py Standardize imports to use make_dataset_params from conftest
conftest.py Add parallel execution helpers (make_dataset_params, get_available_dataset_params), add --datasets CLI option, enhance test filtering
pyproject.toml Add pytest-xdist configuration (--dist=loadgroup) for parallel execution
pca.py Remove ~900 lines of power iteration code, replace with sklearn PCA (from PR #2399)
comparer.py Extensive enhancements for PCA sign flip handling, projection metrics, outlier tolerance
datasets.py Document that get_dataset_info always uses include_local=True
regression_recorder.py Add --include-local option
regression_comparer.py Add PCA-specific options (ignore-pca-sign-flip, outlier-fraction, loose tolerances)
bench_pca.py New benchmark script for PCA performance testing
README.md Document parallel execution and --datasets option
regression_testing.md New comprehensive documentation for regression testing system

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

jucor and others added 2 commits March 4, 2026 14:03
The sklearn PCA implementation has been validated to match Clojure.
Regressions are now caught by test_regression.py golden snapshots.

This test file will be removed once all other Clojure comparison
tests are similarly validated and covered by regression tests.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ion.py

- Delete test_golden_data.py: redundant with test_legacy_clojure_regression.py
- Change @Skip to @xfail for test_group_clustering and test_comment_priorities
  so they run and document expected failures rather than being silently skipped

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@jucor

jucor commented Mar 5, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #2417 (moved head branch to upstream for simpler stacked PR workflow).

@jucor jucor closed this Mar 5, 2026
@jucor
jucor deleted the pca_test_cleanup branch March 5, 2026 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants