Skip to content

Deep analysis of Python-Clojure discrepancies and fix plan - #2394

Closed
jucor wants to merge 7 commits into
compdemocracy:jc/kmeans_clustering_toolingfrom
jucor:kmeans_analysis_docs
Closed

jucor wants to merge 7 commits into
compdemocracy:jc/kmeans_clustering_toolingfrom
jucor:kmeans_analysis_docs

Conversation

@jucor

@jucor jucor commented Feb 25, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Stacked on #2393 (two-level clustering). Please review and merge #2396 → #2397 → #2398 → #2399 → #2400 → #2393 first.

To see only the diff added by this PR (excluding previous commits): compare kmeans_clustering_tooling...kmeans_analysis_docs

Documentation-only PR: deep analysis of Python vs Clojure discrepancies and a TDD fix plan.

Changes

  • Deep analysis documents (deep-analysis-for-julien/) comparing Python and Clojure implementations statement-by-statement
  • Consolidate CLAUDE.md documentation for the delphi project
  • Discrepancy fix plan (docs/PLAN_DISCREPANCY_FIXES.md) with prioritized list of fixes

Test plan

  • Documentation only — no code changes

🤖 Generated with Claude Code

@jucor
jucor force-pushed the kmeans_analysis_docs branch from 4c9a91d to 6695bec Compare February 25, 2026 09:43
@jucor
jucor marked this pull request as ready for review February 25, 2026 09:45
@jucor
jucor force-pushed the kmeans_analysis_docs branch from 6695bec to 1da2e1a Compare February 25, 2026 10:55
@jucor
jucor force-pushed the kmeans_analysis_docs branch 2 times, most recently from 686fca5 to 1f4f9f9 Compare February 25, 2026 23:04
@jucor
jucor requested a review from Copilot February 25, 2026 23:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands the Delphi (Python) math pipeline documentation around Python↔Clojure discrepancies and a fix plan, and also introduces significant supporting code/test/tooling changes to facilitate parity work (dataset selection, xdist grouping, Clojure comparison tooling, sklearn-based clustering/PCA-related adjustments).

Changes:

  • Add deep-dive discrepancy analysis + prioritized fix plan docs (deep-analysis-for-julien/, delphi/docs/*), and update delphi/CLAUDE.md testing guidance.
  • Update clustering to a Clojure-style two-level approach and add supporting comparer/CLI utilities.
  • Improve regression/testing ergonomics: dataset filtering (--datasets), xdist load-grouping, and several test updates/cleanups.

Reviewed changes

Copilot reviewed 49 out of 53 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
delphi/tests/test_repness_smoke.py Switch dataset parametrization to conftest helper; mark structure test xfail.
delphi/tests/test_regression.py Enable PCA sign-flip tolerance in regression comparisons; add comparer tests.
delphi/tests/test_pipeline_integrity.py Parametrize datasets via make_dataset_params for xdist grouping.
delphi/tests/test_pca_unit.py Remove obsolete power-iteration unit tests; add NaN/no-vote coverage for sklearn PCA path.
delphi/tests/test_pca_smoke.py Switch dataset parametrization to conftest helper.
delphi/tests/test_pca_edge_cases.py Remove legacy power-iteration edge-case tests; keep DataFrame PCA edge checks.
delphi/tests/test_math_pipeline_runs_e2e.py Update mock dataset directory name.
delphi/tests/test_legacy_repness_comparison.py Switch dataset parametrization to conftest helper.
delphi/tests/test_legacy_clojure_regression.py Refactor to class-scoped dataset parametrization + caching + shared comparer utilities.
delphi/tests/test_legacy_clojure_output.py Remove legacy direct Clojure-output analysis script/test file.
delphi/tests/test_golden_data.py Remove redundant golden-data comparison test file.
delphi/tests/test_edge_cases.py Update expectation for repness output with insufficient data.
delphi/tests/test_datasets.py Update DatasetInfo tests for cold-start blob support.
delphi/tests/test_conversation_smoke.py Switch dataset parametrization to conftest helper.
delphi/tests/test_conversation.py Avoid unnecessary recompute in sorting/moderation tests; adjust recompute test data; suppress PCA warnings.
delphi/tests/test_clusters.py Update init_clusters expectations when k > n_points.
delphi/tests/legacy_compare_with_clojure.py Remove legacy CLI-like comparer script.
delphi/tests/conftest.py Add --datasets filtering + xdist grouping helpers + improved collection filtering.
delphi/tests/README.md Document new pytest options and parallel execution workflow.
delphi/scripts/regression_recorder.py Add --include-local support and dataset discovery behavior improvements.
delphi/scripts/regression_comparer.py Add PCA sign-flip ignore option + include-local + outlier tolerance knobs.
delphi/scripts/clojure_comparer.py New CLI tool to compare Python outputs vs Clojure math_blob references.
delphi/pyproject.toml Add Click dependency; configure pytest to use --dist=loadgroup; add clojure_comparison marker.
delphi/polismath/regression/datasets.py Add cold-start math blob detection + prefer-cold-start selection in get_dataset_files.
delphi/polismath/regression/init.py Re-export Clojure comparer utilities from regression package.
delphi/polismath/pca_kmeans_rep/clusters.py Add sklearn KMeans + deterministic “first k distinct points” init + sklearn silhouette wrapper.
delphi/polismath/conversation/conversation.py Implement two-level clustering (base clusters + group clusters) and serialize base-clusters folded format.
delphi/polismath/benchmarks/bench_pca.py New PCA benchmark/profile script.
delphi/docs/SUBGROUP_CLUSTERING_THIRD_LEVEL.md Document Clojure’s (unimplemented-in-Python) third-level subgroup clustering.
delphi/docs/CLOJURE_TWO_LEVEL_CLUSTERING.md Document Clojure two-level clustering and storage formats.
delphi/docs/CLOJURE_COMPARISON.md Document Clojure comparison tests and CLI usage.
delphi/CLAUDE.md Update/clarify docs status and add testing/regression guidance.
deep-analysis-for-julien/01-overview-and-architecture.md New architecture overview doc.
deep-analysis-for-julien/02-pca-analysis.md New PCA discrepancy analysis doc.
deep-analysis-for-julien/03-clustering-analysis.md New clustering discrepancy analysis doc.
deep-analysis-for-julien/04-repness-analysis.md New repness discrepancy analysis doc.
deep-analysis-for-julien/05-participant-filtering.md New in-conv/comment-priorities analysis doc.
deep-analysis-for-julien/06-comment-routing.md New TypeScript comment-routing analysis doc.
deep-analysis-for-julien/07-discrepancies.md New consolidated discrepancy index doc.
deep-analysis-for-julien/08-dead-code.md New dead-code inventory doc.
deep-analysis-for-julien/09-fix-plan.md New TDD-oriented fix plan doc.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread delphi/CLAUDE.md Resolved
jucor and others added 6 commits February 26, 2026 00:06
Comprehensive 9-document analysis covering:
- Architecture and data flow comparison
- PCA implementation details (power iteration vs SVD)
- Two-level clustering with k-smoother analysis
- Representativeness metrics with formula verification
- Participant filtering and comment priority systems
- TypeScript comment routing (prioritized vs topical)
- 15 identified discrepancies rated by severity
- Dead code inventory across Python codebase
- Prioritized fix plan to bring Python to Clojure parity

Key findings: 5 CRITICAL discrepancies (in-conv threshold,
proportion test formula, repness metric, z-score thresholds,
missing comment priorities), 3 HIGH, and 7 MEDIUM/LOW.

https://claude.ai/code/session_01FEPFmVHKz1eoqzvXSTmu14
- Doc 02: Correct Clojure sparsity-aware projection code to match actual
  implementation (raw votes with nils skipped, not imputed). Add new
  discrepancy D1b documenting subtle projection input difference.
- Doc 03: Fix same-clustering convergence check description - it checks
  every pairwise distance < threshold, not sum.
- Doc 07: Add D1b projection input discrepancy (LOW severity).

https://claude.ai/code/session_01FEPFmVHKz1eoqzvXSTmu14
- Remove AGENTS.md and CLAUDE.md/GEMINI.md symlinks
- Create delphi/CLAUDE.md as a real file with upstream content plus
  testing, regression, and sklearn migration sections
- Add outdated docs warning
- Move uv-specific instructions to CLAUDE.local.md (gitignored)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jucor
jucor force-pushed the kmeans_analysis_docs branch from 51abc9f to 227fc05 Compare February 26, 2026 00:06
Comprehensive plan covering all 15 documented discrepancies (D1-D15)
between the Delphi Python math pipeline and the Clojure reference
implementation. Fixes are ordered by pipeline execution order with
one PR per fix, stacked for reviewability.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jucor
jucor force-pushed the kmeans_analysis_docs branch 5 times, most recently from b87aa23 to 74d276e Compare February 27, 2026 16:05
@jucor jucor closed this Mar 5, 2026
@jucor
jucor deleted the kmeans_analysis_docs branch March 5, 2026 12:41
@jucor
jucor restored the kmeans_analysis_docs branch March 5, 2026 12:43
@jucor jucor reopened this Mar 5, 2026
@jucor
jucor changed the base branch from edge to jc/kmeans_clustering_tooling March 5, 2026 12:47
@jucor

jucor commented Mar 5, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #2419 (moved head branch to upstream for simpler stacked PR workflow).

@jucor jucor closed this Mar 5, 2026
@jucor
jucor deleted the kmeans_analysis_docs branch March 5, 2026 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants