feat: Add Smart Anomaly Grouping

[ReginaldErzoah]

Summary

Add smart anomaly grouping to organize related drift and quality warnings together.

---

Motivation

Large comparisons can generate many warnings.

Without grouping, users may struggle to understand which issues are related or which ones matter most.

Smart anomaly grouping helps users identify related problems such as:

• numeric drift and outlier spikes in the same column
• null spikes and row changes
• categorical drift across related columns
• multiple warnings caused by the same upstream issue

---

Proposed Improvements

• Group related anomalies by column
• Group related anomalies by issue type
• Add grouped summaries to reports
• Improve readability of console and HTML outputs

---

Suggested Files

Potential implementation areas:

dift/core/stats_diff.py
dift/core/quality_diff.py
dift/core/risk.py
dift/reports/models.py
dift/reports/console_report.py
dift/reports/html_report.py

---

Suggested Tasks

• Define anomaly grouping rules
• Add anomaly grouping utility
• Group warnings by column and category
• Update console report output
• Update HTML report output
• Add tests
• Update documentation

---

How to Test

Run:

pytest
ruff check .

Run targeted tests:

pytest tests/test_stats_diff.py
pytest tests/test_quality_diff.py
pytest tests/test_reports.py

Manual validation:

dift examples/old_drift.csv examples/new_drift.csv --key id --report html --output report.html

Verify:

• related warnings are grouped clearly
• console report remains readable
• HTML report renders grouped anomalies correctly
• JSON output remains stable

---

Documentation Impact

Update:

docs/reports.md
docs/statistical-analysis.md
docs/developer/report-system.md

Documentation should include:

• what smart anomaly grouping does
• how grouped warnings are displayed
• examples of grouped anomalies
• reporting behavior across formats

---

Acceptance Criteria

• Related anomalies are grouped together
• Reports display grouped warnings clearly
• Existing warnings remain available
• Tests pass
• Documentation updated