feat: Add MongoDB Aggregation Pipeline Comparison Support

[ReginaldErzoah]

Summary

Add support for comparing MongoDB aggregation pipeline outputs.

---

Motivation

Many MongoDB workflows use aggregation pipelines rather than raw collection reads.

Supporting aggregation pipeline outputs allows users to validate:

• transformed document views
• analytics aggregations
• filtered datasets
• application-specific MongoDB queries

---

Proposed Improvements

• Support aggregation pipeline execution
• Load aggregation results into Dift
• Compare aggregation output between two sources
• Provide clear configuration or URI-based workflow

---

Suggested Files

Potential implementation areas:

dift/io/mongodb_reader.py
dift/io/config_loader.py
dift/core/comparator.py
tests/test_mongodb_reader.py
tests/test_mongodb_pipeline.py
docs/connectors/mongodb.md

---

Suggested Tasks

• Define how aggregation pipelines should be provided
• Add pipeline execution support
• Add validation for invalid pipelines
• Add tests using mock MongoDB responses
• Add documentation examples

---

How to Test

Run:

pytest
ruff check .

Run targeted tests:

pytest tests/test_mongodb_pipeline.py

Manual validation example using config:

dift --config examples/config_mongodb_pipeline.yaml

Example config concept:

old_dataset: mongodb://localhost:27017/shop.orders
new_dataset: mongodb://localhost:27017/shop.orders_new
key: order_id

mongodb:
  pipeline:
    - $match:
        status: completed

Verify:

• aggregation results load successfully
• invalid pipelines show helpful errors
• reports remain valid
• standard collection reads still work

---

Documentation Impact

Update:

docs/connectors/mongodb.md
docs/configuration.md
docs/examples.md

Documentation should include:

• aggregation pipeline examples
• config-based workflow
• validation behavior
• limitations and caveats

---

Acceptance Criteria

• Aggregation pipeline outputs can be compared
• Invalid pipelines produce helpful errors
• Tests pass
• Documentation updated