fix(pipeline): replace private-attribute and name-based provider coupling with public contracts

[Nitjsefnie]

Description

Two places where core/pipeline.py reached into provider internals instead of a declared contract. #51: the model name for metrics came from the private self._llm._model, silently falling back to the config value for any provider naming that attribute differently. #53: _retrieve branched on getattr(self._retriever, "name", None) == "mock" to decide whether to pass ground-truth contexts, so the core pipeline knew one retriever by name and no other retriever could ever opt in.

Both are fixed by adding the missing public contract rather than removing anything: a model_name property on the LLM provider base, and ground_truth_contexts as a real parameter on Retriever.retrieve().

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring (no functional changes)
• Test update
• CI/CD update

Related Issues

Closes #51
Closes #53

How Has This Been Tested?

Three regression tests, each verified to fail on the code it targets and pass after:

• test_pipeline_uses_public_model_name_property — a fake provider exposing model_name and not _model, with a model id deliberately different from the config's, so a silent fallback is distinguishable from a real read. Pre-fix: AssertionError: assert 'config-model' == 'provider-specific-model'.
• test_pipeline_passes_ground_truth_contexts_to_non_mock_retriever — asserts on the argument the retriever actually received. Pre-fix: AssertionError: assert None == ['ground truth context one', 'ground truth context two'].
• test_pipeline_keeps_legacy_retriever_working — a retriever with the old two-argument signature must still have its documents reach the result. Pre-fix (against the intermediate commit, before the compatibility fix): AssertionError: assert [] == ['legacy doc for What is RAG?'].

Behaviour was also probed directly, per case: a retriever declaring the parameter receives it; one accepting **kwargs receives it; a legacy retriever is called without it and returns its real documents; MockRetriever receives it and behaves exactly as before.

• Unit tests pass (uv run pytest) — full run 1030 passed, 4 skipped; coverage 79.60% against the 75% floor.
• Linter passes (uv run ruff check .) — exits 1 on the same pre-existing findings as main; no new finding on any file this PR touches.
• Type checker passes (uv run mypy openagent_eval/) — not run; CI's step targets a non-existent src/, which is CI type-check step targets a non-existent src/ directory, so mypy has never run on the package #250.
• Manual testing performed — the four retriever cases above, plus an instrumented run confirming the signature check is cached.

Checklist

• My code follows the project's coding standards
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation — none needed; both additions are backward-compatible defaults
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

Additional Notes

Why capability detection rather than just passing the new argument. Removing the "mock" check means the pipeline passes ground_truth_contexts to every retriever. All eleven retrievers in this repo were updated to accept it, so the suite goes green either way — but an out-of-tree Retriever subclass written against retrieve(self, query, k=5) would raise TypeError, and _retrieve's except Exception would absorb it silently, degrading that user's retrieval to the dataset fallback on every call with nothing logged. Since this is a published package, the pipeline now inspects the retriever's signature once, caches the result, and passes the keyword only when it is actually accepted. A retriever taking **kwargs counts as accepting it. Verified the inspection runs exactly once for a pipeline's lifetime, including under the parallel executor.

This is capability detection, not a name check — the pipeline still knows nothing about which retriever it holds, which is the point of #53.

Bug discovered while doing this work, filed separately:

• [BUG] pipeline: every retrieval failure is silently swallowed, so a broken retriever is indistinguishable from an empty result #256 [BUG] pipeline: every retrieval failure is silently swallowed, so a broken retriever is indistinguishable from an empty result

That one is pre-existing (pipeline.py:177 on main) and this PR does not change it — it only narrows what can raise inside the try. It is the reason the compatibility hazard above was worth handling rather than documenting: with the swallow in place, the failure mode is invisible. Happy to send the logging fix as a follow-up; #256 has the detail.

Known limitation, disclosed: if a retriever's retrieve is decorator-wrapped such that inspect.signature reports **kwargs while the underlying function does not accept the keyword, it will still raise into that same handler. And a callable whose signature cannot be introspected at all is treated as not supporting the parameter — safe, but currently silent. Both are noted in #256.

If you'd rather not carry the introspection and prefer to declare the new parameter simply required of all retrievers, that is a reasonable call for a pre-1.0 package and I'll strip it — your repo, your compatibility policy.

Generated by Claude Opus 5 (brief, review), Kimi K2.7 Code (implementation), Claude Sonnet 5 (verification)

[github-actions]

🎉 Congratulations @Nitjsefnie!

Your pull request has been successfully merged into main. 🚀

Thank you for contributing to OpenAgentHQ and helping improve the project.

We truly appreciate your contribution and hope to see you back with more amazing PRs!

Happy Open Sourcing! ❤️