fix(nli): return 1.0 for empty claims in score_faithfulness

[syf2211]

Description

Align EvidenceFinder.score_faithfulness() with the Faithfulness metric wrapper: when the claims list is empty, return a score of 1.0 (vacuously faithful) instead of 0.0.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring (no functional changes)
• Test update
• CI/CD update

Related Issues

Fixes #76

How Has This Been Tested?

• Unit tests pass (pytest tests/unit/test_metrics/test_nli.py)
• Linter passes (ruff check on changed files — pre-existing warnings only)
• Type checker passes (uv run mypy openagent_eval/)
• Manual testing performed

Test output

pytest tests/unit/test_metrics/test_nli.py -v
# 36 passed in 0.06s

pytest tests/unit/test_metrics/test_nli.py::TestEvidenceFinder::test_score_faithfulness_empty_claims -v
# PASSED

Checklist

• My code follows the project's coding standards
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

Additional Notes

The Faithfulness metric already short-circuits on empty extracted claims and returns score=1.0 before calling score_faithfulness(). This fix ensures direct callers of EvidenceFinder.score_faithfulness([], contexts) get the same semantics.

Summary: One-line behavioral fix + corresponding unit test assertion update.

Motivation: Inconsistent API — direct NLI module usage returned 0.0 for empty claims while the Faithfulness wrapper returned 1.0.

Changes:

• evidence_finder.py: return 0.0, [] → return 1.0, [] for empty claims
• test_nli.py: update test_score_faithfulness_empty_claims expectation

Risk: Low — only affects direct score_faithfulness([]) callers; no existing code depends on the old 0.0 behavior.

[himanshu231204]

/oc review pr

[opencode-agent]

Code Review: fix/evidence-finder-empty-claims-score

Summary

One-line semantic fix + test update. Changes score_faithfulness([], contexts) from returning 0.0 to 1.0 (vacuous truth for empty claims).

Files Checked

File	Lines
openagent_eval/metrics/nli/evidence_finder.py:119	0.0 → 1.0
tests/unit/test_metrics/test_nli.py:206	assert score == 0.0 → assert score == 1.0

Verdict: Approved ✅

Why: An empty set of claims is vacuously faithful — no claims can be unfaithful. Both the Faithfulness wrapper (in generation/) and now the raw NLI module agree on this semantics. The fix is minimal, correct, and well-tested.

Risk: Low. No downstream code depends on the old 0.0 return — the Faithfulness wrapper already short-circuited with 1.0 before calling this method.

No issues found.

opencode session  |  github run

[github-actions]

🎉 Congratulations @syf2211!

Your pull request has been successfully merged into main. 🚀

Thank you for contributing to OpenAgentHQ and helping improve the project.

We truly appreciate your contribution and hope to see you back with more amazing PRs!

Happy Open Sourcing! ❤️