Repository navigation
test(score): add edge-case tests for score boundary conditions - #55
Conversation
Covers paths not exercised by the existing test_score.py: - score_next_event: all predictions miss top-3 (top3=0.0), truth at rank 3 (counts for top3 only), truth at rank 4 (misses top3) - score_conformance: perfect model/test overlap (fscore=1.0), single-transition perfect match - score_bottleneck: k < n_transitions with correct top-k, k=1 with correct top prediction - score_remaining_time: n=1 minimum input, zero-prediction model All 9 new tests pass; full suite: 328 passed, 0 failed.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 57c4cb3ac7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| preds = {("a", "b"): 10.0, ("c", "d"): 5.0, ("e", "f"): 1.0} | ||
| s = score_bottleneck(preds, truth, k=2) |
There was a problem hiding this comment.
If score_bottleneck regresses to calculating DCG over every transition instead of slicing at k, this test still passes because preds gives a perfect ranking across all three transitions. Use at least four transitions with the top two ordered correctly and the tail deliberately misordered; then the expected NDCG@2 remains 1.0 while an implementation that ignores the cutoff produces a lower score.
Useful? React with 👍 / 👎.
Why
tests/test_score.pycovers the happy path and several error pathsfor all five scoring functions, but a few boundary conditions are not
exercised. These gaps are easy to hit accidentally during a refactor
and would be caught immediately by the new tests.
What
New file
tests/test_score_edge.pywith 9 targeted tests:score_next_event: all predictions miss top-3 (top3=0.0); truth atexactly rank 3 (counts for top3, not top1); truth at rank 4 (misses
both)
score_conformance: perfect model/test overlap (fitness=precision=fscore=1.0); single-transition perfect match
score_bottleneck: k < n_transitions with a correct top-k ranking(NDCG@k=1.0); k=1 with the correct top prediction
score_remaining_time: n=1 minimum valid input; zero-predictionmodel (predictions can be 0.0)
Tests
tests/test_score_edge.py, 9 testsruff check tests/test_score_edge.py: cleanSelf-merge gate
pm_bench/__init__.py)tests/test_score_edge.py, 9 tests)🤖 Daily bot
Generated by Claude Code