[docs] Add Google-style docstrings for dspy/evaluate/metrics.py #8954
Conversation
dspy/evaluate/metrics.py
Outdated
|
|
||
| def EM(prediction, answers_list): # noqa: N802 | ||
| assert isinstance(answers_list, list) | ||
| """Return True if any reference exactly matches the prediction (after normalization). |
There was a problem hiding this comment.
The opening line should describe what this API is, instead of the API's behavior.
dspy/evaluate/metrics.py
Outdated
| otherwise False. | ||
| Example: | ||
| >>> EM("The Eiffel Tower", ["Eiffel Tower", "Louvre"]) |
There was a problem hiding this comment.
This doesn't render on mkdocs, let's use the block style, e.g.,:
my_code
dspy/evaluate/metrics.py
Outdated
| >>> EM("paris", ["Paris"]) | ||
| True | ||
| """ | ||
| assert isinstance(answers_list, list) |
There was a problem hiding this comment.
let's don't mix fix with docstring changes. Actually this assert statement won't provide more information to users
dspy/evaluate/metrics.py
Outdated
| >>> round(F1("Eiffel Tower is in Paris", ["Paris"]), 2) | ||
| 0.33 | ||
| """ | ||
| assert isinstance(answers_list, list) |
dspy/evaluate/metrics.py
Outdated
| float: The highest HotpotQA-style F1 score in [0.0, 1.0]. | ||
| Example: | ||
| >>> HotPotF1("yes", ["no"]) |
dspy/evaluate/metrics.py
Outdated
|
|
||
| def remove_articles(text): | ||
| return re.sub(r"\b(a|an|the)\b", " ", text) | ||
| return re.sub(r"\\b(a|an|the)\\b", " ", text) |
There was a problem hiding this comment.
do we need to change this?
dspy/evaluate/metrics.py
Outdated
| def answer_exact_match(example, pred, trace=None, frac=1.0): | ||
| """Example/Prediction evaluator for answer strings with EM/F1 thresholding. | ||
| If ``example.answer`` is a string, compare ``pred.answer`` against it. |
There was a problem hiding this comment.
nit: single backtick around variables: example.answer
dspy/evaluate/metrics.py
Outdated
|
|
||
|
|
||
| def answer_exact_match(example, pred, trace=None, frac=1.0): | ||
| """Example/Prediction evaluator for answer strings with EM/F1 thresholding. |
There was a problem hiding this comment.
This is too detailed for the open sentence
…k examples); revert non-doc changes
|
All feedback addressed (concise openings + mkdocs block examples) |
2 participants
This PR adds Google-style docstrings to public metrics in
dspy/evaluate/metrics.py.resolve #8953
cc @chenmoneygithub