Extend the code in utils.py to include validation for inputs, and potentially handling containers of inputs (mapping a TREC run file to either a list of lists, or a dictionary of lists, etc).
Validation needs to ensure that:
- Document ranks are obeyed -- ranks are assumed to be strictly increasing, but gaps are allowed for representing tied elements. For example, having a sequence of ranks like
[1, 1, 1, 4, 5] should be valid,
- Scores are used only as a diagnostic against provided ranks. If the ranks and scores do not agree, either warn or error.
Extend the code in
utils.pyto include validation for inputs, and potentially handling containers of inputs (mapping a TREC run file to either a list of lists, or a dictionary of lists, etc).Validation needs to ensure that:
[1, 1, 1, 4, 5]should be valid,