The problem / use case
Context: cli.py has import-squad and import-beir that emit docs.jsonl +
queries.jsonl with source-span gold. MS MARCO is a natural third target.
What: Add an import-msmarco subcommand that converts an MS MARCO passage/qrels
export into the same format.
Acceptance:
Pointers: mirror import-beir (passage-level relevance → span over the whole passage).
Proposed idea
Add import-msmarco alongside the SQuAD/BEIR importers so users can benchmark on a well-known passage-ranking dataset; its passage-level relevance imports as coarser (passage-span) gold, stated honestly.
The problem / use case
Context:
cli.pyhasimport-squadandimport-beirthat emitdocs.jsonl+queries.jsonlwith source-span gold. MS MARCO is a natural third target.What: Add an
import-msmarcosubcommand that converts an MS MARCO passage/qrelsexport into the same format.
Acceptance:
retrieval-lab import-msmarco … --out-dir …produces validdocs.jsonl+queries.jsonlBEIR import already frames it
tests/test_datasets.py;CHANGELOG.mdupdatedPointers: mirror
import-beir(passage-level relevance → span over the whole passage).Proposed idea
Add import-msmarco alongside the SQuAD/BEIR importers so users can benchmark on a well-known passage-ranking dataset; its passage-level relevance imports as coarser (passage-span) gold, stated honestly.