Datasetpublic

Hyukkyu/beir-trec-covid

TRECCOVID — BEIR, unified schema A normalised copy of the dataset behind the mteb task TRECCOVID, one of the tasks of the BEIR benchmark as mteb defines it. Same queries, documents and relevance judgements as the benchmark evaluates — reshaped into one strict schema shared by every dataset in this collection. Source mteb/trec-covid @ bb9466bac815 (the revision pinned in mteb) Domain · languages biomedical · eng Queries / documents / qrels 50 / 171,332 / 66,336… See the full description on the dataset page: https://huggingface.co/datasets/Hyukkyu/beir-trec-covid.

sourceHugging Faceotherupdated 18d agoView on Hugging Face

0likes82downloads

Dataset Card

TRECCOVID — BEIR, unified schema

A normalised copy of the dataset behind the mteb task `TRECCOVID`, one of the tasks of the BEIR benchmark as mteb defines it. Same queries, documents and relevance judgements as the benchmark evaluates — reshaped into one strict schema shared by every dataset in this collection.


Source	`mteb/trec-covid` @ `bb9466bac815` (the revision pinned in `mteb`)
Domain · languages	biomedical · eng
Queries / documents / qrels	50 / 171,332 / 66,336
Qrels per query	min 631 · mean 1326.72 · max 1941
Score values	-1 ×2, 0 ×41,661, 1 ×10,456, 2 ×14,217
Layout	`queries` · `corpus` · `qrels`, split `test`
License	`cord-19`

Schema

config	columns	rules
`queries`	`id: string`, `text: string`	ids unique and non-empty; every query has ≥ 1 qrel
`corpus`	`id: string`, `title: string`, `text: string`	`title` is always present (`""` when the source has none)
`qrels`	`query-id: string`, `corpus-id: string`, `score: int32`	referential integrity to both tables; no duplicate pairs; no floats

Files are Parquet, sorted by id, zstd-compressed, sharded at 500 MB. Every rule above is enforced by a validator before publishing; provenance.json records the source file hashes, what changed, and the output file hashes.

What changed from the source

—byte-preserved all text — no whitespace, newline, or control-character normalisation
—cast qrels.score double -> int32
—kept 1 documents whose title and text are both empty — they are in the source and the benchmark's corpus
—kept 46,180 documents whose text is identical to another document's — a property of the source; note that models embed such documents identically and retrieval ties among them are broken by corpus order
—renamed _id → id
—renamed source splits (queries←queries/queries, corpus←corpus/corpus) to test

Load it

python

from datasets import load_dataset
queries = load_dataset("Hyukkyu/beir-trec-covid", "queries", split="test")
corpus  = load_dataset("Hyukkyu/beir-trec-covid", "corpus", split="test")
qrels   = load_dataset("Hyukkyu/beir-trec-covid", "qrels", split="test")

Cross-check against the previous layout

This repo previously held the same dataset in BEIR's raw two-repo form (Hyukkyu/beir-trec-covid + Hyukkyu/beir-trec-covid-qrels; that layout is kept under the git tag legacy-layout). Before replacing it, the build was compared against it: corpus id set identical (171,332 documents); qrels rows per split test 66,336 = 66,336.

License and attribution

The data is redistributed under the source's terms — cord-19. All credit belongs to the original authors; see the source repository and the references in mteb's task metadata (https://ir.nist.gov/covidSubmit/index.html). This repository is an independent repackaging and is not affiliated with the RTEB or MTEB maintainers.

License discrepancy. The upstream dataset card declares cord-19 while mteb's TaskMetadata declares not specified. This card carries the upstream value as the more conservative choice.