Datasetpublic

Hyukkyu/beir-cqadupstack-programmers

CQADupstackProgrammersRetrieval — BEIR, unified schema A normalised copy of the dataset behind the mteb task CQADupstackProgrammersRetrieval, one of the tasks of the BEIR benchmark as mteb defines it (a member of the aggregate task CQADupstackRetrieval). Same queries, documents and relevance judgements as the benchmark evaluates — reshaped into one strict schema shared by every dataset in this collection. Source mteb/cqadupstack-programmers @ 6184bc1440d2 (the… See the full description on the dataset page: https://huggingface.co/datasets/Hyukkyu/beir-cqadupstack-programmers.

sourceHugging Facecc-by-sa-4.0updated 14d agoView on Hugging Face

0likes38downloads

Dataset Card

CQADupstackProgrammersRetrieval — BEIR, unified schema

A normalised copy of the dataset behind the mteb task `CQADupstackProgrammersRetrieval`, one of the tasks of the BEIR benchmark as mteb defines it (a member of the aggregate task CQADupstackRetrieval). Same queries, documents and relevance judgements as the benchmark evaluates — reshaped into one strict schema shared by every dataset in this collection.


Source	`mteb/cqadupstack-programmers` @ `6184bc1440d2` (the revision pinned in `mteb`)
Domain · languages	stackexchange · eng
Queries / documents / qrels	876 / 32,176 / 1,675
Qrels per query	min 1 · mean 1.912 · max 149
Score values	1 ×1,675
Layout	`queries` · `corpus` · `qrels`, split `test`
License	`cc-by-sa-4.0`

Schema

config	columns	rules
`queries`	`id: string`, `text: string`	ids unique and non-empty; every query has ≥ 1 qrel
`corpus`	`id: string`, `title: string`, `text: string`	`title` is always present (`""` when the source has none)
`qrels`	`query-id: string`, `corpus-id: string`, `score: int32`	referential integrity to both tables; no duplicate pairs; no floats

Files are Parquet, sorted by id, zstd-compressed, sharded at 500 MB. Every rule above is enforced by a validator before publishing; provenance.json records the source file hashes, what changed, and the output file hashes.

What changed from the source

—byte-preserved all text — no whitespace, newline, or control-character normalisation
—cast qrels.score double -> int32
—kept 3 documents whose text is identical to another document's — a property of the source; note that models embed such documents identically and retrieval ties among them are broken by corpus order
—renamed _id → id
—renamed source splits (queries←queries/queries, corpus←corpus/corpus) to test

Load it

python

from datasets import load_dataset
queries = load_dataset("Hyukkyu/beir-cqadupstack-programmers", "queries", split="test")
corpus  = load_dataset("Hyukkyu/beir-cqadupstack-programmers", "corpus", split="test")
qrels   = load_dataset("Hyukkyu/beir-cqadupstack-programmers", "qrels", split="test")

License and attribution

The data is redistributed under the source's terms — cc-by-sa-4.0. All credit belongs to the original authors; see the source repository and the references in mteb's task metadata (http://nlp.cis.unimelb.edu.au/resources/cqadupstack/). This repository is an independent repackaging and is not affiliated with the RTEB or MTEB maintainers.

License discrepancy. The upstream dataset card declares cc-by-sa-4.0 while mteb's TaskMetadata declares apache-2.0. This card carries the upstream value as the more conservative choice.