Datasetpublic

MTEB-BR/quati-reranking

QuatiReranking Genuine web-domain reranking: for each of 50 PT-BR queries, rerank the ~100 first-stage candidates (BM25 top-100 over the full 1M-passage Quati Brazilian web corpus, unioned with human-judged passages) under graded relevance 0-3. Candidates include lexical hard negatives, so the task is distinct from first-stage retrieval. Complements JurisTCUReranking (legal) with a web-domain probe. Part of MTEB-BR — the native Brazilian-Portuguese MTEB sub-benchmark. Task type:… See the full description on the dataset page: https://huggingface.co/datasets/MTEB-BR/quati-reranking.

sourceHugging Facecc-by-4.0updated 2mo agoView on Hugging Face

0likes1kdownloads

Dataset Card

QuatiReranking

Genuine web-domain reranking: for each of 50 PT-BR queries, rerank the ~100 first-stage candidates (BM25 top-100 over the full 1M-passage Quati Brazilian web corpus, unioned with human-judged passages) under graded relevance 0-3. Candidates include lexical hard negatives, so the task is distinct from first-stage retrieval. Complements JurisTCUReranking (legal) with a web-domain probe.

Part of [MTEB-BR](https://huggingface.co/spaces/MTEB-BR/leaderboard) — the native Brazilian-Portuguese MTEB sub-benchmark. Task type: Reranking · Language: Brazilian Portuguese (native, human-created) · Domains: Web, Written.

Dataset structure

—`corpus` — columns: _id, title, text
—`qrels` — columns: query-id, corpus-id, score
—`queries` — columns: _id, text
—`top_ranked` — columns: query-id, corpus-ids

Source

https://aclanthology.org/2024.stil-1.19/

License

cc-by-4.0

Citation

bibtex

@inproceedings{bueno-etal-2024-quati,
    title = "{Q}uati: A {B}razilian {P}ortuguese Information Retrieval Dataset from Native Speakers",
    author = "Bueno, Mirelle and Maia Sanches, Eduardo and Lotufo, Roberto and Nogueira, Rodrigo",
    booktitle = "Proceedings of the 16th Symposium in Information and Human Language Technology (STIL 2024)",
    year = "2024",
    url = "https://aclanthology.org/2024.stil-1.19/",
}

Repackaged for MTEB-BR; pinned by commit SHA in the task for reproducibility. arxiv.org/abs/2607.04581