Datasetpublic

MTEB-BR/juristcu-reranking

JurisTCUReranking Genuine legal-domain reranking: for each of 150 queries, rerank the ~100 first-stage candidates (BM25 top-100 over the full 16k-document TCU jurisprudence corpus, unioned with human-judged docs) under graded relevance 0-3. Candidates include lexical hard negatives, so the task is distinct from first-stage retrieval. Complements QuatiReranking (web) with a legal-domain reranking probe. Part of MTEB-BR — the native Brazilian-Portuguese MTEB sub-benchmark. Task… See the full description on the dataset page: https://huggingface.co/datasets/MTEB-BR/juristcu-reranking.

sourceHugging Facecc-by-4.0updated 2mo agoView on Hugging Face

0likes1.1kdownloads

Dataset Card

JurisTCUReranking

Genuine legal-domain reranking: for each of 150 queries, rerank the ~100 first-stage candidates (BM25 top-100 over the full 16k-document TCU jurisprudence corpus, unioned with human-judged docs) under graded relevance 0-3. Candidates include lexical hard negatives, so the task is distinct from first-stage retrieval. Complements QuatiReranking (web) with a legal-domain reranking probe.

Part of [MTEB-BR](https://huggingface.co/spaces/MTEB-BR/leaderboard) — the native Brazilian-Portuguese MTEB sub-benchmark. Task type: Reranking · Language: Brazilian Portuguese (native, human-created) · Domains: Legal, Government, Written.

Dataset structure

—`corpus` — columns: _id, title, text
—`qrels` — columns: query-id, corpus-id, score
—`queries` — columns: _id, text
—`top_ranked` — columns: query-id, corpus-ids

Source

https://huggingface.co/datasets/LeandroRibeiro/JurisTCU

License

cc-by-4.0

Citation

bibtex

@article{juristcu2026,
    author    = {Fernandes, Leandro Car{\'i}sio and
                 Ribeiro, Leandro dos Santos and
                 de Castro, Marcos Vin{\'i}cius Borela and
                 da Silva Pacheco, Leonardo Augusto and
                 de Oliveira Sandes, Edans Fl{\'a}vius},
    title     = {{JurisTCU: a Brazilian Portuguese information retrieval dataset with query relevance judgments}},
    journal   = {Language Resources and Evaluation},
    year      = {2026},
    volume    = {60},
    number    = {1},
    doi       = {10.1007/s10579-025-09881-w},
    url       = {https://doi.org/10.1007/s10579-025-09881-w},
    issn      = {1574-0218},
}

Repackaged for MTEB-BR; pinned by commit SHA in the task for reproducibility. arxiv.org/abs/2607.04581