LeandroRibeiro/NormasTCU

NormasTCU

Overview

NormasTCU iis a dataset for Legal Information Retrieval (LIR) in Brazilian Portuguese composed of normative documents from the Brazilian Federal Court of Accounts (Tribunal de Contas da União - TCU), along with queries and human-annotated relevance judgments.

The dataset includes:

• —14,469 legal documents (normative acts);
• —46 queries;
• —812 judge query-document pairs derived from 3,048 human annotations with 3-level graded relevance.

Usage

To use the dataset, load que files docs.csv, query.csv, and qrel.csv:

import pandas as pd

docs = pd.read_csv("docs.csv")
queries = pd.read_csv("query.csv")
qrels = pd.read_csv("qrel.csv")

Dataset Structure

The NormasTCU dataset consists of three main files:

NormasTCU/
│── doc.csv               # Corpus
│── query.csv             # Queries
│── qrel.csv              # Relevance judgments for each query
│── raw_human_eval.csv    # Anonymized relevance judgments from each human annotator (before aggregation)

Documents (`doc.csv`)

The doc.csv file contains 16,045 legal documents from the TCU's selected jurisprudence. Each row represents a legal document and includes the following fields:

Fields of a Document in the Corpus

Queries (`query.csv`)

The query.csv file contains 46 standardized queries. Each row in the file contains the following fields:

Relevance Judgments (qrel.csv)

The qrel.csv file contains relevance assessments for query-document pairs.

Each row in the file contains the following fields:

Citation

If you use the NormasTCU dataset, please cite:

@article{normastcu,
    title={{NormasTCU} — A {B}razilian {P}ortuguese {IR} Dataset and an Evaluation of {LLM}-as-a-Judge for Relevance Assessment}, 
    author={Leandro Carísio Fernandes and Marcus Vinícius Borela de Castro and Leandro dos Santos Ribeiro and Leonardo Augusto da Silva Pacheco and Edans Flávius de Oliveira Sandes},
    journal = {Information Processing \& Management},
    volume = {64},
    number = {2, Part A},
    year = {2027},
    issn = {0306-4573},
    doi={10.1016/j.ipm.2026.105151}
}

License

NormasTCU is provided under the CC-BY 4.0 license.

Contact

• —Leandro Carísio Fernandes Câmara dos Deputados, Brasília, Brazil Email: carisio@gmail.com

• —Marcos Vinícius Borela de Castro Tribunal de Contas da União (TCU), Brasília, Brazil Email: borela@tcu.gov.br

• —Leandro dos Santos Ribeiro Tribunal de Contas da União (TCU), Brasília, Brazil Email: leandro.santos.r@gmail.com

• —Leonardo Augusto da Silva Pacheco Tribunal de Contas da União (TCU), Brasília, Brazil Email: leonardo3108@gmail.com

• —Edans Flávius de Oliveira Sandes Tribunal de Contas da União (TCU), Brasília, Brazil Email: edansfs@tcu.gov.br

Acknowledgments

We would like to thank the Brazilian Federal Court of Accounts (TCU) for providing the documents and supporting this research.