Datasetpublic

nihedb/EUR-Lex-Triples

EUR-Lex-Triples: A Legal Relation Extraction Dataset from European Legislation EUR-Lex-Sum dataset Aumiller, 2022 annotated with triples Relation Extraction Baselines Code/RE-Baselines contains the code used to run the RE baselines : Fine-Tuning and Inference. Results of baseline models for Relation Extraction are : Model Precision Recall F1-Score Legal-Bert 0.64 0.59 0.60 Bert 0.58 0.52 0.54 Rebel-Large 0.88 0.75 0.80 Mistral 7b zero-Shot 0.38… See the full description on the dataset page: https://huggingface.co/datasets/nihedb/EUR-Lex-Triples.

sourceHugging Faceupdated 1y agoView on Hugging Face

1likes27downloads

Dataset Card

EUR-Lex-Triples: A Legal Relation Extraction Dataset from European Legislation

EUR-Lex-Sum dataset Aumiller, 2022 annotated with triples

Dataset Description

—EUR-Lex-Triples consists on 1504 annotated documents. All Documents come from the english part of EUR-Lex-Sum Dataset.
—``Filtered_Annotated_Documents`` contains json files containing for each document its summary, the annotated paragraphs, and for each paragraph the triples derived from the annotations.

Relation Extraction Baselines

—``Code/RE-Baselines`` contains the code used to run the RE baselines : Fine-Tuning and Inference.
—Results of baseline models for Relation Extraction are :

Model	Precision	Recall	F1-Score
Legal-Bert	0.64	0.59	0.60
Bert	0.58	0.52	0.54
Rebel-Large	0.88	0.75	0.80
Mistral 7b zero-Shot	0.38	0.30	0.33
Mistral 7b In-Context	0.42	0.36	0.38
Mistral 7b Finetuning	0.84	0.69	0.75
Zephyr 7b Zero-Shot	0.40	0.36	0.37
Zephyr 7b In-Context	0.52	0.44	0.47
Zephyr 7b Finetuning	0.85	0.61	0.71
Llama 2 13b Zero-Shot	0.31	0.25	0.27
Llama 2 13b In-Context	0.33	0.29	0.30
Llama 2 13b Finetuning	0.82	0.61	0.69

Citation

EUR-Lex-Triples: A Legal Relation Extraction Dataset from European Legislation. Paper accepted at TPDL 2025.

Licence

Copyright for the editorial content of EUR-Lex website, the summaries of EU legislation and the consolidated texts owned by the EU, are licensed under the Creative Commons Attribution 4.0 International licence, i.e., CC BY 4.0 as mentioned on the official EUR-Lex website. Any data artifacts remain licensed under the CC BY 4.0 license.