Datasetpublic

oliverkinch/eur-lex-sum

EUR-Lex-Sum A dataset of EU legislation paired with legislative summaries from EUR-Lex, covering Danish and English. Built from the EU Publications Office CELLAR repository using SPARQL-based discovery and XHTML content extraction. Dataset Description Each record pairs a full EU legislative document with its official plain-language summary from the EUR-Lex Summaries of EU Legislation collection. Configs Config Records Description all… See the full description on the dataset page: https://huggingface.co/datasets/oliverkinch/eur-lex-sum.

sourceHugging Facecc-by-4.0updated 6mo agoView on Hugging Face

1likes14downloads

Dataset Card

EUR-Lex-Sum

A dataset of EU legislation paired with legislative summaries from EUR-Lex, covering Danish and English.

Built from the EU Publications Office CELLAR repository using SPARQL-based discovery and XHTML content extraction.

Dataset Description

Each record pairs a full EU legislative document with its official plain-language summary from the EUR-Lex Summaries of EU Legislation collection.

Configs

Config	Records	Description
`all` (default)	1,605	Bilingual DA+EN — every record has all four text fields
`da`	1,632	Danish only — `celex_id`, `da_document`, `da_summary`
`en`	1,619	English only — `celex_id`, `en_document`, `en_summary`

The all config is the intersection of da and en.

Usage

python

from datasets import load_dataset

# Load default (bilingual) config
ds = load_dataset("oliverkinch/eur-lex-sum")

# Load Danish only
ds_da = load_dataset("oliverkinch/eur-lex-sum", "da")

# Load English only
ds_en = load_dataset("oliverkinch/eur-lex-sum", "en")

Fields

`all` config:

Field	Type	Description
`celex_id`	`string`	CELEX identifier for the legislation
`da_document`	`string`	Full legislative text in Danish
`da_summary`	`string`	Plain-language summary in Danish
`en_document`	`string`	Full legislative text in English
`en_summary`	`string`	Plain-language summary in English

`da` config: celex_id, da_document, da_summary

`en` config: celex_id, en_document, en_summary

Statistics

	DA	EN
Records	1,632	1,619
Mean document tokens	22,706	24,758
Median document tokens	8,304	9,187
Mean summary tokens	910	986
Median compression ratio	10.1x	10.4x

Dataset Construction

Source

Documents and summaries were obtained from the EU Publications Office CELLAR repository via:

1.SPARQL discovery — querying the CELLAR endpoint to find legislation with linked legislative summaries available as XHTML in Danish (and optionally English).
2.Content extraction — fetching XHTML manifestations and extracting body text.

Filtering

The following filters were applied per language:

1.Availability — remove records where document or summary content is missing (HTTP 404 from CELLAR).
2.PDF scan artifacts — remove records containing [NEW PAGE] markers (residual from scanned PDFs).
3.Deduplication — when multiple CELEX IDs share an identical summary, keep the record with the longest document.
4.Short document removal — remove records where the document is shorter than or equal to the summary (by whitespace token count).

Text Cleaning

—Non-breaking spaces (\xa0) normalized to regular spaces.
—Consecutive newlines collapsed into paragraph boundaries.
—Blank lines and .xml identifier lines removed.

Pipeline

Starting from 2,838 discovered CELEX IDs:

Step	DA	EN
After availability filter	2,536	2,521
After scan filter	2,536	2,521
After deduplication	1,701	1,684
After short-document filter	1,632	1,619
Bilingual intersection	1,605	1,605

Related Datasets

—dennlinger/eur-lex-sum — the original EUR-Lex-Sum dataset covering 24 EU languages. This dataset uses a similar methodology but sources data from CELLAR (the original EUR-Lex scraping endpoint is now behind WAF bot protection).

License

The legislative documents and summaries are sourced from EUR-Lex. EU legal documents are available under the reuse policy of the European Commission (CC BY 4.0 compatible).