joelniklaus/eurlex_resources

Dataset Card for EurlexResources: A Corpus Covering the Largest EURLEX Resources

Table of Contents

• —Table of Contents
• —Dataset Description
• —Dataset Summary
• —Supported Tasks and Leaderboards
• —Languages
• —Dataset Structure
• —Data Instances
• —Data Fields
• —Data Splits
• —Dataset Creation
• —Curation Rationale
• —Source Data
• —Annotations
• —Personal and Sensitive Information
• —Considerations for Using the Data
• —Social Impact of Dataset
• —Discussion of Biases
• —Other Known Limitations
• —Additional Information
• —Dataset Curators
• —Licensing Information
• —Citation Information
• —Contributions

Dataset Description

• —Homepage:
• —Repository: GitHub
• —Paper:
• —Leaderboard:
• —Point of Contact: Joel Niklaus

Dataset Summary

This dataset contains large text resources (~179GB in total) from EURLEX that can be used for pretraining language models.

Use the dataset like this:

from datasets import load_dataset
config = "de_caselaw" # {lang}_{resource}
dataset = load_dataset("joelito/eurlex_resources", config, split='train', streaming=True) 

Supported Tasks and Leaderboards

The dataset supports the task of masked language modeling.

Languages

The following languages are supported: bg, cs, da, de, el, en, es, et, fi, fr, ga, hr, hu, it, lt, lv, mt, nl, pl, pt, ro, sk, sl, sv

Dataset Structure

Data Instances

The file format is jsonl.xz and there is one split available ("train"). The following resource types are supported: caselaw, decision, directive, intagr, proposal, recommendation, regulation

More information about the resource types can be found here:

• —Caselaw: EU
• —Decision: EU, Wikipedia)
• —Directive: EU, Wikipedia)
• —Recommendation: EU, Wikipedia)
• —Regulation: EU, Wikipedia)
• —Intagr: EU, Wikipedia
• —Proposal: No resource found

Data Fields

[More Information Needed]

Data Splits

[More Information Needed]

Dataset Creation

Curation Rationale

[More Information Needed]

Source Data

Initial Data Collection and Normalization

The data has been downloaded using the R package eurlex between June and August 2022.

Who are the source language producers?

[More Information Needed]

Annotations

Annotation process

[More Information Needed]

Who are the annotators?

[More Information Needed]

Personal and Sensitive Information

[More Information Needed]

Considerations for Using the Data

Social Impact of Dataset

[More Information Needed]

Discussion of Biases

[More Information Needed]

Other Known Limitations

[More Information Needed]

Additional Information

Dataset Curators

[More Information Needed]

Licensing Information

CC BY 4.0 see also the legal notice

Citation Information

[More Information Needed]

Contributions

Thanks to @JoelNiklaus for adding this dataset.