Modelpublic

JQ1984/legalbert_gdpr_pretrained

sourceHugging Facecc-by-nc-4.0updated 1y agoView on Hugging Face

0likes10downloads

Model Card

Legal-BERT (GDPR Pretrained Version)

This model is based on `nlpaueb/legal-bert-base-uncased`, and has been further pretrained on the full text of the General Data Protection Regulation (GDPR) to adapt it to privacy law and regulatory compliance scenarios.

🧠 What’s New?

We adapted Legal-BERT through masked language modeling (MLM) on GDPR-specific language, enhancing the model’s understanding of:

—Personal data protection terms
—GDPR article structure
—Typical compliance language and risk descriptions

The training corpus includes official GDPR text, split into clean English sentences, formatted for MLM.

🔧 Intended Use

This specialized model is best suited for:

—GDPR compliance assistance
—Legal document classification and clause matching
—Privacy policy analysis
—Regulatory question answering (when further fine-tuned)

💾 Training Details

—Base model: nlpaueb/legal-bert-base-uncased
—Task: Masked Language Modeling (MLM)
—Corpus: Full official GDPR English text (~10,000+ sentences)
—Epochs: 3
—Block size: 128
—Batch size: 16
—MLM Probability: 15%

🛠 How to Use

python

from transformers import AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("JQ1984/legalbert_gdpr_pretrained")
model = AutoModelForMaskedLM.from_pretrained("JQ1984/legalbert_gdpr_pretrained")

# Example
inputs = tokenizer("The data controller shall ensure that personal data is", return_tensors="pt")
outputs = model(**inputs)


## References

* [Model Paper](https://arxiv.org/abs/xxxx.xxxxx)