auren-research/pii-shield

PII Shield: Multilingual PII Detection Dataset

<p align="center"> <img src="https://img.shields.io/badge/Languages-6-blue" alt="6 Languages"> <img src="https://img.shields.io/badge/Examples-2.6M+-green" alt="2.6M+ Examples"> <img src="https://img.shields.io/badge/PII_Types-40+-orange" alt="40+ PII Types"> <img src="https://img.shields.io/badge/License-CC--BY--4.0-lightgrey" alt="CC BY 4.0"> <img src="https://img.shields.io/badge/Domains-5+-purple" alt="5+ Domains"> </p>

PII Shield is a large-scale, multilingual dataset for training and evaluating Personally Identifiable Information (PII) detection models. Built by Auren Research, it combines real-world documents from diverse domains with high-quality span-level PII annotations produced by fastino/gliner2-privacy-filter-PII-multi — achieving the highest F1 on the SPY benchmark among open-source PII detectors.

The dataset is designed to support production-grade compliance workflows under GDPR, LGPD, CCPA, and related privacy regulations.

Dataset Overview

Motivation

Despite growing regulatory pressure around data privacy, high-quality multilingual PII detection datasets remain scarce. Most existing resources are:

• —English-only or cover only 2–3 languages
• —Limited to a single domain (e.g. clinical notes or emails)
• —Fully synthetic, with limited coverage of real-world PII distribution
• —Too small for training production-grade models

PII Shield addresses all four gaps: it is built from real documents across five domains, annotated with a state-of-the-art open-source PII detector, and expanded to six languages through a placeholder-preserving translation pipeline that ensures PII entities are never corrupted during translation.

Data Sources

The English portion of the dataset was collected and deduplicated from the following public sources:

After deduplication via exact-match on the text column, the final English set contains 531,729 documents.

Annotation Pipeline

All English documents were annotated using [fastino/gliner2-privacy-filter-PII-multi](https://huggingface.co/fastino/gliner2-privacy-filter-PII-multi), a GLiNER2-based multilingual PII detector that achieves the highest span-level F1 (0.477) on the SPY benchmark — outperforming OpenAI Privacy Filter, NVIDIA GLiNER-PII, and urchade/glinermultipii-v1.

Annotation settings

• —Detection threshold: 0.5
• —Max document length: 8,000 characters (longer documents truncated)
• —Hardware: AMD Instinct MI300X (192 GB)
• —Throughput: ~8,000–8,500 documents/second

PII entity types detected (40 types)

Annotation results

Translation Pipeline

The 516,880 documents containing PII were translated into 5 additional languages using [tencent/HY-MT1.5-1.8B](https://huggingface.co/tencent/HY-MT1.5-1.8B), a state-of-the-art multilingual machine translation model.

Placeholder-preserving translation

To prevent PII corruption during translation, we implemented a placeholder injection strategy:

1. 1.Before translation, all detected PII spans are replaced with positional tokens (__X0__, __X1__, ...)
2. 2.The document with placeholders is translated
3. 3.After translation, placeholders are replaced with the original PII values

This ensures that names, emails, phone numbers, and other identifiers are never mistranslated or altered.

Translation settings

• —Inference engine: vLLM 0.21.0
• —Hardware: AMD Instinct MI300X (192 GB)
• —Batch size: 512
• —Throughput: ~100–450 documents/second (varies by document length)
• —Max document length: 1,500 characters

Language distribution

Dataset Structure

Files

auren-research/pii-shield/
├── en/
│   └── auren_privacy_annotated.parquet    # English, annotated
├── pt/
│   └── auren_privacy_pt.parquet           # Portuguese translations
├── es/
│   └── auren_privacy_es.parquet           # Spanish translations
├── fr/
│   └── auren_privacy_fr.parquet           # French translations
├── de/
│   └── auren_privacy_de.parquet           # German translations
└── ar/
    └── auren_privacy_ar.parquet           # Arabic translations

Schema

English file (`auren_privacy_annotated.parquet`):

Translated files (`auren_privacy_{lang}.parquet`):

Usage

from datasets import load_dataset
import json

# Load English annotated data
ds = load_dataset("auren-research/pii-shield", "en", split="train")

# Load a specific language
ds_pt = load_dataset("auren-research/pii-shield", "pt", split="train")

# Filter only documents with PII
pii_docs = ds.filter(lambda x: x["has_pii"])

# Parse spans
example = ds[0]
spans = json.loads(example["spans"])
for span in spans:
    print(f"[{span['label']}] {span['text']}")

Intended Use Cases

• —Training PII detection / NER models for GDPR and LGPD compliance pipelines
• —Benchmarking multilingual PII detectors across real-world document types
• —Data governance tooling — redaction, pseudonymization, de-identification
• —Research on cross-lingual transfer of privacy-related information extraction
• —Fine-tuning encoder models (BERT, DeBERTa, ModernBERT) for span-level PII classification

Limitations

The following limitations should be considered before using this dataset in production:

Annotation limitations:

• —The annotation model (GLiNER2-PII) has known precision issues with person and full_name labels — it can confuse common nouns and organization names with personal names
• —IPv6 addresses, URLs, and device identifiers (e.g. IMEI numbers) may be misclassified as phone_number or missed entirely
• —Annotation confidence scores are not stored — all spans at threshold ≥ 0.5 are included

Translation limitations:

• —Machine translation quality varies by language and domain — Arabic and German may contain more errors than Portuguese and Spanish
• —Translations were not validated by human annotators
• —Documents were truncated to 1,500 characters before translation — very long documents may have incomplete translations

Coverage limitations:

• —All source documents are in English — the multilingual examples are translations, not natively authored text
• —Medical domain coverage is limited (Nemotron-PII provides some coverage, but MIMIC-III was not included)
• —No Asian languages (Chinese, Japanese, Korean, Hindi) in this release

Ethical Considerations

The English source documents contain real PII from real people, collected from publicly available datasets (Enron emails, legal proceedings, SEC filings). While these datasets are in the public domain or released under open licenses, users of PII Shield should:

• —Not use this dataset to build surveillance, tracking, or profiling systems
• —Not attempt to identify individuals from the PII spans
• —Follow the upstream licenses of each source dataset
• —Comply with applicable privacy laws in their jurisdiction when deploying models trained on this data

This dataset is intended for privacy-enhancing use cases only — building tools that detect and redact PII, not tools that exploit it.

Source Dataset Licenses

The compiled dataset is released under CC BY 4.0. Please cite the original sources if you use this dataset in academic work.

Citation

If you use PII Shield in your research or products, please cite:

@dataset{auren2026piishield,
  title   = {PII Shield: A Multilingual PII Detection Dataset},
  author  = {Barroso, Francisco Antonio Da Costa},
  year    = {2026},
  publisher = {Auren Research},
  url     = {https://huggingface.co/datasets/auren-research/pii-shield}
}

Please also cite the annotation and translation models:

@misc{fastino2026gliner2pii,
  title  = {GLiNER2-PII: Multilingual PII Extraction via Synthetic Fine-Tuning},
  author = {{Fastino AI Team}},
  year   = {2026},
  url    = {https://huggingface.co/fastino/gliner2-pii-v1}
}

@misc{tencent2025hymt,
  title  = {HY-MT: Tencent Hunyuan Machine Translation},
  author = {{Tencent Hunyuan Team}},
  year   = {2025},
  url    = {https://huggingface.co/tencent/HY-MT1.5-1.8B}
}

About Auren Research

Auren Research is an independent AI research lab focused on AI safety, privacy-preserving machine learning, and open-source model development. Our work includes:

• —Lunaris MoC — a novel sparse Transformer with mediator-based expert collaboration (technical report)
• —Lunaris Guard — a multilingual dual-head safety classifier (ROC-AUC 0.979 on prompt injection, 0.928 on content safety)
• —PII Shield — this dataset

All research is open-source and reproducible.

Built with ❤️ by [Francisco Antonio Da Costa Barroso](https://github.com/MeryylleA) · Auren Research · 2026