gorkem371/pii-intent-classifier-xlmr-large

PII Intent Classifier - XLM-RoBERTa Large (V11)

A multilingual binary classifier that detects PII (Personally Identifiable Information) sharing intent in text messages. Built for content moderation on creator-brand collaboration platforms.

What's New in V11

V11 is the 15th iteration of this model, trained on 41,427 samples (up from 24,012 in V7c). Key improvements:

• —Conversation test: 97.0% accuracy (838/864) - up from 95.0% in V10
• —Targeted training data: ~200 new samples addressing specific failure patterns (scam+real phone, asking about numbers, humorous sharing, room/postal/tracking numbers as NOT-PII)
• —6 new CoT categories: roomnumber, postalcode, timedigits, trackingnumber, usernamedigits, goodnightcasual
• —Strict policy: Any real phone/email/IBAN/handle = PII regardless of context (humor, scam warning, inquiry)

Version History

Model Description

This model classifies whether a message contains an intent to share personal contact information (phone numbers, emails, social media handles, IBANs, etc.) or not. Unlike simple regex-based PII detection, this model understands context and intent:

• —"my number is 05321234567" → PII (sharing intent)
• —"05321234567 is a scammer, block them" → NOT PII (scam warning)
• —"I'll send you my WhatsApp tomorrow" → PII (future sharing intent)
• —"what is an IBAN and how do I get one?" → NOT PII (information question)
• —"numaram pizza siparişi gibi 05321234567 haha 😂" → PII (humor + real number)
• —"oda numaram 532 otelde buluşalım" → NOT PII (room number, not phone)

Key Features

• —Trilingual: Turkish, Arabic, English
• —Context-aware: Understands sarcasm, negation, hypotheticals, quoting, reporting, humor
• —Evasion-resistant: Detects coded sharing, profile redirects, voice note evasion, partial number sharing, spaced text evasion
• —High recall: 97% PII recall across 177 stress test cases
• —Conversation-ready: 97% accuracy on 864 real-world conversation scenarios

Training

• —Base model: xlm-roberta-large (550M parameters)
• —Dataset: gorkem371/pii-intent-detection-multilingual - 41,427 samples across 9 entity types, balanced PII/NOT-PII
• —Loss: Focal loss (gamma=2) with inverse class frequency weights
• —Training: bf16 mixed precision, lr=1.5e-5, gradient accumulation=4 (effective batch=64), 15 epochs
• —Hardware: NVIDIA H100 80GB HBM3
• —Best epoch: 14

Validation Metrics (epoch 14)

Test Results

Stress Test (177 cases)

Conversation Test (864 cases)

Per-Language Breakdown (Stress Test)

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
import torch.nn.functional as F

model_name = "gorkem371/pii-intent-classifier-xlmr-large"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
model.eval()

def classify_pii(context: str, entity: str, entity_type: str) -> dict:
    """
    Classify whether a message contains PII sharing intent.

    Args:
        context: The full message text
        entity: The specific entity to classify (e.g., phone number, "NONE" if implicit)
        entity_type: Type of entity (PHONE, EMAIL, SOCIAL_MEDIA, IBAN, ADDRESS, URL, etc.)

    Returns:
        dict with 'is_pii' (bool) and 'confidence' (float)
    """
    text = f"{context} </s> {entity} | {entity_type}"
    inputs = tokenizer(text, max_length=256, padding="max_length", truncation=True, return_tensors="pt")

    with torch.no_grad():
        outputs = model(**inputs)
        probs = F.softmax(outputs.logits, dim=-1)
        pred = torch.argmax(probs, dim=-1).item()
        confidence = probs[0][pred].item()

    return {
        "is_pii": pred == 1,
        "label": "PII" if pred == 1 else "NOT_PII",
        "confidence": round(confidence, 4)
    }

# Examples
print(classify_pii("my number is 05321234567 call me", "05321234567", "PHONE"))
# {'is_pii': True, 'label': 'PII', 'confidence': 0.9987}

print(classify_pii("order number is ORD-784321", "ORD-784321", "PHONE"))
# {'is_pii': False, 'label': 'NOT_PII', 'confidence': 0.9954}

print(classify_pii("i will send you my whatsapp tomorrow", "NONE", "PHONE"))
# {'is_pii': True, 'label': 'PII', 'confidence': 0.9821}

print(classify_pii("oda numaram 532 otelde buluşalım", "NONE", "PHONE"))
# {'is_pii': False, 'label': 'NOT_PII', 'confidence': 0.9876}

Input Format

The model expects input in the following format:

{context} </s> {entity} | {entity_type}

• —context: The full message text (any language)
• —entity: The specific entity string, or "NONE" for implicit PII intent
• —entity_type: One of: PHONE, EMAIL, SOCIAL_MEDIA, IBAN, CREDIT_CARD, ADDRESS, URL, CRYPTO_ADDRESS, OFF_PLATFORM_ATTEMPT

Supported Entity Types

What This Model Understands

PII = True (sharing intent detected)

• —Direct sharing: "my number is 05321234567"
• —Coded/evasion: "find me on the gram @secret_handle"
• —Future intent: "I'll send you my number tomorrow"
• —Conditional: "if we agree, I'll share my contact"
• —Requesting: "what's your number? send it"
• —Profile redirect: "check my bio, my number is there"
• —Reluctant sharing: "I don't want to but here's my number..."
• —Third-party: "my friend said to contact him at..."
• —Humor + real number: "numaram pizza siparişi gibi 05321234567 haha 😂"
• —Scam warning + real number: "05321234567 dolandırıcı sakın aramayın" (number is still visible)
• —Business/restaurant numbers: "restoran telefonu 02125551234"
• —Asking about a number: "bu numara tanıdık mı 05321234567"

PII = False (no sharing intent)

• —Order/tracking numbers: "your order ORD-784321"
• —Scam warnings (no real number): "dolandırıcılara dikkat edin"
• —Reporting violations: "someone sent me their number, reporting"
• —Hypothetical (no number): "I wish I had a number to share"
• —Sarcasm with fake numbers: "my number is 00000000000 lol"
• —Statistics: "my follower count hit 532000"
• —Non-contact numbers: "bake at 180 degrees for 45 minutes"
• —Price/product codes: "SKU-TR-78431-B", "1250 TL"
• —Room numbers: "oda numaram 532 otelde buluşalım"
• —Postal codes: "posta kodu 34720 Kadıköy İstanbul"
• —Time digits: "saat 05:32 de buluşalım"
• —Tracking numbers: "kargo takip: 1Z999AA10123456784"

Limitations

• —Masked numbers (0532***4567): Model classifies as NOT-PII (partially masked = not fully usable)
• —Scam warnings with real numbers: V11 tends to flag these as PII (the number is still visible/reachable)
• —Math expressions containing phone-like numbers: Sometimes flagged as PII
• —Entity extraction dependency: 19 of 26 conversation errors are from entity extraction returning NONE, not model failures

Model Architecture

• —Base: XLM-RoBERTa Large (24 layers, 16 heads, 1024 hidden, 550M params)
• —Head: Linear(1024→1024) + Tanh + Dropout + Linear(1024→2)
• —Total size: ~2.1GB (safetensors)

Citation

Author: Gorkem Yildiz

If you use this model, please cite:

@misc{pii-intent-classifier-2026,
  title={PII Intent Classifier: Multilingual Context-Aware PII Detection},
  author={Gorkem Yildiz},
  year={2026},
  url={https://huggingface.co/gorkem371/pii-intent-classifier-xlmr-large},
  howpublished={\url{https://gorkemyildiz.com}}
}