vineeth453/qwen25-7b-pii-detection

Qwen2.5-7B PII Detection — Merged Model

Standalone merged model — no PEFT library required. Load and run directly with transformers. For the lightweight adapter version (160MB vs 15GB): vineeth453/qwen25-7b-pii-detection-lora

LoRA adapter weights merged into `Qwen/Qwen2.5-7B-Instruct` after fine-tuning on ai4privacy/pii-masking-200k. Extracts 56 types of personally identifiable information across 4 languages and returns structured JSON output.

Built as the PII Detection component of a Phase 1 Input Guardrail gateway for an enterprise LLM security system.

Evaluation Results

Evaluated on 10,464 held-out samples (5% split from ai4privacy/pii-masking-200k).

Per-Entity F1 Scores

Note on IP label (F1=0.390): The dataset contains three overlapping IP labels (IP, IPV4, IPV6). The low recall on IP is due to the model correctly identifying the address but tagging it as IPV4 or IPV6 — a label ambiguity in the dataset, not a detection failure.

How to Get Started

Installation

pip install transformers accelerate torch
# No PEFT required for this merged model

Load and Run Inference

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch, json

model = AutoModelForCausalLM.from_pretrained(
    "vineeth453/qwen25-7b-pii-detection",
    device_map="auto",
    torch_dtype=torch.bfloat16,   # use bfloat16 for A100/H100; float16 for older GPUs
)
tokenizer = AutoTokenizer.from_pretrained("vineeth453/qwen25-7b-pii-detection")
model.eval()

def detect_pii(text: str) -> dict:
    prompt = (
        "<|im_start|>system\n"
        "You are a PII detection system. Extract all personally identifiable information.\n"
        'Return ONLY valid JSON: {"entities":[{"text":"...","label":"..."}]}\n'
        "<|im_end|>\n"
        "<|im_start|>user\n"
        f"{text}\n"
        "<|im_end|>\n"
        "<|im_start|>assistant\n"
    )
    inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
    with torch.no_grad():
        out = model.generate(
            **inputs,
            max_new_tokens=200,
            do_sample=False,
            pad_token_id=tokenizer.eos_token_id
        )
    response = tokenizer.decode(
        out[0][inputs["input_ids"].shape[1]:],
        skip_special_tokens=True
    ).strip().replace("<|im_end|>", "")
    return json.loads(response)

# English
print(detect_pii("Contact John Smith at john@example.com or call +1-555-867-5309"))
# {"entities": [{"text": "John", "label": "FIRSTNAME"}, {"text": "Smith", "label": "LASTNAME"},
#               {"text": "john@example.com", "label": "EMAIL"}, {"text": "+1-555-867-5309", "label": "PHONENUMBER"}]}

# German
print(detect_pii("Patient Lena Müller, born 14.03.1987, lives at Hauptstraße 22, Berlin."))
# {"entities": [{"text": "Lena", "label": "FIRSTNAME"}, {"text": "Müller", "label": "LASTNAME"},
#               {"text": "14.03.1987", "label": "DOB"}, {"text": "Hauptstraße", "label": "STREET"},
#               {"text": "22", "label": "BUILDINGNUMBER"}, {"text": "Berlin", "label": "STATE"}]}

# French
print(detect_pii("Merci de contacter Marie Dupont à marie.dupont@societe.fr avant le 30 mars."))

Memory Requirements

For GPU-constrained environments, use the adapter version with 4-bit quantization instead.

Training Details

Training Data

• —Dataset: ai4privacy/pii-masking-200k
• —Size: 209,261 samples (198,797 train / 10,464 val, 95/5 split)
• —Languages: English (43k), French (62k), German (53k), Italian (51k)
• —Entity types: 56 PII categories

Training Hyperparameters

Uses

Direct Use

Enterprise input guardrail systems for detecting and redacting PII from user queries before they reach an LLM. Suitable for HR, legal, healthcare, and financial applications.

Downstream Use

• —PII redaction pipelines
• —Compliance auditing tools (GDPR, CCPA, HIPAA)
• —Data anonymization workflows
• —Pre-processing layer in RAG or LLM gateway systems

Out-of-Scope Use

• —Real-time very high-throughput inference without GPU (15GB model, CPU too slow)
• —Languages outside EN/FR/DE/IT without further fine-tuning
• —Should not be the sole PII detection mechanism in high-stakes settings without human review

Bias, Risks, and Limitations

• —IP label ambiguity: Model occasionally routes IP addresses to IPV4/IPV6 instead of IP. Post-processing regex validation recommended.
• —CREDITCARDNUMBER vs PHONEIMEI: 16-digit numeric strings can be confused between these labels (F1=0.879). Luhn algorithm post-processing can mitigate this.
• —Low-support labels: Labels with <10 samples (e.g., CURRENCYNAME) have less reliable F1 estimates.
• —Language coverage: EN/FR/DE/IT only. Other languages may degrade.
• —Merged model note: LoRA weights are merged into bf16 base weights. This model is ~15GB and does not support 4-bit quantization natively — use the adapter repo for memory-constrained inference.

Environmental Impact

• —Hardware: NVIDIA A100 40GB (Google Colab Pro)
• —Training time: ~10.7 hours
• —Cloud provider: Google Cloud (Colab)
• —Compute region: US

Model Card Authors

Vineeth — Masters project, Enterprise Guardrails System Adapter repo: vineeth453/qwen25-7b-pii-detection-lora