CoolFace
Modelpublic

DAXAAI-Research/qwen4b-pii-ner-v4.1-saprse-2503-checkpoint-125

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face
0likes28downloads
Model Card

Qwen3-4B PII NER — LoRA Fine-tuned for PII Entity Extraction

A fine-tuned version of Qwen/Qwen3-4B-Instruct-2507 trained to extract Personally Identifiable Information (PII) from unstructured text. The model outputs a structured JSON object containing detected entities, organized by type.

Trained on the DAXAAI-Research/synthetic-pii-dataset-v2.4-dense dataset using LoRA adapters via TRL's SFTTrainer.


Model Overview

FieldDetails
Base ModelQwen/Qwen3-4B-Instruct-2507
TaskNamed Entity Recognition (NER) — PII Detection
MethodSupervised Fine-Tuning (SFT) with LoRA (PEFT)
FrameworkTRL SFTTrainer + HuggingFace Transformers + PEFT
LanguageEnglish
LicenseApache 2.0
DatasetDAXAAI-Research/synthetic-pii-dataset-v2.4-dense
W&B RunView training run

Target Entity Types (21)

BBAN_CODE, CREDIT_CARD, DATE_OF_BIRTH, EMAIL_ADDRESS, HEALTH_INSURANCE_NUMBER, HONG_KONG_ID, IBAN_CODE, INDIA_AADHAAR, INDIA_PAN, IP_ADDRESS, LICENSE_PLATE_NUMBER, MEDICAL_RECORD_NUMBER, PHONE_NUMBER, ROUTING_NUMBER, SWIFT_CODE, US_BANK_NUMBER, US_DRIVER_LICENSE, US_ITIN, US_PASSPORT, US_SSN, VEHICLE_VIN


Training Configuration

Hyperparameters

ParameterValue
Epochs~1.32 (380 steps)
Batch Size (per device)10
Gradient Accumulation Steps3
Effective Batch Size30
Learning Rate1e-5
LR SchedulerCosine
Warmup Ratio0.1
Weight Decay0.01
PrecisionBF16
Max Sequence Length8,192
OptimizerAdamW (β1=0.9, β2=0.999, ε=1e-8)
LossAssistant-only (SFT)

LoRA Configuration

ParameterValue
Rank (r)256
Alpha128
Dropout0.05
Target Modulesq_proj, k_proj, v_proj, o_proj
Task TypeCAUSAL_LM

Training Results

MetricTrainEval
Loss0.01440.0239
Mean Token Accuracy99.6%99.4%
Entropy1.0861.072

<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>


System Prompt

The model expects the following system prompt at inference time:

You are a Named Entity Recognition assistant. Extract the following entities from the input text and output as JSON.

Output format: a JSON object with entity types as keys and arrays of extracted values. Do NOT include character positions, start/end indices, or spans — only entity types and their values.

Entity types to extract:
- BBAN_CODE
- CREDIT_CARD
- DATE_OF_BIRTH
- EMAIL_ADDRESS
- HEALTH_INSURANCE_NUMBER
- HONG_KONG_ID
- IBAN_CODE
- INDIA_AADHAAR
- INDIA_PAN
- IP_ADDRESS
- LICENSE_PLATE_NUMBER
- MEDICAL_RECORD_NUMBER
- PHONE_NUMBER
- ROUTING_NUMBER
- SWIFT_CODE
- US_BANK_NUMBER
- US_DRIVER_LICENSE
- US_ITIN
- US_PASSPORT
- US_SSN
- VEHICLE_VIN

IMPORTANT RULES:
- Only include entity types that have extracted values in your output
- Do NOT include entity types with empty arrays — omit them entirely
- Extract the exact entity values as they appear in the text
- Do not infer or guess entities that are not explicitly present
- Output valid JSON only (entity types + values, no positions or indices)
- If no entities are found at all, output an empty JSON object: {}

Example — if the text contains an email, a phone number, and an SSN but nothing else, output:
{"EMAIL_ADDRESS": ["john.doe@example.com"], "PHONE_NUMBER": ["555-123-4567"], "US_SSN": ["123-45-6789"]}

Do NOT include keys like "CREDIT_CARD": [] or "IBAN_CODE": [] — if an entity type has no matches, leave it out completely.

Usage

python
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "DAXAAI-Research/qwen-pii-ner-adapters-v4-sparse"
base_model = "Qwen/Qwen3-4B-Instruct-2507"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(base_model)
model.load_adapter(model_id)

text = "Contact John at john.doe@example.com or 555-123-4567. His SSN is 123-45-6789."

messages = [
    {"role": "system", "content": "<system prompt above>"},
    {"role": "user", "content": text},
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(inputs.to(model.device), max_new_tokens=1500, temperature=0.0, do_sample=False)
result = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
print(result)
# {"EMAIL_ADDRESS": ["john.doe@example.com"], "PHONE_NUMBER": ["555-123-4567"], "US_SSN": ["123-45-6789"]}

Links:

  1. 1.wandb: https://wandb.ai/daxa/qwen-dft-ner/runs/mlwf2yyu
  2. 2.dataset: https://huggingface.co/datasets/DAXAAI-Research/synthetic-pii-dataset-v2.4-dense

Framework Versions

LibraryVersion
TRL0.29.1
Transformers5.3.0
PyTorch2.10.0
Datasets4.8.4
Tokenizers0.22.2
PEFT(bundled with TRL)