Modelpublic

shivam14245/phi3-mini-sensitive-lora

sourceHugging Facemitupdated 2mo agoView on Hugging Face

2likes8downloads

Model Card

phi3-mini-sensitive-lora

A LoRA adapter that fine-tunes Phi-3-mini to classify text as sensitive (1) or not sensitive (0) — for detecting PII, financial data, credentials, HR/medical info, and government IDs mixed into company databases.

Part of the AI Data Governance Platform project.

Results (held-out test set, n = 900)

Metric	Baseline (untrained head)	Fine-tuned (this adapter)
Accuracy	0.529	1.000
Precision	0.645	1.000
Recall	0.091	1.000
F1	0.159	1.000

Note: 1.00 reflects strong learning of a controlled synthetic dataset; real-world data would need a human-labeled test set to confirm generalization.

Training

—Base model: microsoft/Phi-3-mini-4k-instruct (loaded in 4-bit / QLoRA)
—Method: LoRA (rank 16, α 32, dropout 0.05) on the sequence-classification head
—Trainable params: 25.2M (0.67%)
—Data: 6,000 synthetic supermarket records (Indian locale), 70/15/15 split
—Hardware: RTX 3060 12GB, ~38 min, 3 epochs

Usage

python

import torch
from peft import PeftModel
from transformers import AutoModelForSequenceClassification, AutoTokenizer, BitsAndBytesConfig

BASE = "microsoft/Phi-3-mini-4k-instruct"
ADAPTER = "shivam14245/phi3-mini-sensitive-lora"

tok = AutoTokenizer.from_pretrained(BASE)
tok.pad_token = tok.pad_token or tok.eos_token

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True)
model = AutoModelForSequenceClassification.from_pretrained(BASE, num_labels=2,
                                                           quantization_config=bnb, device_map="cuda")
model.config.pad_token_id = tok.pad_token_id
model = PeftModel.from_pretrained(model, ADAPTER).eval()

text = "employee salary 85000 bank_account 9876543210 ifsc HDFC0001234"
inputs = tok(text, return_tensors="pt", truncation=True, max_length=256).to("cuda")
label = model(**inputs).logits.argmax(-1).item()   # 1 = sensitive, 0 = not
print("sensitive" if label == 1 else "not sensitive")

Labels

0 = not sensitive · 1 = sensitive (pii / financial / credentials / hrmedical / govtid)