digitalsolutiosai/PrivacyGuardML

PrivacyGuardML

PII confirmation model for Synthelion's PrivacyGuard — the opt-in ML-assisted confirmation tier (privacy.use_ml).

A small, fully offline transformer encoder trained with self-distillation: synthetic PII values are generated from Synthelion's own checksum validators and rule patterns, inserted into real multilingual sentences, and the encoder learns to find them back by shape and context via per-token BIO tagging. It is the sole ML backend PrivacyGuard supports — no third-party model.

What it's for, precisely. PrivacyGuard's core tier is regex + algorithmic checksum (or context keyword, for the rules with no checksum), zero-ML, and stays the default. This model is only the confirmation signal: a value that matches a rule's shape but has no context keyword nearby can be confirmed by an overlapping prediction from this model. A failed checksum still vetoes detection regardless of what the model says — it can only recover recall the strict context gate intentionally trades away, never introduce a false positive the validator would reject.

Architecture

Char-ngram hashing -> word embedding + shape-feature embedding -> positional encoding
-> 2-layer TransformerEncoder (d=128, h=4) -> Linear(33) BIO-tag logits

Categories

Checksum-backed: PHONE, EMAIL, CREDITCARD, IBAN, NATIONALID, TAXID, SSN, GPS. Context-only (no checksum, PrivacyGuard's remaining rule categories): VEHICLEPLATE, BADGEID, BUSINESSID, SECRET, SOCIALHANDLE, LEGALCASE, PNRCODE, MINORDATA.

Quick start

from synthelion.privacyguardml import PrivacyGuardMLDetector, resolve_model_path

detector = PrivacyGuardMLDetector(resolve_model_path(), min_confidence=0.6)
for span in detector.detect("Contact me at mario.rossi@example.com or IT60X0542811101000000123456"):
    print(span)  # MLSpan(start, end, value, label, confidence)

Training

• —Data: synthetic PII values (from Synthelion's checksum validators and privacy_rules.yaml patterns) inserted into real multilingual carrier sentences (Wikipedia corpora)
• —Labels: self-distillation — ground truth is exact by construction (the inserted value's span is known at insertion time)
• —Sampling: checksum-backed categories weighted 4x over context-only categories, to keep primary-category confidence high while still covering the full catalog
• —Languages: 39 (see config.json -> trained_languages)
• —Examples: 106,123 training / 5,585 eval
• —Runtime: fully offline, CPU-only

Files

License

MIT License. See LICENSE for details.