Mozilla/tinybert-address-autofill

TinyBERT Address Autofill

A compact field-type classifier for HTML form autofill developed by the Credentials Management Team on Firefox. Given a string describing a single form field's attributes, it predicts one of 66 autofill field types (given-name, family-name, email, postal-code, address-line1, cc-number, etc.) or other when the field should not be filled.

The model is fine-tuned from huawei-noah/TinyBERT_General_4L_312D on a corpus of manually annotated shopping and address forms collected by Mozilla, and is intended to run client-side inside Firefox (or any Transformers.js host) as a replacement or augmentation for the existing regex-based heuristic field detector.

ONNX variants

All variants live under onnx/ and are loadable through Transformers.js by passing the corresponding dtype argument.

How to use

Transformers.js (browser)

import { pipeline } from "@huggingface/transformers";

const classifier = await pipeline(
  "text-classification",
  "vazish/tinybert-address-autofill",
  { dtype: "q8" }   // try "fp16" for highest fidelity, "q4f16" for smallest
);

const out = await classifier(
  "a-c-postal-code billing zip code dwfrm billing address fields postal code"
);
// → [{ label: "postal-code", score: 0.99 }]

Python (Optimum + ONNX Runtime)

from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer, pipeline

model = ORTModelForSequenceClassification.from_pretrained(
    "vazish/tinybert-address-autofill",
    file_name="onnx/model.onnx",   # or onnx/model_quantized.onnx, etc.
)
tokenizer = AutoTokenizer.from_pretrained("vazish/tinybert-address-autofill")
clf = pipeline("text-classification", model=model, tokenizer=tokenizer)

clf("email email mail **email")
# → [{"label": "email", "score": 0.99}]

Input format

The model expects a single string per field, built by concatenating that field's HTML attributes after light normalisation:

1. 1.Concatenate (in order): type + autocomplete + id + name + placeholder + the field's computed <label> text.
2. 2.Split camelCase boundaries to whitespace (firstName → first name).
3. 3.Lowercase the whole thing.
4. 4.If the field declares an autocomplete attribute, prepend an a-c-<value> token (e.g. a-c-postal-code).
5. 5.Optionally include adjacent-field context — bb-prefixed tokens for the previous field on the same form and aa-prefixed tokens for the next. Including adjacent context improves accuracy by roughly 8 percentage points relative to the same model trained on isolated fields.

Example input for a "first name" field followed by a "last name" field:

first name first name enter first name aaa-c-family-name aalast aaname

Training

Each form field is annotated with data-mozautofill-type="<type>" set to the expected autofill class; fields that should not be filled receive no attribute and are mapped to other.

Evaluation

Evaluated on the project's held-out test set (2,168 labelled fields drawn from real address / shopping forms) using ONNX Runtime on CPU.

• —Total — strict exact-match accuracy.
• —Close — counts predictions on closely related labels as correct (e.g. street-address predicted when ground truth is address-line1, tel predicted when ground truth is tel-national).
• —Blank — false-fill rate. Fraction of other-labelled fields the model predicted as a real autofill type. Lower is better; this metric matters most for user experience because high false-fill means filling search boxes, comments, and gift-card fields with personal data.

For reference, the existing Firefox regex-based heuristic detector reaches roughly 85% total accuracy on comparable test sets.

Highlights:

• —fp16 is statistically indistinguishable from fp32 across all metrics while halving the file size. It is the recommended high-fidelity variant. Latency on CPU is ~2× fp32 because most CPUs lack native fp16 ops, but the gap closes on hardware with fp16 support and on WebGPU.
• —int8 / quantized has the lowest exact accuracy but the lowest false-fill rate of any variant (1.94%, below the fp32 baseline). It errs toward other when uncertain — the safer failure mode for an autofill UI. This is the recommended size-constrained default.
• —4-bit variants (q4, q4f16, bnb4) cluster around 88% total accuracy with q4f16 being the smallest at 22 MB.

Limitations

• —Trained primarily on the supported-region list above. Accuracy on unsupported regions trained-without-data drops ~5–10 percentage points; adding region-specific samples to the training set typically recovers most of that gap.
• —Underrepresented field types (address-line3, additional-name, phonetic-*, tel-local-prefix, etc.) have very few training examples and are sometimes confidently misclassified.
• —Quantized variants disagree with fp32 on roughly 0.1% (fp16) to ~5% (int8) of inputs. The exact disagreement pattern is captured in the evaluation table above.
• —The model assumes the team's preprocessing format (camelCase-split, lowercased, with optional a-c-/bb/aa markers). Feeding raw HTML attribute strings without this normalisation will degrade accuracy.

Citation

This model is built on TinyBERT:

@inproceedings{jiao-etal-2020-tinybert,
  title     = {{TinyBERT}: Distilling {BERT} for Natural Language Understanding},
  author    = {Jiao, Xiaoqi and Yin, Yichun and Shang, Lifeng and Jiang, Xin
               and Chen, Xiao and Li, Linlin and Wang, Fang and Liu, Qun},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2020},
  year      = {2020},
  pages     = {4163--4174},
  url       = {https://aclanthology.org/2020.findings-emnlp.372}
}

If you use this checkpoint, please also cite the Mozilla autofill ML investigation that produced it (citation forthcoming).

License

Apache 2.0.