nrl-ai/vn-spell-correction-eval-real

vn-spell-correction-eval-real

Out-of-distribution evaluation corpus for Vietnamese spell-correction models — 150 hand-curated (noisy, clean) pairs sampled from real VN error sources, not generated by nom.text.noise.

This is the test set we use to verify a spell-correction model generalises beyond its own synthetic training distribution. A model that scores 95 % on nom-vn's synthetic eval grid and 60 % on this set is overfit to the noise generator.

Splits

Each row is {"input": noisy, "target": clean} UTF-8 NFC.

Usage

from datasets import load_dataset

# All 6 registers — each is a separate config
forum = load_dataset("nrl-ai/vn-spell-correction-eval-real", "forum", split="train")
ocr = load_dataset("nrl-ai/vn-spell-correction-eval-real", "ocr", split="train")

# Inference example with a HF spell-correction model
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tok = AutoTokenizer.from_pretrained("nrl-ai/vn-spell-correction-base")
model = AutoModelForSeq2SeqLM.from_pretrained("nrl-ai/vn-spell-correction-base")
for row in forum:
    pred = tok.batch_decode(
        model.generate(**tok(row["input"], return_tensors="pt"), max_length=256),
        skip_special_tokens=True,
    )[0]
    print(row["input"], "->", pred, "  | gold:", row["target"])

Honesty notes

• —150 sentences is statistically noisy. Each register slice (n=25) has a ±9 pp 95 %-CI on word-accuracy; the aggregate (n=150) is ±5 pp. Use this as a directional smell-test, not a leaderboard.
• —No per-row source citation. Many noisy strings are composites observed across multiple posts / scans — copying verbatim would leak PII or violate platform ToS. Structural patterns (which chars flip, which abbreviations fire) are real; surrounding sentence content is paraphrased from public VN text.
• —Forum slang ages fast. vcl today may read as archaic in 12-18 months. Re-curate periodically.

Reference baseline

Latest in-house bench (source): `nrl-ai/vn-spell-correction-base` hits 78.33 % aggregate word-acc, 65 / 150 sentence-exact on this corpus. Per register: ocr 97.6 / news 96.5 / mobile 95.8 / legal 95.6 / forum 63.4 / telex 18.0. Telex is the disaster register and the single biggest gap in current public VN spell models.

Citation

@dataset{nguyen_vn_spell_correction_eval_real_2026,
  author = {Nguyen, Viet-Anh and {Neural Research Lab}},
  title  = {{vn-spell-correction-eval-real: Out-of-distribution
             evaluation corpus for Vietnamese spell correction}},
  year   = {2026},
  url    = {https://huggingface.co/datasets/nrl-ai/vn-spell-correction-eval-real}
}

Released CC0 — public domain dedication. Use freely.

Maintained as part of the `nom-vn` project by Viet-Anh Nguyen (vietanh@nrl.ai) and Neural Research Lab.