coung21/vi-spelling-correction

Vietnamese Spelling Correction Dataset

This dataset contains 978,417 pairs of noisy (source) and clean (target) Vietnamese sentences, designed for training spelling correction models.

The dataset was synthetically generated by injecting realistic noise into a clean Vietnamese corpus.

Dataset Structure

The dataset is divided into training and testing sets:

• —Train: 880,575 examples
• —Test: 97,842 examples

Data Fields

• —source: The text with injected errors (input).
• —target: The original clean text (label).

Sample

Noise Generation Logic

The noise was generated using a custom script with a 0.5 noise rate (approx. 50% of tokens affected) and guaranteed at least one error per sample. The errors mimic real-world Vietnamese typing and spelling mistakes:

1. 1.Teencode & Lexical Variants (~40%):
2. 2.Syllable contractions: ng $\to$ g, nh $\to$ h, qu $\to$ w, yê $\to$ i.
3. 3.Phonetic substitutions: ph $\to$ f, gi $\to$ j, c/k $\to$ k.
4. 4.Dictionary slang: vợ $\to$ vk, không $\to$ ko.

1. 1.Regional Phonological Errors (~30%):
2. 2.North: tr $\leftrightarrow$ ch, s $\leftrightarrow$ x, r $\leftrightarrow$ d $\leftrightarrow$ gi.
3. 3.South: n $\leftrightarrow$ ng (final), t $\leftrightarrow$ c.

1. 1.Typing & Mechanical Errors (~20%):
2. 2.Spatial: Hitting adjacent keys on QWERTY keyboard.
3. 3.Telex: Wrong accent codes (s $\to$ d), double typing (đ $\to$ ddd).
4. 4.Operations: Random insertions, deletions, transpositions.

1. 1.Unaccented (~10%):
2. 2.Removing tone marks (e.g., trường $\to$ truong).

Usage

from datasets import load_dataset

dataset = load_dataset("coung21/vi-spelling-correction")

print(dataset["train"][0])
# Output: {'source': '...', 'target': '...'}

Credits

The source data for this dataset was extracted from a Vietnamese Wikipedia dump. The noise was synthetically generated using a custom noise injection pipeline to simulate realistic Vietnamese spelling errors.