350016z/ErrorSpanAnnotation-for-Taiwanese-Hokkien

Error Span Annotation for Taiwanese Hokkien

The Taiwanese Hokkien subset of the SiniticMTError benchmark (Liu et al., 2026).

Human-annotated machine-translation error-span evaluation data for the Mandarin → Taiwanese Hokkien (Tâi-gí) direction. Each instance contains a Mandarin source sentence, a Taiwanese Hokkien machine translation, a reference translation, and expert error-span annotations with severity labels and a segment-level quality score.

• —Language pair: Mandarin (zh) → Taiwanese Hokkien, Han script (nan)
• —MT system: all machine translations (mt) were produced by Taigi-Llama-2-Chat-7B
• —Size: 154 sentence pairs (single test split)
• —Annotation scheme: MQM-based (Lommel et al., 2013) — error spans with a customized error-category set and two severity levels (Minor / Major); no Critical level is used

Dataset structure

Each line in data/data.jsonl is one instance:

Each element of spans has:

Offsets index into mt at the character level; mt[start:end] == text.

Example

{
  "system": "Taigi-Llama-2-Chat-7B",
  "id": "0",
  "lp": "zh-han",
  "doc": "moedict",
  "src": "頭髮留那麼長還不剪，莫非是想把錢省下來買花生糖吃。",
  "mt": "頭毛留甲遐爾長毋去鉸，敢是欲儉錢來買塗豆麩食？",
  "ref": "頭毛留長長，儉錢食塗豆糖。",
  "spans": [
    {"text": "塗豆麩", "start": 18, "end": 21, "severity": "Major", "category": "Accuracy/Mistranslation"}
  ],
  "score100": 99.17,
  "score7": 5.95,
  "explanation": "1. 中文「花生糖」，台文是「塗豆糖」。",
  "rater": "rater1"
}

Label sets

Severity: Minor, Major, No-error (no Critical level). In this subset: Major 156, Minor 161, No-error 12.

Error categories. Annotators chose from an MQM-based typology (Class/Subclass):

• —Accuracy: Mistranslation, Addition, Omission
• —Fluency: Grammar, Spelling, Punctuation, Inconsistency, Register
• —Terminology: Inappropriate, Inconsistent
• —Style: Awkward
• —Locale: Currency / Time / Name / Date / Address format
• —Purity: Mandarin interference — Mandarin lexical items left untranslated in an otherwise Taiwanese Hokkien sentence (specific to this close language pair)

Not every category occurs in this 154-sentence subset. The data additionally uses Non-translated and No-error markers, and a .../Other subcategory where an error did not fit a listed subclass.

Error-type distribution (306 error spans over 154 sentences)

Collection & annotation

Mandarin source sentences and their Taiwanese Hokkien machine translations were annotated by expert linguists (native Taiwanese Hokkien speakers). Annotators first calibrated on shared examples to align on applying the MQM-based annotation scheme to Taiwanese Hokkien, then annotated each sentence and resolved disagreements through discussion to reach consensus. Because the two annotators were not independent, conventional inter-annotator agreement is not reported.

Usage

from datasets import load_dataset

ds = load_dataset("350016z/ErrorSpanAnnotation-for-Taiwanese-Hokkien", split="test")
print(ds[0]["src"], ds[0]["mt"])
for s in ds[0]["spans"]:
    print(s["text"], s["severity"], s["category"])

Licensing & attribution

Released under CC BY-NC-SA 4.0 (attribution, non-commercial, share-alike). This reflects the licenses of the underlying materials:

• —Machine translations (`mt`) were generated by Taigi-Llama-2-Chat-7B (Bohanlu), released under CC BY-NC-SA 4.0. Per the underlying Llama 2 license and Acceptable Use Policy, these outputs may not be used to train or improve other large language models (other than Llama 2 or its derivatives).
• —Source sentences (`src`) are derived from the 教育部臺灣台語常用詞辭典 (MoE Taiwanese Dictionary), © Ministry of Education, R.O.C., and are included here for non-commercial research with attribution.
• —Annotations (spans, score100, score7, explanation) are contributed by the dataset authors and released under CC BY-NC-SA 4.0.

This dataset is intended for non-commercial research use only.

This is guidance, not legal advice. Some MoE dictionary materials carry a no-derivatives (ND) clause; if in doubt, confirm the terms with the Ministry of Education and your institution before redistribution.

Citation

@article{liu2026siniticmterror,
  title   = {SiniticMTError: A Machine Translation Dataset with Error Annotations for Sinitic Languages},
  author  = {Liu, Hannah and Min, Junghyun and Lee, En-Shiun Annie and Cheung, Ethan Yue Heng and Hung, Shou-Yi and Chan, Elsie and Qian, Shiyao and Liang, Runtong and Huynh, Kimlan and Yip, Wing Yu and Ng, York Hay and Yau, Tsz Fung and Lo, Ka Ieng Charlotte and Wu, You-Wei and Tsai, Rich