Shramadeepd/uyghur-ASR-dataset

Uyghur ASR Corpus (Latin Transliteration)

A speech corpus for Uyghur automatic speech recognition, with transcriptions in a case-sensitive Latin transliteration scheme. Approximately 23 hours of audio across 9,468 clips.

Uyghur is a Turkic language spoken by roughly 10–12 million people. It is severely under-represented in open speech datasets, and this corpus is intended to support ASR research for the language.

Dataset summary

Repository structure

.
├── README.md
├── train.csv            # ID, filepath, transcription
├── test.csv             # ID, filepath
├── sample.csv           # submission template: ID, transcription
└── wavs/
    ├── aca14f95f3c4487fa955a9360e324ca5.wav
    ├── 7f61bb1206384a468f93d36eb4409cfd.wav
    └── ...

train.csv

test.csv

Example record

ID             : aca14f95f3c4487fa955a9360e324ca5
filepath       : wavs/aca14f95f3c4487fa955a9360e324ca5.wav
transcription  : vuyGur HAlqiniN fevudal bAglArniN wA pomexciklarniN ...

The transliteration scheme — read this before using the data

The transcriptions use a Latin scheme in which case carries phonemic meaning. The characters A, G, H, J, N, O, U are distinct phonemes, not capitalised forms of a, g, h, j, n, o, u.

Do not lowercase these transcriptions. Doing so silently collapses seven phonemes and will substantially degrade both training and evaluation.

Full character inventory (34 tokens)

' ' A G H J N O U a b c d e f g h i j k l m n o p q r s t u v w x y z

Note what is absent: no punctuation, no digits, no diacritics. The text is already normalised, so no cleaning step is required before training.

Loading the dataset

With datasets

from datasets import load_dataset, Audio

ds = load_dataset("your-username/uyghur-asr-latin")
ds = ds.cast_column("audio", Audio(sampling_rate=16000))

print(ds["train"][0]["transcription"])
print(ds["train"][0]["audio"]["sampling_rate"])   # 16000

From local CSV

import os, pandas as pd
from datasets import Dataset, Audio

DATA_DIR = "./uyghur-asr-latin"

df = pd.read_csv(f"{DATA_DIR}/train.csv")
df["audio"] = df["filepath"].apply(lambda p: os.path.join(DATA_DIR, p))

ds = Dataset.from_pandas(df[["audio", "transcription"]])
ds = ds.cast_column("audio", Audio(sampling_rate=16000))

cast_column with Audio(sampling_rate=16000) decodes lazily and resamples on the fly, so the full 23 hours never has to sit in memory.

Building a CTC vocabulary

def extract_chars(batch):
    return {"vocab": [list(set(" ".join(batch["transcription"])))]}

vocab = sorted(set(ds.map(extract_chars, batched=True, batch_size=-1,
                          remove_columns=ds.column_names)["vocab"][0]))

vocab_dict = {c: i for i, c in enumerate(vocab)}
vocab_dict["|"] = vocab_dict.pop(" ")        # explicit word delimiter for CTC
vocab_dict["[UNK]"] = len(vocab_dict)
vocab_dict["[PAD]"] = len(vocab_dict)        # doubles as the CTC blank token

Recommended splits

The test split has no public transcriptions. For local evaluation, carve a validation set out of train:

from sklearn.model_selection import train_test_split
train_df, valid_df = train_test_split(df, test_size=0.1, random_state=42, shuffle=True)
# 6,816 train / 758 validation

Build the character vocabulary from the union of train and validation text so no held-out character maps to [UNK].

Evaluation protocol

The standard metric for this corpus is Character Error Rate (CER):

$$\mathrm{CER} = \frac{S + D + I}{N}$$

where S, D, I are substitutions, deletions and insertions, and N is the number of characters in the reference. Lower is better.

import evaluate
cer = evaluate.load("cer")
print(cer.compute(predictions=preds, references=refs))

Published baseline

Submission format

For challenge-style evaluation, predictions go in a CSV with exactly two columns:

ID,transcription
f068a206b84c4632865e0629a1b62fb8,bu dorini helila qaynatqan caqqan bol vissiqidA icin
a9d8cfab47b34f12b8f4b4769075713e,yamGurdin keyinki hawa Huddi sUzUp tazlanGandAk

Limitations and considerations

• —Style. Predominantly read or broadcast-style speech; not representative of spontaneous conversational Uyghur.
• —Speaker metadata. No speaker IDs, age, gender or dialect labels are provided, so speaker-disjoint splits cannot be guaranteed. Random splits may leak speakers across train and validation, which can make local CER optimistic.
• —Script coverage. Latin transliteration only. Models trained here do not directly produce Perso-Arabic Uyghur orthography; a transliteration post-processing step is required.
• —Scale. 23 hours is small by ASR standards. Training from scratch is not advisable — transfer learning from a multilingual checkpoint (e.g. MMS, XLS-R, Whisper) is strongly recommended.
• —Audio quality. Recording conditions, microphones and loudness vary. Per-utterance normalisation (do_normalize=True in Wav2Vec2FeatureExtractor) is recommended.

Ethical considerations

Uyghur is spoken by a community that has been the subject of documented surveillance concerns. Speech recognition technology for this language should be developed and deployed with care. This dataset is intended for language preservation, accessibility and open research, and should not be used to build identification, tracking or surveillance systems.

Licensing and attribution

Please verify the licensing terms of the original corpus before redistribution or commercial use. Audio and transcriptions remain the property of their original contributors.

Citation

@misc{uyghur_asr_latin,
  title  = {Uyghur ASR Corpus (Latin Transliteration)},
  year   = {2026},
  note   = {~23 hours of Uyghur speech with case-sensitive Latin transliteration},
  howpublished = {\url{https://huggingface.co/datasets/your-username/uyghur-asr-latin}}
}