B-K/umt5-thai-g2p-v2-0.5k

umt5-thai-g2p-v2-0.5k

This model is a fine-tuned version of B-K/umt5-thai-g2p-v2-pretraining-0.5k on the B-K/thai-g2p dataset for Thai Grapheme-to-Phoneme (G2P) conversion.. It achieves the following results on the evaluation set:

• —Loss: 1.0480
• —Cer: 0.0369

Model description

umt5-thai-g2p-v2-0.5k is designed to convert Thai text (words or sentences) into their corresponding phonemic International Phonetic Alphabet (IPA) representations.

Intended uses & limitations

Intended Uses

• —Thai Grapheme-to-Phoneme (G2P) Conversion: The primary use of this model is to generate phonemic transcriptions (IPA) for Thai text.
• —Speech Synthesis Preprocessing: Can be used as a component in a Text-to-Speech (TTS) pipeline to convert input text into phonemes before acoustic model processing.

Limitations

• —Accuracy: While the model achieves a Character Error Rate (CER) of approximately 0.0369 on the evaluation set, it is not 100% accurate. Users should expect some errors in the generated phonemes.
• —Out-of-Distribution Data: Performance may degrade on words, phrases, or sentence structures significantly different from those present in the B-K/thai-g2p training dataset. This includes very rare words, neologisms, or complex named entities.
• —Ambiguity: Thai orthography can sometimes be ambiguous, and the model might not always resolve such ambiguities correctly to the intended pronunciation in all contexts.
• —Sentence-Level vs. Word-Level: While trained on a dataset that includes sentences, its robustness for very long or highly complex sentences might vary.
• —Inherited Limitations: As a fine-tuned version of google/umt5-small, it inherits the general architectural limitations and scale of the base model.

How to use

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("B-K/umt5-thai-g2p-v2-0.5k")
model = AutoModelForSeq2SeqLM.from_pretrained("B-K/umt5-thai-g2p-v2-0.5k")

thai_text = "สวัสดีครับนี่คือโมเดลจีทูพีขนาดสี่สิบห้าล้านพารามิเตอร์มันเล็กมาก" # Example Thai text
inputs = tokenizer(thai_text, return_tensors="pt", padding=True, truncation=True)

outputs = model.generate(**inputs, num_beams=3, max_new_tokens=256)
phonemes = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(f"Thai Text: {thai_text}")
# สวัสดีครับนี่คือโมเดลจีทูพีขนาดสี่สิบห้าล้านพารามิเตอร์มันเล็กมาก
print(f"Phonemes: {phonemes.replace(" ", "")}") # <-- Removing the space to make it more readable
# sa˨˩.wat̚˨˩.diː˧.kʰrap̚˦˥.niː˥˩.kʰɯː˧.moː˧.deːl˧.t͡ɕiː˧.tʰuː˧.pʰiː˧.kʰa˨˩.naːt̚˨˩.siː˨˩.sip̚˨˩.haː˥˩.laːn˦˥.pʰaː˧.raː˧.mi˦˥.tɤː˥˩.man˧.lek̚˦˥.maːk̚˥˩

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 0.0003
• —trainbatchsize: 128
• —evalbatchsize: 128
• —seed: 42
• —optimizer: Use OptimizerNames.ADAMWTORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
• —lrschedulertype: cosine
• —lrschedulerwarmup_steps: 500
• —num_epochs: 50
• —labelsmoothingfactor: 0.1

Training results

Framework versions

• —Transformers 4.47.0
• —Pytorch 2.5.1
• —Datasets 3.6.0
• —Tokenizers 0.21.0