KittipatPaisanpudinun/Whisper_Finetune_ThaiFood-Ordering

🎙️ Whisper Fine-tuned for Thai Food Ordering (CTranslate2 / faster-whisper)

    

A domain-adapted Thai Automatic Speech Recognition (ASR) model fine-tuned specifically for Thai restaurant, food court, and beverage ordering contexts. Built upon OpenAI Whisper Large-v3-Turbo, this model has been optimized and converted into CTranslate2 (faster-whisper) format for ultra-fast, real-time inference on both GPU (CUDA) and CPU environments.

🌟 Key Highlights

• —🎯 Domain-Specific Precision: Specialized in Thai culinary vocabulary, ingredient customization phrases (e.g., "หวานน้อย" [less sweet], "ไม่ใส่ผัก" [no vegetables], "เผ็ดกลาง" [medium spice], "พิเศษไข่ดาว" [extra fried egg]), and quantitative units.
• —📉 Drastic CER Reduction: Character Error Rate (CER) drops from 18.8% down to 1.3%, achieving an outstanding 98.7% accuracy on the test set.
• —⚡ Real-Time Latency: Achieves an exceptional Real-Time Factor (RTF) of 0.051 (processing audio nearly 20 times faster than real-time playback).
• —🚀 CTranslate2 & faster-whisper Ready: Fully compatible with faster-whisper, reducing VRAM consumption by up to 50% while accelerating transcription speed with float16 and int8_float16 quantizations.

📊 Evaluation & Benchmark Results

The model was evaluated against a dedicated Thai food ordering test benchmark (500 samples), comparing the fine-tuned model directly with the pre-trained baseline model (openai/whisper-large-v3-turbo):

<p align="center"> <img src="summarybarchart.png" alt="Overall Performance Summary (Finetuned vs Non-Finetuned)" width="750"/> </p>

📈 Detailed Benchmark Comparison

Definitions: - CER (Character Error Rate): Lower is better. Measures character-level transcription discrepancies for Thai script. - Accuracy: Calculated as $1 - \text{CER}$. - RTF (Real-Time Factor): Lower is better. Processing execution time divided by the audio duration.

📦 Dataset Information

• —Dataset Repository: 🔗 KittipatPaisanpudinun/Thai-Food-Ordering-Dataset
• —Total Utterances: 5,000 Thai spoken food and drink ordering commands.
• —Dataset Partition:
• —Training Set: 4,000 samples (80%)
• —Validation Set: 500 samples (10%)
• —Test Set: 500 samples (10%)
• —Data Quality Control: Conducted pre-screening and manual verification to confirm audio quality and ground-truth text alignment for all high-risk items.

⚙️ Training Hyperparameters & Setup

• —Base Architecture: openai/whisper-large-v3-turbo (32 Encoder layers, 4 Decoder layers, 128 Mel frequency bins)
• —Optimizer: AdamW (8-bit quantized optimizer for reduced memory overhead)
• —Learning Rate: $2.0 \times 10^{-6}$ with Linear decay
• —Warmup: 100 steps
• —Effective Batch Size: 64 (Batch Size 2 $\times$ Gradient Accumulation 32)
• —Training Epochs: 10
• —Data Augmentation:
• —SpecAugment: Frequency and time masking to prevent overfitting.
• —Audio Noise Simulation: Office and ambient background noise injected to simulate realistic restaurant soundscapes.
• —Regularization: Weight Decay 0.1, BPE Dropout 0.1

💻 Quickstart with faster-whisper

This model is formatted in CTranslate2, allowing direct inference with the lightweight and highly optimized faster-whisper library.

1. Installation

pip install faster-whisper

2. Python Inference Example

GPU (CUDA) - Recommended for Production & Lowest Latency:

from faster_whisper import WhisperModel

# Load model directly from Hugging Face Hub or local path
model_id = "KittipatPaisanpudinun/Whisper_Finetune_ThaiFood-Ordering"

# Initialize model with float16 or int8_float16 on GPU
model = WhisperModel(model_id, device="cuda", compute_type="float16")

# Transcribe Thai ordering audio
segments, info = model.transcribe(
    "order_sample.wav",
    language="th",
    vad_filter=True,                  # Filter out non-speech silence
    beam_size=5,
    initial_prompt="สั่งอาหาร เมนู ข้าวมันไก่ กะเพราหมูกรอบ หวานน้อย เผ็ดกลาง"
)

print(f"Detected language: '{info.language}' with probability {info.language_probability:.2f}")

full_text = ""
for segment in segments:
    print(f"[{segment.start:.2f}s -> {segment.end:.2f}s] {segment.text}")
    full_text += segment.text + " "

print("\nResult:", full_text.strip())

CPU Execution:

from faster_whisper import WhisperModel

model_id = "KittipatPaisanpudinun/Whisper_Finetune_ThaiFood-Ordering"

# Initialize on CPU using int8 quantization to save memory
model = WhisperModel(model_id, device="cpu", compute_type="int8")

segments, _ = model.transcribe("order_sample.mp3", language="th")
for segment in segments:
    print(segment.text)

🎯 Target Use Cases

1. 1.Self-Ordering Kiosks: Voice-operated touchscreens in quick-service restaurants and food courts.
2. 2.Smart Drive-Thru Systems: Ambient-noise resistant speech recognition for car lanes.
3. 3.Voice Ordering Chatbots & AI Waiters: Automated telephone or app-based food ordering assistants.
4. 4.Kitchen Display System (KDS) Transcription: Transcribing waiter voice input directly to the kitchen display.

⚠️ Limitations

• —Domain Scope: The model is heavily specialized for Thai food, drink, and ordering dialogues. Accuracy on unrelated domains (e.g., medical, legal, or general conversation) may vary.
• —Ambient Noise: In exceptionally noisy food court environments, enabling vad_filter=True in model.transcribe() is recommended to avoid false positive triggers.

📚 Citation

If you use this model or dataset in your research or application, please cite:

@article{kittipat2026whisperthaifood,
  title={Comparative Study and Fine-tuning of Whisper AI Models for Thai Speech Recognition in Food Ordering Contexts},
  author={Kittipat Paisanpudinun},
  year={2026},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/KittipatPaisanpudinun/Whisper_Finetune_ThaiFood-Ordering}}
}

License: MIT License Developed by: Kittipat Paisanpudinun