jensenlwt/whisper-small-singlish-122k

Whisper-small-singlish-122k.

This model is a openai/whisper-small, fine-tuned on a subset (122k samples) of the National Speech Corpus.

The following results on the evaluation set (43,788k samples) are reported:

• —Loss: 0.171377
• —WER: 9.69

Model Details

Model Description

• —Developed by: jensenlwt
• —Model type: automatic-speech-recognition
• —License: MIT
• —Finetuned from model: openai/whisper-small

Uses

The model is intended as exploration exercise to develop better ASR model for Singapore English (singlish).

The recommended audio usage for testing should be:

1. 1.Involves local Singapore slang, dialect, names, and terms etc.
2. 2.Involves Singaporean accent.

Direct Use

To use the model in an application, you can make use of transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("automatic-speech-recognition", model="jensenlwt/whisper-small-singlish-122k")

Out-of-Scope Use

• —Long form audio
• —Broken Singlish (typically from older generation)
• —Poor quality audio (audio samples are recorded in a controlled environment)
• —Conversation (as the model is not trained on conversation)

Training Details

Training Data

We made use of the National Speech Corpus for training. In specific, we made use of Part 2 – which is a series of audio samples of prompted read speech recordings that involves local named entities, slang, and dialect.

To train, I make used of the first 300 transcripts in the corpus, which is around 122k samples from ~161 speakers.

Training Procedure

The model is fine-tuned with occasional interruptions to adjust batch size to maximise GPU utilisation. In addition, I also end training early if eval_loss does not decrease in two evaluation steps as per previous training experience.

Training Hyperparameters

The following hyperparameters are used:

• —batch_size: 128
• —gradient_accumulation_steps: 1
• —learning_rate: 1e-5
• —warmup_steps: 500
• —max_steps: 5000
• —fp16: true
• —eval_batch_size: 32
• —eval_step: 500
• —max_grad_norm: 1.0
• —generation_max_length: 225

Training Results

The model with the lowest evaluation loss is used as the final checkpoint.

Testing Data, Factors & Metrics

Testing Data

To test the model, I made use of the last 100 transcripts (held-out test set) in the corpus, which is around 43k samples.

Results

Summary

The overall model is not perfect, but if audio is spoken clearly, the model is able to transcribe Singaporean terms and slang accurately.

Compute Infrastructure

Trained on VM instance provisioned on jarvislabs.ai.

Hardware

• —Single A6000 GPU

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

Low Wei Teck