CoolFace
Modelpublic

khanhld/chunkformer-rnnt-large-vie

sourceHugging Facecc-by-4.0updated 1y agoView on Hugging Face
4likes84downloads
Model Card

ChunkFormer-RNNT-Large-Vie: Large-Scale Pretrained ChunkFormer-RNNT for Vietnamese Automatic Speech Recognition

<style> img { display: inline; } </style> ![License: CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) ![GitHub](https://github.com/khanld/chunkformer) ![Paper](https://arxiv.org/abs/2502.14673) ![Model size](#description)

Table of contents

  1. 1.Model Description
  2. 2.Documentation and Implementation
  3. 3.Benchmark Results
  4. 4.Usage
  5. 5.Citation
  6. 6.Contact

<a name = "description" ></a>

Model Description

ChunkFormer-RNNT-Large-Vie is a large-scale Vietnamese Automatic Speech Recognition (ASR) model based on the ChunkFormer architecture, introduced at ICASSP 2025. The model has been fine-tuned on approximately 5000 hours of public Vietnamese speech data sourced from diverse datasets.

<a name = "implementation" ></a>

Documentation and Implementation

The Documentation and Implementation of ChunkFormer are publicly available.

<a name = "benchmark" ></a>

Benchmark Results

We evaluate the models using Word Error Rate (WER). To ensure consistency and fairness in comparison, we manually apply Text Normalization, including the handling of numbers, uppercase letters, and punctuation.

  1. 1.Public Models:
STTModel#ParamsVivosCommon VoiceVLSP 2020 - Task 1VLSP 2020 - Task 2Avg.
1ChunkFormer-RNNT-Large-Vie113M2.495.1812.7520.4710.22
2ChunkFormer-CTC-Large-Vie110M4.186.6614.0925.8112.69
3vinai/PhoWhisper-large1.55B4.678.1413.7526.6813.31
4nguyenvulebinh/wav2vec2-base-vietnamese-250h95M10.7718.3413.3351.4523.47
5openai/whisper-large-v31.55B8.8115.4520.4168.6128.32
6khanhld/wav2vec2-base-vietnamese-160h95M15.0510.7831.6262.0129.87
7homebrewltd/Ichigo-whisper-v0.122M13.4623.5221.6462.9230.39
  1. 1.Private Models (API):
STTModelVLSP - Task 1
1ChunkFormer-RNNT-Large-Vie12.8
2ChunkFormer-CTC-Large-Vie14.1
3Viettel14.5
4Google19.5
5FPT28.8

<a name = "usage" ></a>

Quick Usage

To use the ChunkFormer model for Vietnamese Automatic Speech Recognition, follow these steps:

Option 1: Install from PyPI (Recommended)

bash
pip install chunkformer

Option 2: Install from source

bash
git clone https://github.com/khanld/chunkformer.git
cd chunkformer
pip install -e .

Python API Usage

python
from chunkformer import ChunkFormerModel

# Load the Vietnamese model from Hugging Face
model = ChunkFormerModel.from_pretrained("khanhld/chunkformer-rnnt-large-vie")

# For single long-form audio transcription
transcription = model.endless_decode(
    audio_path="path/to/long_audio.wav",
    chunk_size=64,
    left_context_size=128,
    right_context_size=128,
    total_batch_duration=14400,  # in seconds
    return_timestamps=True
)
print(transcription)

# For batch processing of multiple audio files
audio_files = ["audio1.wav", "audio2.wav", "audio3.wav"]
transcriptions = model.batch_decode(
    audio_paths=audio_files,
    chunk_size=64,
    left_context_size=128,
    right_context_size=128,
    total_batch_duration=1800  # Total batch duration in seconds
)

for i, transcription in enumerate(transcriptions):
    print(f"Audio {i+1}: {transcription}")

Command Line Usage

After installation, you can use the command line interface:

bash
chunkformer-decode \
    --model_checkpoint khanhld/chunkformer-rnnt-large-vie \
    --long_form_audio path/to/audio.wav \
    --total_batch_duration 14400 \
    --chunk_size 64 \
    --left_context_size 128 \
    --right_context_size 128

Example Output:

[00:00:01.200] - [00:00:02.400]: this is a transcription example
[00:00:02.500] - [00:00:03.700]: testing the long-form audio

Advanced Usage can be found HERE

<a name = "citation" ></a>

Citation

If you use this work in your research, please cite:

bibtex
@INPROCEEDINGS{10888640,
  author={Le, Khanh and Ho, Tuan Vu and Tran, Dung and Chau, Duc Thanh},
  booktitle={ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, 
  title={ChunkFormer: Masked Chunking Conformer For Long-Form Speech Transcription}, 
  year={2025},
  volume={},
  number={},
  pages={1-5},
  keywords={Scalability;Memory management;Graphics processing units;Signal processing;Performance gain;Hardware;Resource management;Speech processing;Standards;Context modeling;chunkformer;masked batch;long-form transcription},
  doi={10.1109/ICASSP49660.2025.10888640}}
}

<a name = "contact"></a>

Contact

  • —khanhld218@gmail.com
  • —![GitHub](https://github.com/khanld)
  • —![LinkedIn](https://www.linkedin.com/in/khanhld257/)