RobotsMali/soloba-tdt-0.6b-v0.5

Soloba-TDT-600M Series

<style> img { display: inline; } </style>

 |  | 

soloba-tdt-0.6b-v0.5 is a fine tuned version of `nvidia/parakeet-tdt-0.6b-v2` on the African Next Voices dataset (ANV). This model does not consistently produce Capitalizations and Punctuations and it cannot produce acoustic event tags like those found in the ANV dataset in its transcriptions. It was fine-tuned using NVIDIA NeMo.

This model is the only one of the v0 series that was not trained on RobotsMali/bam-asr-early or any derivative of Jeli-ASR, hence its particular naming (v0.5)

🚨 Important Note

This model, along with its associated resources, is part of an ongoing research effort, improvements and refinements are expected in future versions. Users should be aware that:

• —The model may not generalize very well across all speaking conditions and dialects.
• —Community feedback is welcome, and contributions are encouraged to refine the model further.

NVIDIA NeMo: Training

To fine-tune or play with the model you will need to install NVIDIA NeMo. We recommend you install it after you've installed latest PyTorch version.

pip install nemo-toolkit['asr']

How to Use This Model

Note that this model has been released for research purposes primarily.

Load Model with NeMo

import nemo.collections.asr as nemo_asr
asr_model = nemo_asr.models.ASRModel.from_pretrained(model_name="RobotsMali/soloba-tdt-0.6b-v0.5")

NeMo 2.5.0 Compatibility

This checkpoint was created with NeMo 2.5.0. Loading it with newer NeMo versions (observed with 2.7.x) can fail because the strict decoding schema expects key_phrase_items_list; see NVIDIA-NeMo/Speech#15658. This workaround was tested with Python 3.12:

from pathlib import Path

from nemo.collections.asr.models import ASRModel
from omegaconf import OmegaConf

model_name = "RobotsMali/soloba-tdt-0.6b-v0.5"
cfg = ASRModel.from_pretrained(model_name, return_config=True)
OmegaConf.set_struct(cfg, False)

for decoder in ("greedy", "beam"):
    boosting_tree = OmegaConf.select(cfg, f"decoding.{decoder}.boosting_tree")
    if boosting_tree is not None:
        boosting_tree.key_phrase_items_list = None

config_path = Path("patched_config.yaml").resolve()
OmegaConf.save(cfg, config_path)
asr_model = ASRModel.from_pretrained(
    model_name=model_name,
    override_config_path=str(config_path),
    strict=False,
)
config_path.unlink()

Transcribe Audio

asr_model.eval()
# Assuming you have a test audio file named sample_audio.wav
asr_model.transcribe(['sample_audio.wav'])

Input

This model accepts any mono-channel audio (wav files) as input and resamples them to 16 kHz sample rate before performing the forward pass

Output

This model provides transcribed speech as an hypothesis object with a text attribute containing the transcription string for a given speech sample. (nemo>=2.3)

Model Architecture

This model uses a FastConformer encoder and an autoregressive Token-and-Duration Transducer decoder, a variant of RNN-T that predicts jointly learn to predict a token and its duration. FastConformer is an optimized version of the Conformer model with 8x depthwise-separable convolutional downsampling. You may find more information on the details of FastConformer here: Fast-Conformer Model.

Training

The NeMo toolkit was used for finetuning this model for 82,628 steps over nvidia/parakeet-tdt-0.6b-v2 model. The finetuning codes and configurations can be found at RobotsMali-AI/bambara-asr.

The tokenizer for this model was trained on the text transcripts of the train set of RobotsMali/afvoices using this script.

Dataset

This model was fine-tuned on a 100 hours pre-completion subset of the African Next Voices dataset. You can reconstitute that subset with these manifest files

Performance

We report the Word Error Rate (WER) and Character Error Rate (CER) for this model:

License

This model is released under the CC-BY-4.0 license. By using this model, you agree to the terms of the license.

Feel free to open a discussion on Hugging Face or file an issue on GitHub for help or contributions.