waxal-benchmarking/omniasr-llm-300m-waxal-semitic

omniasr-llm-300m-waxal-semitic

A Ethiosemitic (Afro-Asiatic) multilingual fine-tune of Meta's Omnilingual ASR (wav2vec2_llama, 300M-parameter shared wav2vec2 encoder + autoregressive Llama-style decoder), trained jointly on 2 WAXAL languages. Part of the WAXAL ASR benchmark's training-granularity study (monolingual vs. language-family vs. all-19 pooling).

• —Base model: `facebook/omniASR-LLM-300M`
• —Languages (2): Amharic, Tigrinya
• —Macro-averaged WER (this model): 30.8%  |  monolingual baselines: 31.0%

Training data

Fine-tuned on the pooled train splits of the WAXAL corpus for these languages (16 kHz mono; transcripts NFC-normalized and lower-cased, punctuation removed, phonemic diacritics/tone marks preserved). Total: 78,878 clips / 371.6 hours.

Training procedure

Fine-tuned with the Omnilingual-ASR wav2vec2_llama recipe (fairseq2) on 2× NVIDIA H200 (DistributedDataParallel). All granularity conditions use an identical budget so the only variable is the language mixture.

Evaluation

Scored on each language's held-out test split (utterances ≥ 1.5 s, matching the benchmark's filtered-test protocol). WER and CER computed with jiwer on NFC-normalized, lower-cased text (diacritics preserved). The Monolingual WER column is the corresponding per-language model (`omniasr-llm-300m-waxal-<iso>`) evaluated identically, for a same-protocol comparison.

Usage

# pip install git+https://github.com/facebookresearch/omnilingual-asr.git
from pathlib import Path
import torch
from huggingface_hub import snapshot_download
from fairseq2.data.tokenizers.hub import load_tokenizer
from omnilingual_asr.models.inference.pipeline import ASRInferencePipeline
from omnilingual_asr.models.wav2vec2_llama.hub import get_wav2vec2_llama_model_hub

ckpt = snapshot_download("waxal-benchmarking/omniasr-llm-300m-waxal-semitic")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
dtype = torch.bfloat16 if device.type == "cuda" else torch.float32

hub = get_wav2vec2_llama_model_hub()
model = hub.load_custom_model(Path(ckpt) / "model", hub.get_arch_config("300m"), device=device, dtype=dtype)
tokenizer = load_tokenizer("omniASR_tokenizer_v1")

pipe = ASRInferencePipeline(model_card=None, model=model, tokenizer=tokenizer, device=device, dtype=dtype)
# pass the target language's Omnilingual token, e.g. Amharic -> "amh_Ethi"
texts = pipe.transcribe(["your_audio.flac"], lang=["amh_Ethi"])
print(texts)

Language tokens for this model: Amharic amh_Ethi, Tigrinya tir_Ethi. Audio should be mono 16 kHz (the pipeline resamples if needed); keep clips under 40 s.

Checkpoint format

Native fairseq2 sharded checkpoint (model/pp_00/tp_00/sdp_00.pt + model.yaml) — not a transformers model, so AutoModel will not load it. Load with omnilingual_asr / fairseq2 as shown above.

Citation

Part of the WAXAL ASR Benchmark (arXiv:2606.02375).

@article{waxalnet2026,
  title  = {The WAXAL ASR Benchmark: Fine-Tuned Edge Models Across 19 African Languages},
  author = {Olufemi, Victor Tolulope and Babatunde, Oreoluwa and Njema, Ramsey and others},
  year   = {2026},
  note   = {arXiv preprint arXiv:2606.02375}
}

Acknowledgements

Supported by [Lynguallabs](https://lynguallabs.org/) (compute, researchers & storage), [Open Token](https://opentoken.global/) (compute), and [CMU Africa](https://www.africa.engineering.cmu.edu/) (researchers & native speakers).