Tonykip/parakeet-tdt-0.6b-kalenjin

parakeet-tdt-0.6b-kalenjin

⚠️ Experimental — not recommended for use. This Parakeet fine-tune was a research experiment. On real-world Kalenjin audio it is less reliable than the recommended model, `Tonykip/whisper-kalenjin-v3-turbo` — use that instead. This repo is kept for reproducibility and research transparency.

An open automatic speech recognition (ASR) model for Kalenjin — a Highland Nilotic language of Kenya's Rift Valley (~2M speakers). This is an honest fine-tune of NVIDIA's `parakeet-tdt-0.6b-v3` (a 0.6B-parameter FastConformer encoder with a Token-and-Duration Transducer decoder). It is not trained from scratch, and it does not beat the strongest closed baseline overall — see Results for the honest comparison.

Adaptation rebuilds the SentencePiece tokenizer to Kalenjin orthography (vocab 2048, BPE, preserving the velar-nasal apostrophe in `ng'` — a real Kalenjin letter, not punctuation), reinitializes the decoder embedding + joint head to the new vocabulary, and keeps the acoustic encoder warm. Covered dialects: Kipsigis and Nandi.

Results (KaleBench-ASR, held-out unscripted, n=198)

Character Error Rate (CER) is the primary metric — Kalenjin has no single agreed orthography, which inflates WER. 95% confidence intervals are paired bootstrap (B=2000, seed=1234).

Honest reading:

• —This is the best _open_ Kalenjin ASR model, and it is dialect-balanced (Kipsigis 0.260 ≈ Nandi 0.267), unlike the incumbent's Nandi tilt.
• —Under matched greedy decoding it leads the closed incumbent decisively (the incumbent's greedy CER is 0.483). The incumbent's headline 0.211 uses beam-5 search; Parakeet-TDT in 🤗 Transformers currently supports greedy decoding only, so the comparison is asymmetric in decoding budget.
• —The reported scores are comparable but not yet a formally citable KaleBench score: all 198 clips currently carry needs_native_validation=true. Numbers are reproducible to the digit and use the same gold + normalizer for every system; native by-ear validation (in progress) is what makes them citable.

Usage

Parakeet-TDT's joint head emits token logits and duration logits; during greedy decoding you must mask the duration columns so the argmax can never select a duration slot as a token.

import torch, soundfile as sf
from transformers import (ParakeetForTDT, ParakeetProcessor, ParakeetTokenizer,
                          LogitsProcessorList)

repo = "Tonykip/parakeet-tdt-0.6b-kalenjin"
model = ParakeetForTDT.from_pretrained(repo, dtype=torch.bfloat16).to("cuda").eval()
processor = ParakeetProcessor.from_pretrained(repo)
processor.tokenizer = ParakeetTokenizer.from_pretrained(repo)   # the rebuilt Kalenjin tokenizer

V = model.config.vocab_size
class MaskDurations:                      # keep argmax inside the real vocab
    def __call__(self, input_ids, scores):
        if scores.shape[-1] > V:
            scores[..., V:] = float("-inf")
        return scores

wav, sr = sf.read("clip.wav")             # mono; resample to 16 kHz if needed
feats = processor.feature_extractor(wav, sampling_rate=16000, return_tensors="pt")
feats = {k: (v.to("cuda", dtype=model.dtype) if torch.is_floating_point(v) else v.to("cuda"))
         for k, v in feats.items()}
with torch.no_grad():
    out = model.generate(**feats, logits_processor=LogitsProcessorList([MaskDurations()]))
seq = out.sequences if hasattr(out, "sequences") else out
print(processor.tokenizer.decode(seq[0], skip_special_tokens=True))

Training

• —Base: nvidia/parakeet-tdt-0.6b-v3 (FastConformer + TDT, 0.6B, CC-BY-4.0).
• —Data: `Anv-ke/Kalenjin` (AfriVoices-KE; Wanzare et al., arXiv:2604.08448) — full training corpus, 82,378 clips, streamed (~3.4 passes over clips ≤32s).
• —Tokenizer: rebuilt Kalenjin SentencePiece (BPE, vocab 2048; ng' preserved as a piece); decoder embedding + joint head reinitialized to the new vocabulary; encoder kept warm.
• —Recipe: L40S (Modal), bf16, gradient checkpointing, per-device batch 8 × grad-accum 4, lr 1e-4, max_steps=7723. A TDT-loss correction (sigma=0.05; the 🤗 default 0.0 collapses training to empty output) and int64 gather indices were applied.

Intended use & limitations

• —Intended: transcription of spoken Kalenjin (Kipsigis / Nandi), research on low-resource African ASR, and as a baseline for the KaleBench-ASR benchmark.
• —Limitations: only Kipsigis + Nandi are covered (no Marakwet/Sabaot/Tugen/Pokot evaluation); greedy decoding only (no beam in Transformers for TDT); code-switched English/Swahili spans are transcribed phonetically rather than preserved; training transcripts are themselves unvalidated, which likely caps accuracy. Not intended for surveillance, profiling, or any non-consensual use.

License & attribution

This fine-tune and rebuilt tokenizer are released under CC-BY-4.0, inheriting the base model's license. The training audio comes from Anv-ke/Kalenjin (CC-BY-4.0, gated) — please cite AfriVoices-KE (arXiv:2604.08448) in any derivative work. Built by Tony Kipkemboi, a native Kalenjin speaker.

Citation

@misc{kipkemboi2026parakeetkalenjin,
  title  = {Open Kalenjin Automatic Speech Recognition: Fine-tuned Parakeet Models and the KaleBench-ASR Benchmark},
  author = {Kipkemboi, Tony},
  year   = {2026},
  note   = {Preprint}
}