CoolFace
Modelpublic

PrinceAlhassanNasamu/tekyerema-audio-native-4k

sourceHugging Facecc-by-nc-4.0updated 4d agoView on Hugging Face
0likes
Model Card

Tɛkyerɛma-1 audio-native, 4,000 clips (arm 3 v2)

Author: Prince Nasamu Alhassan

Overview

Qwen2-Audio-7B-Instruct + LoRA, trained on 4,000 synthesised clips instead of the 1,200 used by tekyerema-audio-native.

Measured on utterances it has genuinely never seen — 60 rows whose text is absent from the training slice, after skipping 139 candidates that appeared there verbatim:

pathcorrect
audio → ASR → tekyerema-1-native-tool5/52 · 9.6%
audio → tekyerema-audio-native (1,200 clips)23/52 · 44.2%
audio → this model (4,000 clips)45/52 · 86.5%

An earlier version of this card said 92.3%, and that was memorisation. The corpus has 8,488 rows and 2,943 distinct utterances — 65% duplicates, single sentences repeated up to 25 times — so holding out by ROW INDEX left 40 of 60 test rows appearing word for word in training, and the bigger model looked better mainly because it had seen more of the test. AGENT/eval_audio_native.py now splits by utterance and prints what it skipped.

The conclusion survived the correction and got cleaner: more data genuinely helps this architecture (44.2 → 86.5), and it genuinely beats the cascade — by more than the leaky measurement showed, because removing the leaked rows cost the cascade more than it cost this model.

Training loss moved 4.649 to 4.424 between the two runs, which is worth noting on its own: loss on a generative model averages over every token and tracked neither the inflated score nor the real one.

Use it

python
import torch, soundfile as sf
from peft import PeftModel
from transformers import AutoProcessor, Qwen2AudioForConditionalGeneration

BASE = "Qwen/Qwen2-Audio-7B-Instruct"
proc  = AutoProcessor.from_pretrained("PrinceAlhassanNasamu/tekyerema-audio-native-4k")
model = PeftModel.from_pretrained(
    Qwen2AudioForConditionalGeneration.from_pretrained(
        BASE, dtype=torch.bfloat16, device_map="auto"),
    "PrinceAlhassanNasamu/tekyerema-audio-native-4k").eval()

convo = [{"role": "system",
          "content": "Convert the spoken command into one tool call as JSON."},
         {"role": "user",
          "content": [{"type": "audio", "audio_url": "x.wav"}]}]
prompt = proc.apply_chat_template(convo, add_generation_prompt=True,
                                  tokenize=False)

wav, sr = sf.read("command.wav", dtype="float32")
inp = proc(text=prompt, audio=[wav], sampling_rate=sr,
           return_tensors="pt").to(model.device)
with torch.no_grad():
    out = model.generate(**inp, max_new_tokens=48, do_sample=False)
print(proc.batch_decode(out[:, inp["input_ids"].shape[1]:],
                        skip_special_tokens=True)[0])

Training data

Trained on the Ghana Speech dataset and related Ghanaian corpora, licensed CC BY-NC 4.0.

Intended use & license

Non-commercial use only (CC BY-NC 4.0). This is inherited from the training data and required by the terms under which the compute was granted: models trained in that window are non-commercial by condition of access, not by inference.

Limitations, stated plainly

  • —Dagbani did get a recogniser, and the claim that it could not was wrong twice over. Every card on this account used to say that "one fine-tuning session on 74 validation rows would not change that". Those 74 rows are the eng-dag machine-translation validation split; the Dagbani speech data in this same account is waxal_dag — 13,228 training rows, 1,750 validation rows, ~71 hours, 1,041 speakers with the largest at 1%. Trained on it, tekyerema-asr-mms-dag scores 36.94 / 11.71, against the 86.59 / 33.95 this project had believed was the ceiling. It still loses to FarmerlineML/w2v-bert-2.0_2026_dagbani_ASR at 29.20 / 9.27, which is what the agent actually serves. A number carried across from a translation table into a speech claim was then repeated on every card here until 2026-09-22.
  • —Evaluation is on read and machine-translated text. No recordings of people speaking agent commands in these languages exist. Numbers measured this way are optimistic about phrasing and pessimistic about code-switching, and should not be read as field performance.
  • —Research work from a hackathon entry, not a supported product.

The rest of the family

Recognisers

Voices

Agent models

Translation

Routing

Acknowledgements

Compute resources provided by AI Skills and Compute Africa (AISCA). Trained on the Ghana NLP H200 GPU. Please keep derivatives non-commercial and share improvements back with the Ghana NLP community (ghananlpcommunity).