Tilas/qwen2-audio-darija-marco-30h

Qwen2-Audio-7B fine-tuned on Moroccan Darija (30h, Marco-ASR Algorithm 1)

LoRA adapter for Qwen/Qwen2-Audio-7B-Instruct, fine-tuned for Moroccan Darija (الدارجة) ASR. Part of a Phase 1 study comparing end-to-end audio-LLM (Qwen2-Audio) vs cascaded ASR (Whisper-LV3) under an identical protocol.

Training data

• —atlasia/MoulSot-Full, config 100-gt-2.5, 30h sampled with seed=42 (after carving 2000 val samples).
• —Test: publisher's official MoulSot test split. OOD: 500 samples from UBC-NLP/Casablanca (Morocco).

Methodology

Marco-ASR (Ni et al. 2025), Algorithm 1 (single WER-driven adaptive LR).

• —LoRA: r=32, alpha=32, dropout=0.1, target_modules=all-linear, rsLoRA=True
• —Effective batch 16 (per-device=4, grad-accum=4), BF16, SDPA
• —Adaptive LR + early stopping (patience=3, min_delta=0.5pp) on a held-out val subset

Training run

• —Wall-clock: 8.89 GPU-hours
• —Best step: 2200 (val WER 39.73% on held-out subset)
• —WER₀ at start: 124.79%

Evaluation

Casablanca normalization (Talafha et al. 2024); code-switching partitioning; 95% CIs from 1000-resample bootstrap.

MoulSot test (in-distribution)

Casablanca Morocco (OOD)

Usage

from transformers import Qwen2AudioForConditionalGeneration, AutoProcessor
from peft import PeftModel
import torch, librosa

base = Qwen2AudioForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2-Audio-7B-Instruct", torch_dtype=torch.bfloat16, attn_implementation="sdpa").to("cuda")
model = PeftModel.from_pretrained(base, "Tilas/qwen2-audio-darija-marco-30h").to("cuda").eval()
processor = AutoProcessor.from_pretrained("Tilas/qwen2-audio-darija-marco-30h", trust_remote_code=True)

audio, sr = librosa.load("darija.wav", sr=16000)
conv = [{"role": "user", "content": [
    {"type": "audio", "audio_url": "darija.wav"},
    {"type": "text", "text": "Transcribe speech to text "}]}]
text = processor.apply_chat_template(conv, add_generation_prompt=True, tokenize=False)
inputs = processor(text=text, audios=[audio], sampling_rate=sr, return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(processor.batch_decode(out[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0])

Limitations

Trained on 30h of Darija; coverage limited to MoulSot 100-gt-2.5. This is an adapter — load on top of the base model.

License

Inherits from base model: see Qwen/Qwen2-Audio-7B-Instruct.