h-gajdov/full_moonshine_base_all_data_wup1_ep3_lr1e-4

fullmoonshinebasealldatawup1ep3_lr1e-4

Full fine-tune of UsefulSensors/moonshine-base (61.5M parameters) for Macedonian speech-to-text, trained on the full ~2.99M-clip corpus of the study.

Moonshine is a raw-waveform encoder-decoder: it takes input_values rather than Whisper's fixed 30 s log-mel window, so its cost scales with actual audio length instead of being flat per clip. That is what makes it viable on-device, and it is why its real-time factor barely moves with utterance length where Whisper's does.

This is one of eight runs sweeping peak learning rate and warmup ratio over an otherwise identical recipe -- the same sweep already run on moonshine-tiny, at 2.2x the parameters. All eight are in this collection.

Evaluation

Held-out test set rachno_provereno_od_yt (44 manually verified clips, 0.32 h), never seen in training. CPU, float32, batch 1, greedy (1 beam), no silence padding, no length floor.

SER is exact-match over whole utterances. These test clips average 62 words, so at this WER an exactly-correct utterance is vanishingly unlikely -- read WER/CER.

Best WER seen during training (same held-out set): 8.96 at step 204,000. That comes from the trainer's own in-loop generation -- batched, greedy -- so it is a training signal rather than a number to compare across projects; on these eight runs it lands within 0.25 WER of the table above.

This checkpoint is not the end of its schedule. The run was stopped at step 204,000 of a planned 284,226. Its last evaluation is also its best, so WER had not started regressing when it stopped. The eight runs in the sweep stopped at different points (epoch 0.85 to 2.60 of their schedules), so the ranking below partly reflects how far each one got, not learning rate alone.

Compared with the other models

[image]

Ranked by WER this model is 4 of 8 in the sweep.

All three decoded in the same harness, same clips, same CPU, batch 1. The fine-tuned Whisper is the more accurate model and will stay that way; what it costs is 3.9x the latency per utterance at 13x the parameters. Against the tiny fine-tune this model is 1.09 WER better for 2.1x the latency -- the base-vs-tiny trade this sweep exists to measure.

The untrained moonshine-base sits at 191.55 WER on this set: the stock checkpoint is English-only, so it is a floor reference, not a baseline.

The sweep

Training

Corpus (~2.99M clips, each capped at 30 s): vezilka-asri (~2.96M clips, ~99% of the mix), videa_so_transkript_od_yt (6,481), mozzila_common_voice (5,205), alfa_audios (3,896), doniraj (3,157 accepted donations), sitel_audios (3,136), fleurs_mk (1,853), jargon (1,274). The dialect repos and the held-out test set are excluded by construction.

Usage

from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq
import librosa, torch

model_id = "h-gajdov/full_moonshine_base_all_data_wup1_ep3_lr1e-4"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForSpeechSeq2Seq.from_pretrained(model_id).eval()

wav, _ = librosa.load("clip.wav", sr=16000, mono=True)
feats = processor.feature_extractor([wav], sampling_rate=16000,
                                    return_tensors="pt", padding=True)
with torch.no_grad():
    ids = model.generate(**feats, max_length=192)
print(processor.batch_decode(ids, skip_special_tokens=True)[0])

max_length is capped at 192 because the decoder has 194 positions (max_position_embeddings). Moonshine has no forced language/task tokens, so do not pass language= or task= to generate() -- it will raise.