dys-asr/parakeet-ctc-0.6b-all-aug

parakeet-ctc-0.6b-all-aug

nvidia/parakeet-ctc-0.6b fine-tuned on every Speech Accessibility Project recording available -- SAPC1 train, SAPC1 dev and SAPC2 train, deduplicated -- with online speed perturbation and SpecAugment.

It is `dys-asr/parakeet-ctc-0.6b-all` with augmentation added and nothing else changed, so the pair is a clean ablation:

All rows are the same 17,582 utterances from 48 speakers, greedy CTC decoding, no language model, both sides through the same normaliser.

Augmentation is worth 0.43 WER points here, on identical data and an identical recipe. It also puts this model ahead of the weight-averaged soup, which was previously the best of the family.

What the augmentation is

Both are applied online in the collator, so the batch size never changes and neither costs disk:

• —Speed perturbation at 0.9x, 1.0x or 1.1x, one factor drawn per utterance. The waveform is resampled by a rational ratio and the sampling rate left declared as it was, which shifts tempo and pitch together. Unlike the usual offline recipe this does not triple the corpus, so an epoch is still an epoch.
• —SpecAugment, masking spans of the mel spectrogram along both axes. Roughly 5% of frames and 34% of the 80 mel bins, the latter matching what NeMo's Conformer recipes average. Transformers' Parakeet port has no SpecAugment, so the span sampler is borrowed from its Wav2Vec2 implementation.

Speed runs before feature extraction, so the frame count and attention mask follow the perturbed audio; the time mask honours that attention mask, so spans land inside each utterance rather than in padding where they would mask nothing.

Read the evaluation set carefully

Training includes SAPC1 dev, which shares 28,253 recordings and 76 of SAPC2 dev's 124 speakers, so neither sapc1_dev nor the whole of sapc2_dev is a fair test for this model. The set used above is the 48 SAPC2 dev speakers appearing nowhere in training, 17,582 of 47,929 records.

That set is harder than SAPC2 dev as a whole: all 35 of SAPC2 dev's Parkinson's speakers also appear in SAPC1 dev, so it contains no Parkinson's speech, easily the lowest-error cohort here. Do not compare this model's 11.65% against parakeet-ctc-0.6b-sapc2's published 11.51% on all of SAPC2 dev; they are different sets. Scored head to head, as above, this model wins by 1.70 points.

Output text convention

Numbers as words, upper-case, unpunctuated, as for every model in this family.

audio:  "lower the temperature three degrees"
output: LOWER THE TEMPERATURE THREE DEGREES     # not "... 3 DEGREES"

The tokeniser holds 1,025 tokens and no digit characters. Verbalise your references the same way or numerals will dominate your error count.

Training data

SAPC1 train + SAPC1 dev + SAPC2 train, deduplicated by audio filename, filtered to 0.5-30 s: 392,378 utterances, 752.6 h, 1,055 speakers. SAPC2 re-releases 182,575 of SAPC1's train recordings under the same filenames, so concatenating the manifests would train on them twice and double the weight of the 483 speakers in both.

Square-bracketed spans, which in SAPC2 hold the interview prompt shown to the speaker rather than read aloud, are stripped before normalisation. Leaving them in trains the model to transcribe a question nobody asked.

Training hyperparameters

Identical to parakeet-ctc-0.6b-all apart from the two augmentation rows.

Dev WER by epoch, still falling at the end:

The trainer's metric reads slightly optimistic against a standalone pass: 11.25% there against the 11.65% reported above.

Results by speaker

Pooled over words the figure is 11.65%; averaged with equal weight per speaker it is 14.32%:

20 of the 48 speakers are under 10% WER; 3 are above 30%.

Usage

import torch
from transformers import AutoModelForCTC, AutoProcessor

model_id = "dys-asr/parakeet-ctc-0.6b-all-aug"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForCTC.from_pretrained(model_id).eval()

inputs = processor(audio, sampling_rate=16_000, return_tensors="pt")  # 16 kHz mono
with torch.inference_mode():
    logits = model(**inputs).logits
print(processor.batch_decode(logits.argmax(dim=-1)))

Augmentation is training-time only and has no effect at inference.

Limitations

• —No fair comparison against the sibling models on their own test sets, since this trains on SAPC1 dev.
• —The held-out set excludes Parkinson's speech entirely, so it is harder than SAPC2 dev and the number is not interchangeable with the siblings'.
• —Numerals are written as words; no punctuation or casing.
• —Accuracy is very uneven across speakers, 0.8% to 68.8% WER.
• —Not trained to convergence. Dev WER was still improving at epoch 10.
• —Augmentation was not tuned. The masking rates are defaults, not searched.
• —English only, 16 kHz mono.
• —Single training seed. No variance estimate, and the 0.43-point gain over the unaugmented twin is a single-seed difference.