dys-asr/parakeet-tdt-0.6b-all

Parakeet TDT 0.6B — every hour, nothing held back

nvidia/parakeet-tdt-0.6b-v3 fine-tuned on all 1,047.7 hours this project holds: SAPC1 train and dev, SAPC2 train, the SAPC2 dev split the rest of this family scores against, 103.1 hours of synthetic dysarthric speech, 79.2 hours recovered by force-aligning and cutting recordings past the 45-second training cap, and 15.5 hours of AtaxiaUK and HeyJay!, which are outside the challenge corpora and make this an unconstrained-track model.

This model has no reported WER or CER, and cannot have one. Every held-out hour is in its training data. That was the point: the hyperparameters were settled on the sibling runs that do hold out a dev split, and this run spends that split as training data instead of as a measurement. It is a competition entry, not a result.

Read the numbers off `dys-asr/parakeet-rnnt-0.6b-all-syn-chunk-cutout-soup` or its non-souped sibling if you need a measured checkpoint from this family. Nothing here says whether the extra 35.6 hours helped.

Usage

import soundfile as sf
import torch
from transformers import AutoProcessor, ParakeetForTDT

model_id = "dys-asr/parakeet-tdt-0.6b-all"
device = "cuda" if torch.cuda.is_available() else "cpu"

processor = AutoProcessor.from_pretrained(model_id)
model = ParakeetForTDT.from_pretrained(model_id).to(device).eval()

audio, rate = sf.read("utterance.wav", dtype="float32")
inputs = processor(audio, sampling_rate=rate, return_tensors="pt").to(device)
with torch.inference_mode():
    generated = model.generate(**inputs)
print(processor.batch_decode(generated, skip_special_tokens=True)[0])

Requires transformers>=5.9. Audio must be 16 kHz mono.

Training data

An earlier version of this card left the last two rows out, and its own table therefore did not add up: the rows summed to 481,135 records and 1,032.2 hours against a stated total of 491,063 and 1,047.7, and the gap was exactly the 9,928 records and 15.5 hours of external speech. The table above is the run's TRAIN_MANIFEST counted directly.

Filters are 0.5 to 45 seconds and at most 200 label tokens, which dropped 138 utterances. SAPC2 re-releases 182,575 of SAPC1's train recordings under the same filenames, so real audio is deduplicated by filename rather than trained on twice. Square-bracketed spans, which in SAPC2 hold the interview prompt shown to the speaker rather than read aloud, are stripped before normalisation.

The chunked hours

MAX_DURATION=45 discards 4,875 recordings — 89.9 hours, 92% of them Spontaneous Speech Prompts, so what the cap removes is disproportionately the connected speech read prompts cannot stand in for. Each was force-aligned against its own transcript with the project's SAP-fine-tuned CTC checkpoint and cut at the pauses between words, recovering 79.2 hours in 9,014 chunks that fit under the cap. Cuts land at the longest pause in the window between 40 s and the cap, and chunk transcripts are carved out of the source text rather than the normalised text the alignment ran on — normalize_transcript is not idempotent, and storing its output would hand the training collator a transcript its own normaliser then corrupts.

Synthetic speech

Generated with a fine-tuned Fun-CosyVoice3 in zero-shot voice-cloning mode from real SAP reference recordings, sampled within etiology in proportion to the square root of each speaker's baseline CER. It can carry TTS artifacts, can imperfectly reproduce a speaker's impairment, and creates no independent speaker diversity when cloned from voices already in the corpus.

Training procedure

Ten epochs on sixteen GH200s, effective batch 32 at per-device 2, AdamW at 1e-4 with a tri-stage schedule (10% warmup, 40% hold), weight decay 0.01, layerdrop 0.05, gradient clip 1.0, bf16, seed 42, 153,460 optimizer steps.

Augmentation is online speed perturbation over 0.9, 0.95, 1.0, 1.05, 1.1 and SpecAugment at 5% of the time axis in spans of 10 frames and 40% of the mel axis in spans of 27 bins. The speed range is narrower than the 0.8–1.2 the sibling runs use, which also caps the longest perturbed input at 50 s rather than 56 s.

The run took three allocations across three days, resuming from the newest complete checkpoint each time.

Checkpoint selection, and why there is none

With no dev manifest nothing ranks checkpoints, so this run writes no best-N. The published weights are epoch 10, chosen because it is the last one, not because it was measured to be the best. The repository also carries nothing to choose from: on the sibling run that did hold out a dev split, epoch 9 beat epoch 10 (6.06% against 6.09% CER), so "last" is not reliably "best" for this recipe.

The training run kept the last five epochs locally. Averaging them is the obvious thing to try — the same soup of epochs 6–10 was worth 0.27 CER points on the sibling — and it has not been done here.

Intended use and limitations

A competition entry for the Speech Accessibility Project Challenge, and research on atypical-speech recognition.

• —No held-out evaluation exists, or can. Any score computed on SAPC1 dev or SAPC2 dev is a score on training data. Treat any such number as meaningless.
• —Not comparable with its siblings. Every other model in this family reports on a 48-speaker subset of SAPC2 dev that it never trained on. This one trained on all of it.
• —Epoch 10 is the last epoch, not a selected one. See above.
• —Requires `transformers>=5.9`. Not loadable by the 4.x line.
• —Slow relative to CTC. Decoding is autoregressive.
• —Lower-case, unpunctuated, numerals written as words, unlike the CTC models in this family.
• —Single seed. No variance estimate.
• —English only, 16 kHz mono. The multilingual base was fine-tuned on English alone, and that capability is neither preserved nor evaluated.
• —This is an unconstrained-track model. 15.5 hours of its training data — HeyJay! and AtaxiaUK — come from outside the challenge corpora, so it may not be compared with constrained-track entries. A previous version of this card said the opposite; that was wrong, and the training-data table above shows where the claim came apart.
• —Not a clinical tool. Nothing here supports inference about any diagnosis.

License and attribution

The Speech Accessibility Project corpora are governed by their own data use agreement and are not redistributed here; reproducing this training set requires authorized access. Fun-CosyVoice3's terms apply to the synthesis component.