dys-asr/parakeet-rnnt-0.6b-all-syn-chunk-cutout-fade-bpedrop

Parakeet RNN-T 0.6B — FadeOutIn and BPE-dropout together

The ten-epoch recipe behind `dys-asr/parakeet-rnnt-0.6b-all-syn-chunk-cutout`, rerun with both FadeOutIn and BPE-dropout at p = 0.1. The two single-factor arms exist so this one can be attributed.

What it is worth

On the held-out dev split, against the run it varies:

Treat both columns with suspicion. The subset and the full split disagree about the ordering across this family of runs, the spread is under 0.2 CER points, and there is one seed behind each row. The full split is 17,492 utterances against the subset's 4,000 and is the more trustworthy of the two, but neither is a significance test.

Every run in this family peaked at epoch 9 or 10 with no sign of overfitting, so ten epochs may simply be short.

The two together

FadeOutIn at sixteen segments of 2%, and BPE-dropout at 0.1; the sibling cards describe each. On this corpus the combination scored between the two single arms rather than below both, which is the one thing these four runs say about whether they compose. One seed, so not much.

Training data

The same corpus as `dys-asr/parakeet-rnnt-0.6b-all-syn-chunk-cutout`, unchanged, so the two are directly comparable.

463,177 records survive the 0.5-45 s and 200-label-token filters. Nothing comes from outside the challenge corpora, so this is a constrained-track model.

The held-out dev split is genuinely held out. sapc2_dev_heldout.jsonl is 17,582 clips and its intersection with the training manifests is empty; that was checked by comparing file paths, not assumed.

Training procedure

Ten epochs on sixteen GH200s, effective batch 32 at per-device 2, AdamW at 1e-4 with a tri-stage schedule (10% warmup, 40% hold), weight decay 0.01, layerdrop 0.05, gradient clip 1.0, bf16, seed 42, 144,750 optimizer steps. Augmentation is online speed perturbation over 0.8-1.2, SpecAugment (5% of the time axis in spans of 10 frames, 40% of the mel axis in spans of 27 bins) and SpecCutout (two 20x20 rectangles).

Usage

import soundfile as sf
import torch
from transformers import AutoProcessor, ParakeetForRNNT

model_id = "dys-asr/parakeet-rnnt-0.6b-all-syn-chunk-cutout-fade-bpedrop"
device = "cuda" if torch.cuda.is_available() else "cpu"

processor = AutoProcessor.from_pretrained(model_id)
model = ParakeetForRNNT.from_pretrained(model_id).to(device).eval()

audio, rate = sf.read("utterance.wav", dtype="float32")
inputs = processor(audio, sampling_rate=rate, return_tensors="pt").to(device)
with torch.inference_mode():
    generated = model.generate(**inputs)
print(processor.batch_decode(generated, skip_special_tokens=True)[0])

Requires transformers>=5.9. Audio must be 16 kHz mono.

Intended use and limitations

A competition entry for the Speech Accessibility Project Challenge, and research on atypical-speech recognition.

• —Single run, single seed. No variance estimate. The differences reported above are smaller than a seed sweep would likely show, and should not be read as a ranking of methods.
• —Requires `transformers>=5.9`. Not loadable by the 4.x line.
• —Slow relative to CTC. Decoding is autoregressive.
• —Lower-case, unpunctuated, numerals written as words.
• —English only, 16 kHz mono.
• —Not a clinical tool. Nothing here supports inference about any diagnosis.

License and attribution

The Speech Accessibility Project corpora are governed by their own data use agreement and are not redistributed here; reproducing this training set requires authorized access. Fun-CosyVoice3's terms apply to the synthesis component.