DedeProGames/SoralASR-23M-Preview

SoralASR-23M-Preview

Preview status: experimental research checkpoint. This model is not production-ready and its current transcription quality is limited.

SoralASR-23M-Preview is a compact English automatic speech recognition model with 23,062,272 parameters. It uses a reduced Whisper-style encoder-decoder architecture and was trained from randomly initialized ASR weights. The Whisper English tokenizer vocabulary and preprocessing format were reused for compatibility; pretrained Whisper model weights were not loaded.

Model details

Training

• —Dataset: LibriSpeech ASR, clean/train.100.
• —Available unique training audio: approximately 100 hours of clean read English speech.
• —Optimization: 50,000 optimizer updates on an NVIDIA T4.
• —Effective batch size: 8 utterances.
• —Decoder training tokens processed: 15,699,868.
• —The checkpoint with the best small validation score was selected instead of the final-step checkpoint.

Indicative evaluation

The published checkpoint reached WER 0.984 (98.4%) on the first 96 eligible utterances from LibriSpeech clean validation.

This is only a small checkpoint-selection evaluation. It is not a full LibriSpeech evaluation and not an official Open ASR Leaderboard result. The score and observed repetitive decoding indicate that the model is not yet suitable for reliable transcription.

Usage

from transformers import pipeline

transcriber = pipeline(
    "automatic-speech-recognition",
    model="DedeProGames/SoralASR-23M-Preview",
    chunk_length_s=20,
    stride_length_s=(2, 2),
)

result = transcriber("english_audio.wav")
print(result["text"])

Audio longer than 20 seconds

The model has a 20-second acoustic window. Long recordings must be split into overlapping chunks and recomposed. The pipeline configuration above enables chunking; do not pass long audio through a path that silently truncates it.

For best results, use mono English speech at 16 kHz. The bundled processor can resample supported audio inputs.

Intended use

This preview is intended for research, architecture experiments, continued training, and educational inspection of a small ASR system trained from scratch. It should not be used where transcription accuracy is safety-critical or where errors could materially affect people.

Limitations

The training data is clean audiobook speech from a single domain. Expect substantial errors, repetition, hallucinated phrases, and limited robustness to noise, accents, conversational speech, music, overlapping speakers, and non-English audio. No accuracy guarantee is made.

Attribution

LibriSpeech: Panayotov et al., LibriSpeech: An ASR Corpus Based on Public Domain Audio Books, ICASSP 2015. Dataset license: CC BY 4.0.

Tokenizer vocabulary and preprocessing compatibility derive from OpenAI Whisper.

Open ASR Leaderboard evaluation

On 2026-09-23, the published checkpoint was evaluated on the eight English short-form test sets in the current Open ASR Leaderboard scorer. These are self-reported T4 results, not independently verified results or an official leaderboard placement. The older ESB description retained on the benchmark dataset card lists a different historical selection of tests; the eight sets below follow the scorer's current English short-form columns.

The structured results in `.eval_results/open_asr_leaderboard.yaml` are parsed by the Hub for this model. The Hub benchmark dataset leaderboard and the separately curated Open ASR Leaderboard Space are distinct views. The Space reads the maintainers' `english_short_latest.csv`; uploading this YAML does not add a row to that CSV. As of 2026-09-23, this model was absent from both the Hub dataset leaderboard API and the Space results CSV. No official rank is claimed.

Mean WER: 123.86%; aggregate RTFx: 121.74 on an NVIDIA T4. The mean is the arithmetic average of the eight per-set WERs rounded to two decimals. WER is (substitutions + deletions + insertions) / reference words; lower is better, and repetitions or other insertions can push it above 100%. Aggregate RTFx is the total 588203.26 seconds of audio divided by 4831.51 seconds of measured inference (warmup excluded). RTFx on a T4 is not directly comparable with the leaderboard project's H200 throughput measurements.

The model revision was ec5db5710827ad648f58f9a51569c39bde172649. For every input, the Transformers ASR pipeline split audio at 20 seconds with 2-second overlap on each side and recomposed the prediction; longer recordings were never silently truncated. Inference used float16, batch size 8, one warmup batch and the published generation defaults. WER used the leaderboard's English text normalizer and kaldialign.batch_error_rate(..., merge_compounds=True). Job links above contain the exact commands, sample counts and timing; the upstream scorer documents the normalization and aggregation.

Earnings22 uses 341 non-overlapping audio chunks from the chunked dataset, grouped by parent_id and ordered by chunk_index. Predictions for each call were concatenated before comparison with its single parent transcript. Scoring each chunk against the full parent transcript would give an invalid result. The six calls yielded WER 131.76%.

SPGISpeech shards

The complete, duration-sorted 39,341-row test split was partitioned into four disjoint slices with roughly 25 hours of audio each:

• —test[0:6710]: Job; errors 298337 / 241506 reference words.
• —test[6710:14982]: Job; errors 305064 / 241869 reference words.
• —test[14982:25229]: Job; errors 304980 / 243244 reference words.
• —test[25229:39341]: Job; errors 296326 / 242034 reference words.

Their total substitutions, insertions and deletions were divided by total reference words to obtain WER 124.37%. Summed duration / summed inference time gives RTFx 129.51.

Dataset revisions used: hf-audio/open-asr-leaderboard@b6bdcd0beb34f8975dc659796176d88f43aff502, ArtificialAnalysis/Earnings22-Cleaned-AA-chunked@27f7d0056d593231974a752ce03c1e965da69b3f, parent transcripts ArtificialAnalysis/Earnings22-Cleaned-AA@5999936ee3d8d3cde160e3c65a8cc6ff5b7c5276, and VoiceArena/MonsoonASR-Open-ASR-leaderboard-en-IN@bc1da7b42ef6e2853123c97bf6d22067e4802d11.

The Hub benchmark's `eval.yaml` defines matching individual task IDs for LibriSpeech clean/other, SPGISpeech and cleaned VoxPopuli. It does not currently define matching task IDs for the evaluated cleaned AMI, cleaned GigaSpeech, cleaned Earnings22 and Monsoon sets. Their WERs are included in the eight-set mean, listed above and recorded in the mean result's notes; they have not been mislabeled as older, different per-set tasks. The model's high WER and repetitive outputs make this preview unsuitable for reliable transcription.