AirCaps/mega-asr-noise-a5sv2

Mega-ASR Noise A5SV2

Mega-ASR Noise A5SV2 is a deterministic, English-only robustness evaluation subset derived from `zhifeixie/Voices-in-the-Wild-2M`, the training corpus released with Mega-ASR.

We sampled from Mega-ASR-Train, rather than the standard Mega-ASR test set, because in our experiments the standard test set was not acoustically challenging enough to clearly discriminate among robust ASR systems. This is a derived evaluation set; it should not be mixed into training when reporting results on it.

The a5sv2 suffix identifies the robustness-evaluation streaming model for which this subset was assembled. The audio and reference transcripts remain Mega-ASR derived.

Contents

The dataset contains 1,250 unique utterances (3.6465 hours), with 250 examples in each split:

All files are mono WAV audio. Original sample rates are preserved (16, 24, or 48 kHz). Every row includes the embedded audio, the original Mega-ASR reference text, saved A5S and ElevenLabs predictions, upstream sample identifiers, duration and signal metadata, the audio SHA-256, and the pinned upstream revision.

Reference transcripts

The text column is copied from Mega-ASR's original text field.

Saved system predictions

Each row contains two raw prediction fields:

• —a5s_prediction: output from the AirCaps A5S streaming ASR system, evaluated with greedy, cache-aware streaming.
• —elevenlabs_prediction: output from ElevenLabs Scribe v2 Realtime, using an independent WebSocket session per utterance, forced English, mono PCM16 at 16 kHz, and 100 ms chunks paced in wall-clock real time.

These predictions are provided for matched comparison and error analysis. They are not reference labels. Empty prediction strings are preserved where a system emitted no transcript.

Construction

• —Upstream repository: zhifeixie/Voices-in-the-Wild-2M
• —Pinned revision: a8a35d3319737190d6fd3d39157b258eaab35980
• —Deterministic selection seed: 20260823
• —Shuffle buffer: 512
• —English selection: a non-empty reference/instruction row with no Han characters, appropriate for the upstream English/Chinese corpus
• —Selection target: 250 unique examples from each listed acoustic condition
• —Final checks: 1,250 unique source names, 1,250 unique audio hashes, full audio decode/metadata checks, and SHA-256 verification

The provenance/ directory contains the available original selection metadata and the materialization code used to construct the saved subset. AUDIT_REPORT.json records the final counts, hours, sample-rate distribution, and integrity results.

Intended use

This dataset is intended for evaluating ASR robustness under difficult noise, far-field, recording, and obstruction conditions. Report results per split as well as an aggregate, since the five conditions have different acoustic failure modes.

This subset is sampled from a training corpus, not from a speaker-disjoint held-out benchmark. It is therefore best used as a fixed diagnostic suite, and results should describe this provenance explicitly.

Loading

from datasets import load_dataset

dataset = load_dataset("AirCaps/Mega-ASR-A5SV2")
example = dataset["far_field_noise"][0]
print(example["text"])

Recent versions of datasets may require torchcodec to decode the embedded audio column. Use cast_column("audio", Audio(decode=False)) if only the encoded bytes and metadata are needed.

License and attribution

The upstream dataset is released under Apache-2.0. Users should also review the upstream dataset card and Mega-ASR paper for source provenance and limitations.

@misc{xie2026megaasrinthewild2speechrecognition,
  title={Mega-ASR: Towards In-the-wild^2 Speech Recognition via Scaling up Real-world Acoustic Simulation},
  author={Zhifei Xie and Kaiyu Pang and Haobin Zhang and Deheng Ye and Xiaobin Hu and Shuicheng Yan and Chunyan Miao},
  year={2026},
  eprint={2605.19833},
  archivePrefix={arXiv},
  primaryClass={cs.SD},
  url={https://arxiv.org/abs/2605.19833}
}