soaring0616/asr-semantic-probe-eng

ASR Semantic Probing Dataset (English)

Synthetic English audio dataset for probing whether ASR encoder representations encode semantic category information beyond acoustic features. Constructed for mechanistic interpretability studies of speech recognition models.

Dataset Description

• —Created: 2026-05-09
• —Total samples: 550
• —Categories: 11 semantic categories × 50 words each = 550 single-word utterances
• —Audio: 16kHz mono WAV, 16-bit PCM
• —TTS engine: Coqui TTS, VCTK VITS multi-speaker model
• —Speakers: 109 VCTK speakers, randomly assigned per word with seed=42

Splits

This dataset is released as a single unsplit collection. Downstream users are expected to define their own train/test splits based on the experimental design. For probing experiments where speaker confounds are a concern, we recommend splitting on speaker (i.e. ensuring train/test speaker pools are disjoint).

Categories

• —academic_subjects (50 words)
• —animals (50 words)
• —body_parts (50 words)
• —clothing (50 words)
• —countries (50 words)
• —fruits (50 words)
• —musical_instruments (50 words)
• —professions (50 words)
• —tools (50 words)
• —transportation (50 words)
• —weather (50 words)

Intended Use

This dataset is designed to support layer-wise linear probing experiments on ASR encoder representations, following the methodology of Glazer et al. (2025). Each utterance contains a single content word; for a pair of categories, a binary linear probe is trained on encoder activations to assess whether semantic distinctions are linearly decodable from intermediate layers.

Dataset Construction

Words were selected to have at least 2 syllables where possible, to ensure sufficient frame-level activations after typical encoder subsampling (e.g. 4× downsampling in Conformer-based models with 10ms frame shift yields ~40ms per frame). Cross-category overlap was checked and avoided.

Each word was synthesized with a single VCTK speaker, randomly assigned with a fixed seed (42) for reproducibility. Audio was synthesized at 22050Hz (VCTK VITS native rate) then resampled to 16kHz to match the input sampling rate of common ASR models (Whisper, FireRedASR, etc.)

Loading

from datasets import load_dataset

ds = load_dataset("soaring0616/asr-semantic-probe-eng", split="train")
sample = ds[0]
print(sample["word"], sample["category"])
audio_array = sample["audio"]["array"]   # numpy float32, 16kHz

Limitations

• —Synthetic audio only; not representative of natural speech variability (prosody, disfluency, channel noise, etc.).
• —VCTK speakers cover predominantly British, Irish, and a few American accents; results may not generalize to other accents or languages without adaptation.
• —One sample per word; no within-word speaker variation. Users concerned about speaker confounds should refer to evidence that speaker identity and linguistic content occupy approximately orthogonal subspaces in self-supervised speech representations.
• —50 words per category is small; probe accuracy estimates may be noisy.

Citation

If you use this dataset, please cite:

• —TTS: Coqui TTS (idiap fork): https://github.com/idiap/coqui-ai-TTS
• —VCTK speakers: Yamagishi et al., CSTR VCTK Corpus, University of Edinburgh
• —Probing methodology: Glazer et al., "Beyond Transcription: Mechanistic Interpretability in ASR", arXiv:2508.15882, 2025

License

CC BY 4.0 (consistent with VCTK and the underlying TTS models)