alefiury/Echoes-Platos-Cave

<h1 align="center">Echoes in Plato's Cave — Controlled Speech–Text Corpus</h1>

<p align="center"> <a href="#citation"> <img src="https://img.shields.io/badge/paper-coming%20soon-lightgrey" alt="Paper"> </a> <a href="https://github.com/alefiury/Echoes-Platos-Cave"> <img src="https://img.shields.io/badge/GitHub-Code-black?logo=github" alt="Code"> </a> </p>

Controlled corpus of 14,400 synthetic English utterances in which the same 600 sentences are rendered by 6 speakers × 4 emotions, so that speaker identity and prosody vary while linguistic content is held fixed. It was built for the paper: Echoes in Plato's Cave: Measuring Global and Local Alignment Between Speech and Language Representations, accepted as an oral presentation at the Speech and Audio Language Models Workshop (SALMA 2026), co-located with EMNLP 2026. It is used to measure local and global representational alignment between independently trained speech encoders and text encoders. Each row includes the synthesized waveform in the audio column. Github Repository.

Configurations

How the corpus was built

Sentences (SICK). The training, validation and test splits of SICK were pooled. Pairs were removed if either sentence had fewer than 4 or more than 30 words, was a question or exclamation, or contained URLs/HTML; identical and reversed-duplicate pairs were dropped. The remaining pairs were split into three relatedness bands; low (≤ 2.3), medium (3.0–3.7), high (≥ 4.2); and 100 pairs were sampled from each band with the constraint that no sentence appears in more than one selected pair. Result: 300 pairs, 600 unique sentences (mean relatedness 1.44 / 3.34 / 4.65 per band). Entailment labels were not rebalanced.

Reference prompts (RAVDESS). Three male and three female RAVDESS actors (ids 4, 6, 8, 11, 15, 21) and the emotions neutral, happy, sad, angry. All reference recordings contain the same statement ("Dogs are sitting by the door"), so prompt content is constant across speakers and emotions. Candidate recordings were screened with emotion2vec+ large; those whose predicted emotion disagreed with the RAVDESS label were discarded, and one recording per speaker–emotion cell was kept by prioritizing emotional intensity and then classifier confidence → 24 reference prompts.

Synthesis. Every sentence was synthesized under all 24 reference conditions with four zero-shot TTS systems — IndexTTS2, OmniVoice, Qwen3-TTS and VoxCPM2 — each receiving the same reference waveform (and the prompt transcript when supported): 600 × 24 × 4 = 57,600 candidates.

Candidate selection. For each (sentence, speaker, emotion) cell the four candidates were ranked by:

1. 1.Agreement between the intended emotion and the emotion2vec+ large prediction;
2. 2.emotion2vec+ large confidence for the intended emotion;
3. 3.Mean WER across Parakeet-TDT-0.6B-v3, Qwen3-ASR-1.7B and Whisper-large-v3;
4. 4.Mean CER across the same three ASR systems;
5. 5.Fixed tie-break order: OmniVoice, VoxCPM2, IndexTTS2, Qwen3-TTS.

If no candidate is classified as the target emotion, the one with the highest target-emotion probability is kept. This yields the 14,400 utterances in this dataset.

Sources and models

Columns

WER/CER are fractions (0.0 = perfect). A handful of rows have null for one ASR system where it produced no usable transcript; wer_avg/cer_avg are computed over the available systems.

all_candidates columns

sample_id, audio, relative_path, filename, tts_model, speaker_id, gender, emotion (intended emotion), text, word_count, language, and selected (true for the 14,400 rows that appear in filtered). Per-candidate emotion and ASR scores are only provided for the selected subset.

Corpus quality (computed from this release)

Agreement with the intended emotion (emotion2vec+ large) — overall 80.6%

Transcription quality

92.6% of utterances are transcribed without word errors by all three systems and 97.4% have mean WER below 10%.

TTS Models Representation

Share (%) of selected utterances contributed by each system, per intended emotion:

Loading

from datasets import load_dataset

# selected corpus (default config)
ds = load_dataset("alefiury/Echos-Platos-Cave", "filtered", split="train")
row = ds[0]
print(row["text"], ds.features["orig_emotion"].int2str(row["orig_emotion"]))
print(row['audio']['sampling_rate'], row['audio']['array'].shape)

# every candidate from all four TTS systems
cands = load_dataset("alefiury/Echos-Platos-Cave", "all_candidates", split="train")
one_cell = cands.filter(lambda r: r["sample_id"] == "sick_sentence_000001" and r["speaker_id"] == 4)

# e.g. all utterances of one sentence across speakers and emotions
same_sentence = ds.filter(lambda r: r["sample_id"] == "sick_sentence_000001")

The raw CSVs used to build this release are in metadata/.

Limitations

The corpus is synthetic speech from six speakers, four emotions and four TTS systems, so findings may not transfer directly to natural speech. Emotion and transcription scores come from the same models used for candidate selection and are therefore not independent quality estimates. Emotion agreement is high for angry and neutral but considerably lower for happy and sad (see table above).

Citation

If you use this dataset, please cite the paper:

@inproceedings{ferreira2026echoes,
  title     = {Echoes in Plato's Cave: Measuring Global and Local Alignment Between Speech and Language Representations},
  author    = {Ferreira, Alef Iury Siqueira and Gris, Lucas Rafael Stefanel and de Oliveira, Frederico Santos and Galv{\~a}o Filho, Arlindo Rodrigues and Soares, Anderson da Silva},
  booktitle = {Proceedings of the Speech and Audio Language Models Workshop (SALMA)},
  year      = {2026}
}

and the underlying resources: SICK (Marelli et al., 2014), RAVDESS (Livingstone & Russo, 2018), emotion2vec (Ma et al., 2023), and the TTS / ASR systems listed above.

License

Sentence text comes from SICK and reference voices from RAVDESS, both distributed under CC BY-NC licences; this dataset is released under CC BY-NC 4.0. The synthesized audio is subject to the terms of the respective TTS systems.