Datasetpublic

anonymous-submission-1/SEABED

SEABED: SouthEast Asian Benchmark for Evaluating Audio Reasoning SEABED (SouthEast Asian Benchmark for Evaluating Audio Reasoning) covers six audio-reasoning tasks: speech emotion recognition, speech affective interpretation, dialect and language identification, dialectal speech comprehension, prosodic ambiguity resolution, and long-form audio reasoning. This repository releases a stratified 10% sample (541 of 5,404 records) of its QA data for anonymous peer review, so reviewers… See the full description on the dataset page: https://huggingface.co/datasets/anonymous-submission-1/SEABED.

sourceHugging Facecc-by-nc-nd-4.0updated 2mo agoView on Hugging Face

0likes33downloads

Dataset Card

SEABED: SouthEast Asian Benchmark for Evaluating Audio Reasoning

SEABED (SouthEast Asian Benchmark for Evaluating Audio Reasoning) covers six audio-reasoning tasks: speech emotion recognition, speech affective interpretation, dialect and language identification, dialectal speech comprehension, prosodic ambiguity resolution, and long-form audio reasoning. This repository releases a stratified 10% sample (541 of 5,404 records) of its QA data for anonymous peer review, so reviewers can inspect the benchmark's question–answer data and verify that every task and every ground-truth label category is present. No model predictions of any kind are included.

Audio is not included - and why

Audio files are deliberately NOT uploaded to this repository. Several of the underlying source corpora are distributed under CC BY-NC-ND 4.0, whose NoDerivatives clause does not permit redistributing the segmented / re-packaged clips used by the benchmark, and other corpora carry their own distribution terms.

If you need the audio, please download it from the original source corpora (all openly available) under their own licenses. Each record's dataset column identifies its source corpus (e.g. thai_ser, e-seravd, indowavesentiment, seabench, LOTUSDIS, ASR-IndoCSC, ASR-MalCSC), and the audio_path column identifies the exact clip within the benchmark's layout, so records can be re-linked to the source audio.

Dataset structure

Every row is one QA item with the same flat columns for every task:

Column	Meaning
`task`	one of the six benchmark tasks (below) — filter on this to see one task
`qa_format`	`mcq` or `open_ended` (the sample is 50:50 across formats per task)
`id`	the record's benchmark id
`audio_path`	reference to the source audio clip (see the audio note above)
`question`	the question; for MCQ the ten answer options (A–J) are merged into this field
`groundtruth_answer`	the gold answer — for MCQ `"(<letter>) <option text>"`, for open-ended the reference answer (for speech-affective-interpretation this is the affect label itself)
`language`, `dataset`, `emotion`, `category`, `capability`, `ambiguity_type`, `task_variant`	ground-truth label columns (`null` where a task does not use that label)

Note: speech-emotion-recognition and speech-affective-interpretation are built over the same audio corpus, so some id values appear in both tasks (with different questions and ground truths). Always key records on (task, qa_format, id).

Languages

Audio is in three Southeast-Asian languages: Indonesian, Thai and Malay (questions and answers are in English). Dialect tasks additionally cover regional varieties (e.g. Javanese, Sundanese, Batak; Korat, Khummuang, Pattani, Central Thai).

Tasks and sample sizes

Task	MCQ	Open-ended	Total	Label fields fully covered
speech-emotion-recognition	50	51	101	language, dataset, emotion
speech-affective-interpretation	50	51	101	language, dataset, sai_label (= the GT answer)
dialect-identification	50	51	101	language, category
dialectal-speech-comprehension	50	51	101	language
ambiguous-queries	50	51	101	taskvariant (v1/v2/v3), ambiguitytype
long-form-reasoning	18	18	36	language, dataset, capability
Total	268	273	541

Coverage guarantee: for every task and every question format, every value of every label field that exists in the full benchmark population also appears in this sample — no label category is missing. All three ambiguous-queries variants (v1, v2, v3) are represented in both formats.

Usage

python

from datasets import load_dataset

ds = load_dataset("anonymous-submission-1/SEABED", split="sample")

# one task
di = ds.filter(lambda r: r["task"] == "dialect-identification")

# MCQ only
mcq = ds.filter(lambda r: r["qa_format"] == "mcq")

License

The QA annotations in this repository are released under CC BY-NC-ND 4.0 (matching the strictest upstream audio license). Source audio remains under the license of its original corpus and is not distributed here.

Anonymity

This dataset accompanies an anonymous submission. It intentionally contains no author, institution, or model-prediction information.

Intended use

Data inspection and verification for peer review. This is a sample of the benchmark, not the full evaluation set.