AudioMarathon/AudioMarathon

AudioMarathon

AudioMarathon is a long-context audio benchmark for evaluating multimodal LLMs on speech, music, environmental audio, and meetings. The release package in this directory is organized around 11 benchmark tasks spanning meeting summarization, automatic speech recognition, reading comprehension, authenticity detection, music genre classification, acoustic scene classification, emotion recognition, spoken named entity reasoning, sound event detection, speaker gender classification, and speaker age classification.

This README is written for paper submission and dataset hosting. It focuses on the locally verifiable release contents, the reviewer-facing sample subset, and the redistribution constraints that matter at submission time.

Dataset page:

• —AudioMarathon on Hugging Face

Release scope

The benchmark release relevant to this submission consists of these directories:

• —AliMeeting/
• —librispeech-long/
• —race_audio/
• —HAD/
• —GTZAN/
• —TAU/
• —VESUS/
• —SLUE/
• —DESED/
• —VoxCeleb/

Auxiliary folders such as .cache/ are not part of the 11-task benchmark described in this README.

Locally verified release inventory

The counts below were generated from the files currently present in this directory by scripts/prepare_submission_assets.py. They describe the release package as stored locally, not a paper-draft summary copied by hand.

Locally verified totals across the benchmark directories above:

• —6204 released audio files
• —68.075 GB of audio payload

The full machine-readable inventory is stored in submission_assets/release_inventory.json. The task-level catalog is stored in submission_assets/task_catalog.json.

Reviewer sample subset

Because the full benchmark is larger than the small-upload threshold, this repository includes a deterministic reviewer-facing sample subset in review_sample.

Current sample statistics:

• —60 audio files
• —2.34 GB of audio payload
• —sidecar text files for sampled AliMeeting, LibriSpeech, and RACE examples
• —machine-readable manifests in sample_manifest.json and sample_manifest.csv

Suggested reviewer sample URL:

• —review_sample

The sample was created with fixed, reproducible rules:

• —AliMeeting: the first, median, and last far-field meeting mixture after sorting released meeting audio paths; the task is to generate meeting minutes summarizing key conclusions, decisions, and follow-up actions
• —LibriSpeech-long: the lexicographically first FLAC from each released split
• —RACE-audio: the first, median, and last article audio after sorting unique article paths
• —HAD: the first existing sample for each authenticity label
• —GTZAN: the first existing sample for each genre label
• —TAU: the first existing sample for each scene label
• —VESUS: the first existing sample for each emotion label
• —SLUE: the first existing sample for each split-and-construction combination
• —DESED: the lexicographically first released sample in each event folder
• —VoxCeleb gender: the first existing sample for each gender label
• —VoxCeleb age: the first existing sample for each age group

The reviewer subset description is stored in review_sample/README.md.

Directory map

AudioMarathon/
  README.md
  scripts/
    prepare_submission_assets.py
  submission_assets/
    AudioMarathon.croissant.json
    release_inventory.json
    review_sample_summary.json
    task_catalog.json
  review_sample/
    README.md
    sample_manifest.json
    sample_manifest.csv
    files/
  AliMeeting/
  librispeech-long/
  race_audio/
  HAD/
  GTZAN/
  TAU/
  VESUS/
  SLUE/
  DESED/
  VoxCeleb/

Licensing and redistribution notes

This benchmark aggregates data derived from multiple upstream resources. Redistribution must respect the original terms of each component.

• —AliMeeting: verify the redistribution terms for the released package and meeting transcripts before public hosting
• —LibriSpeech-long: CC BY 4.0
• —RACE-audio: derived from RACE; verify the exact redistribution terms used for the hosted package
• —HAD: CC BY 4.0
• —GTZAN: research-use restrictions apply
• —TAU: CC BY 4.0
• —VESUS: academic-use restrictions apply
• —SLUE: verify redistribution terms for the released package
• —DESED: per-file attribution must be preserved; see DESED/DESED_dataset/license_public_eval.tsv
• —VoxCeleb: verify redistribution terms for the hosted package and derived annotations

Before public hosting, ensure that the final destination and README preserve any required attribution files and restriction notices.

AliMeeting task note

The AliMeeting component in this release is treated as a meeting summarization task:

• —the evaluation input is the far-field meeting mixture audio listed in AliMeeting/Test_Ali.csv
• —the instruction is to generate a factual meeting summary or meeting minutes
• —the desired output focuses on key conclusions, decisions, and follow-up action items
• —the near-field tracks and TextGrid files act as reference material and auxiliary supervision, rather than additional meeting-summary evaluation items

Submission files

The main files intended for submission and hosting are:

• —README.md
• —submission_assets/AudioMarathon.croissant.json
• —submission_assets/release_inventory.json
• —submission_assets/task_catalog.json
• —review_sample/

Reproducibility

Regenerate the submission artifacts from the repository root with:

python scripts/prepare_submission_assets.py