espnet/Bagpiper_SFT_Data

Bagpiper SFT Data

Release status: the validated Parquet release is being uploaded. The homepage and metadata may appear before every large shard is committed.

Bagpiper SFT Data is the supervised fine-tuning corpus for Bagpiper, an open-ended audio language model that understands and generates speech, music, environmental sound, and their mixtures through rich textual captions and planning.

The public release has exactly two configurations:

The original source groups are recorded as provenance inside each row; they do not create additional public configurations.

Loading the data

Named Hugging Face configurations are deferred. Load a partition with an explicit Parquet glob; streaming is recommended for this 1.13 TB release:

from datasets import load_dataset

generation = load_dataset(
    "parquet",
    data_files={"train": "hf://datasets/espnet/Bagpiper_SFT_Data/generation/*.parquet"},
    split="train",
    streaming=True,
)
understanding = load_dataset(
    "parquet",
    data_files={"train": "hf://datasets/espnet/Bagpiper_SFT_Data/understanding/*.parquet"},
    split="train",
    streaming=True,
)

example = next(iter(generation))
audio_bytes = example["audio"]["bytes"]

Every row will be self-contained: the ordered text conversation and complete encoded audio bytes are stored in the same Parquet row. No external media download, filesystem path, or table join will be required. Data will be split across many deterministic Parquet shards targeting approximately 256 MiB each.

See Schema for the planned row format and Release status for the publication gates.

Data construction

Bagpiper SFT examples are synthesized from rich-captioned audio. Understanding sequences combine an input audio clip and task request with a rich caption, reasoning trace, and answer. Generation sequences reverse the direction: a request is expanded into planning and a rich caption before the target audio. The project uses large language models for request/reasoning simulation and quality filtering.

Release statistics

Seven understanding rows sharing one header-only, zero-frame FoR_4308 WAV were removed after content audit. Intended reuse of valid audio across multiple rows is retained.

Known limitations

• —Captions, reasoning, and quality judgments are machine-generated and can contain hallucinations or biases.
• —The Bagpiper paper reports audio-fingerprint overlap between its SFT inputs and portions of AudioBench (12.8%) and AIR-Bench (3.2%). Results on those benchmarks should be interpreted with this disclosure.
• —Source audio has heterogeneous provenance and licensing. Configurations or examples without confirmed redistribution rights will be withheld rather than assigned an unsupported blanket license.
• —The corpus may include synthetic or transformed audio and should not be treated as verified human annotation.
• —The current understanding partition follows the checked-in recipe's v1 mixture. It is larger than the approximately 845k understanding sequences reported by the paper; paper-v2 alignment is deferred by owner decision.
• —The corpus is not exhaustively moderated for personal information, unsafe content, offensive language, or copyrighted text embedded in transcripts.

Related resources

• —Paper: Bagpiper: Solving Open-Ended Audio Tasks via Rich Captions
• —Project: bagpiper-cmu.github.io
• —Base model: espnet/bagpiper
• —Pretraining data: espnet/Bagpiper_PreTrain_Data
• —ESPnet: espnet/espnet

Citation

Please cite the accepted Bagpiper paper when using this dataset:

@inproceedings{anonymous2026bagpiper,
  title={Bagpiper: Solving Open-Ended Audio Tasks via Rich Captions},
  author={Jinchuan Tian and Haoran Wang and Bo-Hao Su and Chien-yu Huang and
          Qingzheng Wang and Jiatong Shi and William Chen and Xun Gong and
          Siddhant Arora and Chin-Jou Li and Masao Someki and Takashi Maekaku and
          Keita Goto and Yusuke Shinohara and Jin Sakuma and
          Chao-Han Huck Yang and Shinji Watanabe},
  booktitle={Third Conference on Language Modeling},
  year={2026},
  url={https://openreview.net/forum?id=FuHs64E3X6}
}

Contact and takedown

Please use the repository community tab for data issues, provenance corrections, or takedown requests. Include the configuration and example_id; do not repost sensitive media in the report.