Telecom-Paris/iamd_v0

Internet Archive Music Dataset (IAMD v0)

~4.2M thirty-second music segments (34,469 hours) sourced from Creative-Commons audio on the Internet Archive, each paired with machine-generated natural-language captions and the original item metadata.

Loading

A standard Parquet dataset with the audio embedded as an Audio feature — no special loader, no sidecar, no join:

from datasets import load_dataset

ds = load_dataset("Telecom-Paris/iamd_v0", split="train", streaming=True)
sample = next(iter(ds))
sample["audio"]            # {"array": np.ndarray, "sampling_rate": int} — decoded
sample["caption"]   # caption + all metadata are columns on the same row

Audio decoding needs a backend — pip install soundfile (or torchcodec, depending on your datasets version).

Structure

Each shard is a Parquet file; each row is one segment, with the audio embedded as an Audio column ({bytes, path}) next to its caption and metadata columns:

data/iamd-00000.parquet
row: audio = {bytes: <mp3>, path: "iamd_00000261.mp3"}
     key, segment_path, caption_tinymu, artist, license_type, license_url, ...

Repeated item-level fields (artist, license, …) are dictionary-compressed by Parquet rather than duplicated per row. Keys are synthetic (iamd_<8 digits>) because source basenames contain dots, spaces and non-ASCII characters and are not unique across items; the original relative path is kept in the segment_path column.

Metadata fields

Carried through from the Internet Archive item record. Coverage varies widely — most items supply little beyond title and license.

Audio provenance

Segments were cut from source files in several formats and then re-encoded to 320 kbps CBR MP3 for this release:

This is a lossy re-encode. 87% of the sources were already MP3 at a lower bitrate (~224 kbps on average), so those segments have been through two lossy generations — encoding them at 320 kbps makes the files larger without recovering any information. The 12% whose source was WAV/FLAC/AIFF are a single generation from lossless. If you need the highest-fidelity version, work from the source corpus rather than this release.

Licensing

Every item carries its own Creative Commons license, recorded per sample in the license_type and license_url columns.

73% of segments are NonCommercial (NC). The dataset as a whole is therefore not usable for commercial purposes without filtering to the permissive subset:

ds = ds.filter(lambda s: "NC" not in (s["license_type"] or ""))

NoDerivatives (*-ND) items are excluded from this release, because re-encoding produces a derivative work their license does not permit us to redistribute.

Attribution requirements (BY) apply to nearly all segments; use the artist, title and identifier columns to credit sources.

Citation

If you use this dataset, please cite the Internet Archive as the source of the underlying recordings and credit the individual works per their licenses.