nz00shuuuu/urbansound-haystack

Urban-Sound-Haystack

A long-context urban-audio QA benchmark across 10 task types and 4 context lengths (100 s, 15 min, 30 min, 1 h). Each soundscape is synthesised by Scaper from UrbanSound8K foreground events over TUT acoustic-scene backgrounds, sampled at 16 kHz mono PCM_32. Two of the ten tasks (`anomaly_detection, anomaly_localization) draw from a parallel pool where every soundscape contains exactly one out-of-vocabulary event from ESC-50 (glass_breaking or crying_baby`).

This is the audio variant of the TSLM-Arena haystack benchmark family, joining ts_haystack (accelerometer), ltaf_haystack (ECG), and sleep_psg_haystack (13-channel polysomnography).

Repo layout

audio/
  output/<ctx>/<split>/<idx>/{soundscape.wav, soundscape.jams, soundscape.txt}
  output_anomalies/<ctx>/<split>/<idx>/{...}
existence/{train,validation,test}-*.parquet
localization/...
...
anomaly_localization/...
README.md

QA parquet shards reference audio by `audio_path + audio_subroot ("normal" or "anomaly"`). Audio is uploaded once (each WAV serves ~40 QA rows across tasks/QA-per-soundscape multiplicity), so total repo size is ~69 GB rather than the ~1.6 TB it would be if audio bytes were embedded inline.

Quick start

from datasets import load_dataset
from huggingface_hub import hf_hub_download
import soundfile as sf

ds = load_dataset("nicozumarraga/urbansound-haystack", "existence", split="train")
sample = ds[0]

# Resolve the audio path against the right pool subroot
subroot = "output_anomalies" if sample["audio_subroot"] == "anomaly" else "output"
wav_path = hf_hub_download(
    "nicozumarraga/urbansound-haystack",
    f"audio/{subroot}/{sample['audio_path']}",
    repo_type="dataset",
)
audio, sr = sf.read(wav_path)  # bytes-identical to what the benchmark trained on

print(sample["question"])
print("Answer:", sample["answer"])
print("Audio:", audio.shape, sr)

A reusable helper:

def load_sample(repo_id, sample):
    """Return the (audio, sampling_rate) for an Audio-Haystack QA sample."""
    sub = "output_anomalies" if sample["audio_subroot"] == "anomaly" else "output"
    wav_path = hf_hub_download(
        repo_id, f"audio/{sub}/{sample['audio_path']}", repo_type="dataset"
    )
    return sf.read(wav_path)

Configs and splits

Available configs (10): `existence, localization, counting, ordering, antecedent, comparison, multi_hop, state_query, anomaly_detection, anomaly_localization. Each has three splits with all 4 context lengths interleaved (filterable via the context_length_seconds` column).

Per-split row counts (identical for every task):

Total: 34,400 QA pairs across 10 tasks × 4 contexts × 3 splits.

Task contract

The 8 normal tasks draw from soundscapes whose foreground events come exclusively from the 9 UrbanSound8K classes (air_conditioner, car_horn, children_playing, dog_bark, drilling, engine_idling, jackhammer, siren, street_music). The two anomaly tasks additionally draw from a parallel pool where every soundscape contains exactly one event from a distinct OOV vocabulary (glass_breaking, crying_baby); the model is told the urban-9 vocabulary in the prompt and is expected to detect / localise events outside it.

Data isolation

Source-clip stratification — no audio file appears in more than one split:

• —UrbanSound8K foreground: folds 1–8 → train, 9 → val, 10 → test.
• —ESC-50 anomaly: folds 1–3 → train, 4 → val, 5 → test (same two classes across splits — the per-class budget is too small to hold any out, ~24 train clips per class).
• —TUT acoustic-scene backgrounds are shared across splits — they don't drive answers, only set ambience.

The audit script in the source repo (`scripts/data/audio_haystack/check_no_leakage.py`) walks both pools and asserts pairwise-disjoint source-clip sets per split.

License

CC BY-NC 3.0. Derived from:

• —UrbanSound8K (Salamon, Jacoby, Bello, MM 2014) — CC BY-NC 3.0.
• —ESC-50 (Piczak, MM 2015) — CC BY-NC 3.0.
• —TUT Acoustic Scenes 2016 (Mesaros et al., EUSIPCO 2016) — CC BY-NC-SA 4.0.

This dataset inherits the most restrictive (non-commercial) terms of its sources. Use is permitted for academic and research purposes only.

Citation

If you use this dataset, please cite the source corpora and the TSLM-Arena benchmark family.