laion/more-synthetic-vocalbursts-raw

More Synthetic Vocal Bursts (Raw)

Synthetic vocal burst audio samples generated from a taxonomy of 202 vocal burst types across multiple text-to-audio and TTS models. Each sample is a short (3–10 second) non-speech vocalization — laughs, cries, gasps, sighs, growls, etc. — generated from text prompts describing the burst type, gender, and age group.

Models Used

All outputs also have RE-USE enhanced versions processed through NVIDIA RE-USE (9.6M param speech enhancement model).

Dataset Structure

All files are in WebDataset .tar format. Each shard contains pairs of .wav (audio) and .json (metadata) files.

dramabox/              # DramaBox TTS outputs (2000 samples, 4 shards)
  shard-0000.tar
  shard-0001.tar
  shard-0002.tar
  shard-0003.tar

sa3/                   # Stable Audio 3 Small SFX outputs (2000 samples, 4 shards)
  shard-0000.tar
  ...

moss/                  # MOSS SoundEffect v2.0 outputs (781 samples, 2 shards)
  shard-0000.tar
  shard-0001.tar

dramabox-reuse/        # DramaBox + RE-USE enhanced (2000 samples, 4 shards)
sa3-reuse/             # SA3 + RE-USE enhanced (2000 samples, 4 shards)
moss-reuse/            # MOSS + RE-USE enhanced (781 samples, 2 shards)

nsfw/                  # NSFW comparison samples (78 per model variant)
  nsfw-original.tar    # DramaBox original
  nsfw-sulfur.tar      # DramaBox Sulfur DiT variant
  nsfw-sa3.tar         # SA3
  nsfw-moss.tar        # MOSS
  nsfw-original_reuse.tar   # + RE-USE enhanced versions
  nsfw-sulfur_reuse.tar
  nsfw-sa3_reuse.tar
  nsfw-moss_reuse.tar

Metadata Format

Each .json sidecar contains:

{
  "id": 42,
  "prompt": "A teenage girl performing belly_laugh, A deep, uncontrollable laugh...",
  "duration_s": 5.2,
  "gender": "female",
  "age_group": "teenage_girl",
  "vocal_burst_key": "belly_laugh",
  "vocal_burst_description": "A deep, uncontrollable laugh that involves the whole body..."
}

Taxonomy

The vocal burst taxonomy covers 202 entries organized by category:

• —Laughter (8 types): belly laugh, chuckle, giggle, cackle, snicker, etc.
• —Crying & Distress (10 types): sobbing, whimpering, wailing, etc.
• —Breathing & Sighs (11 types): heavy panting, exasperated sigh, etc.
• —Surprise & Shock (6 types): startled yelp, dramatic gasp, etc.
• —Disgust & Disapproval (8 types): retching, scoff, tsk, etc.
• —Pain & Discomfort (8 types): sharp yelp, prolonged groan, etc.
• —Effort & Exertion (8 types): heavy lifting grunt, battle cry, etc.
• —Communication Signals (11 types): shush, psst, wolf whistle, etc.
• —Eating & Drinking (6 types): slurping, lip smacking, etc.
• —Sleep & Unconscious (5 types): snoring, sleep talking, etc.
• —Animal Imitations (6 types): growling, purring, hissing, etc.
• —Musical & Rhythmic (7 types): humming, beatboxing, etc.
• —Nervous & Anxious (7 types): nervous laughter, teeth chattering, etc.
• —Age-Specific (6 types): baby cooing, elderly wheeze, etc.
• —Bodily Functions (9 types): hiccup, sneeze, burp, etc.
• —Whistling (6 types): casual whistle, wolf whistle, etc.
• —Oral/Mouth Sounds (8 types): tongue click, teeth sucking, etc.
• —Throat Sounds (7 types): throat clearing, gargling, etc.
• —Nasal Sounds (5 types): sniffling, snorting, etc.
• —Vocal Tics & Reflexes (6 types): hiccup, involuntary yelp, etc.
• —Temperature & Environment (4 types): shivering chatter, heat exhaustion panting, etc.
• —Expressive Interjections (7 types): eureka exclamation, frustrated argh, etc.
• —NSFW (22 types): intimate vocalizations for adult content

The SFW version (180 entries, NSFW removed) is available in the Voice-Acting-Pipeline repo.

Demographics

Samples span 10 age/gender groups:

• —Female: toddler girl, pre-puberty girl, teenage girl, young adult woman, middle-aged woman
• —Male: toddler boy, pre-puberty boy, teenage boy, young adult man, middle-aged man

Usage

import webdataset as wds
import soundfile as sf
import io

dataset = wds.WebDataset("dramabox/shard-{0000..0003}.tar")
for sample in dataset:
    audio_bytes = sample["wav"]
    metadata = json.loads(sample["json"])
    audio, sr = sf.read(io.BytesIO(audio_bytes))
    print(f"{metadata['vocal_burst_key']} - {metadata['gender']} - {sr}Hz - {len(audio)/sr:.1f}s")

Related Resources

• —Taxonomy JSON (full)
• —Taxonomy JSON (SFW)
• —NVIDIA RE-USE — speech enhancement model used for postprocessing
• —MOSS SoundEffect v2.0
• —Stable Audio 3 Small SFX

License

CC-BY-4.0