laion/voice-acting-burst-prompts

Scene prompts for 49 vocal-burst classes

24,500 screenplay-style scene prompts, 500 per class across 49 vocal-burst classes, written to make a text-to-speech model produce a specific non-verbal vocalisation inside ordinary speech rather than as an isolated sound effect.

This dataset contains prompts only. No audio is included.

Why the prompts look the way they do

These prompts exist to fix a measured defect. An earlier run generated 60 clips from twenty hand-written scream and shriek scenes and measured their dynamic contrast — the loudest 300 ms window against the clip's own median window, in dB:

The model performed the entire scene at burst intensity — the dialogue around the burst was itself shouted. Audio like that teaches a burst model "this utterance is loud", not "there is a burst here", which is worse than useless for training a burst detector or adapter. The cause was the scene writing, not the model: none of those twenty scenes had a calm opening.

So every prompt in this corpus opens calm:

1. 1.the direction before the first spoken line explicitly instructs ordinary, level, conversational delivery, and contains no word of panic, shouting or high emotion;
2. 2.the first spoken line is a full, mundane sentence about an everyday subject — a bus timetable, the price of coffee, whether the washing will dry — long enough to take four or five seconds to say, with no exclamation mark, no shouting capitals and no burst in it;
3. 3.the burst is a departure from that baseline;
4. 4.in 62 % of scenes the speaker then settles back into exactly the opening delivery.

The contrast actually achieved

Same statistic, same code, on audio generated from these prompts:

The loud band is the one directly comparable with the 6.1 dB figure, and it roughly doubles it. Note, though, that all three bands moved together (quiet 11.5, mid 11.2, loud 12.0). We had expected the quiet classes to stay low, and they did not. Peak-over-median measures departure from the lead-in, not absolute loudness, and a soft hum after a level spoken line is a large relative excursion too. Read this statistic as evidence that the calm lead-in is present and working -- not as a loudness ranking, and not as a way to select loud clips.

How the prompts were written

Written by `google/gemma-4-31B-it` (bf16, temperature 1.0, top-p 0.95, top-k 64), one scene per completion, from a fixed few-shot instruction with three hand-written exemplars spanning a loud, a moderate and a quiet class.

Diversity is structural, not sampled. Each of the 500 scenes in a class is pinned to a distinct (place, topic, trigger) triple drawn from banks of 60 places, 50 everyday conversation topics and ≥ 15 per-class triggers, plus a speaker (2 genders × 3 age bands × 12 speaking registers), the position of the burst in the scene, and whether the scene returns to calm. Sampling is deterministic per class (sha256 of the class name), so the corpus reproduces exactly.

Speaker demographics — the balance actually achieved

The six (gender × age band) cells are assigned by scene index, not sampled: cell i % 6 walks a fixed order, so every class gets the same split and any prefix of a class is balanced too. Drawing the cell at random 500 times would give cells of roughly 68–99 and a different imbalance in each of the 49 classes.

Counted off the corpus as staged, not as intended:

Across all 49 classes the smallest cell holds 83 prompts and the largest 84.

The age band and sex are written into the speaker description inside the prompt itself, not only into the metadata: the voice-acting model reads the voice out of the paragraph, so a gender column that never reached the prose would balance the spreadsheet and nothing else. The gate rejects any generation whose opening direction does not carry both (demo_gender_missing, demo_age_missing).

Children and teenagers appear nowhere in this corpus. The three bands are the three the brief names, all adult.

Every burst is spelled phonetically inside the quotes and the sound is never named there: the voice-acting model speaks the words Sigh, Gasp, Cough, Ahem, Pfft aloud as words when they stand inside quotes. The sound is named in the direction outside the quotes, where the model interprets it but does not say it.

The quality gate

Every generation was checked before any GPU time was spent on audio. 24,500 accepted, 219 rejected (0.9 % of all generations). Rejected specs were regenerated with a fresh sampling seed, up to six attempts.

Accepted on attempt: 0 → 24,307, 1 → 171, 2 → 19, 3 → 2, 4 → 1 (attempt 0 is the first try).

data/rejections.parquet ships every rejected generation with its reason and text, so the gate can be audited or re-tuned rather than trusted.

Intended generation settings

The prompts were written for, and the reference audio generated with, the voice-acting model at:

voice_ref=None, so the voice comes from the scene's own speaker description and ref_duration is inert; it is recorded anyway so the configuration is never silently different from the stated one. Seeds 1234 / 5678 / 9012. Watermarking off.

Fields

Classes

Two classes present in the source list were dropped, because the list's own exclusion rules remove them:

• —`wolf_whistle` — a lip whistle, not phonation -- the list's own exclusion set removes whistles
• —`tsk` — a dental/alveolar tongue click -- the list's own exclusion set removes tongue clicks

Licensing

The prompts are our own text: they were written by a language model from instructions and word banks authored for this corpus. They are released under CC BY 4.0 and may be used with any speech model, commercially or otherwise. The reference audio generated from them is published at `laion/voice-acting-burst-audio` under the same licence.

Reproducing

$SC/code/vb_promptgen/
  vbclasses.py   49 classes: description, phonetic spellings, recogniser regex,
                 intensity, >=15 triggers each; plus the two dropped classes and why
  specs.py       the combinatorial sampler, the instruction, the three exemplars
  pg_gate.py     the quality gate
  pg_gen.py      batched generation with gemma-4-31B-it, resumable per prompt
  pg_audio.py    generated audio into tar shards
  pg_contrast.py the peak-over-median dB measurement
  pg_publish.py  this staging directory

The reference audio for these prompts was generated with a voice-acting AI model.