laion/vocal-bursts-gemini-segments

burstgeminisegments (Dataset B)

5,161 vocal-burst segments cut out of 3,304 real speech utterances, one per event that Gemini 3.8 Flash asserted. Each file is a single burst — a laugh, a sigh, a gasp — and nothing else. Median length 0.76 s; 1.39 hours in total.

Built to retrain a burst classifier. The detector this project shipped emitted Shriek zero times over a 60-clip audit, used 8 of its 83 labels, and put the requested burst in its top-3 on 3 of 60 clips. Every burst adapter here was trained on that detector's labels; this is the material for replacing them.

🎧 [Listen to 60 samples](https://huggingface.co/spaces/laion/vocal-burst-segments-samples) — three from each of the twenty most common labels.

What it is measured to do

A small MLP (768-d audio embedding → 256 → 11) trained on the 2,451 non-overlapping segments in the eleven classes with at least 100 examples, five seeds, speaker-disjoint splits (whole speakers go to train or test, never both):

Per class, best to worst: Chuckle 89 %, Exhausted Groan 76 %, Humming 72 %, Scream 71 %, Panting 67 %, Sharp Inhale 63 %, Relief Sigh 53 %, Exasperated Sigh 41 %, Yawn 33 %, Deep Breath 32 %, Heavy Breathing 20 %.

Scream matters most: it is exactly what the old detector could never hear.

How much to believe that number. The test labels are Gemini's, so the new model is judged by the standard it was trained on and the old detector by one it never agreed with — part of the 17-point gap is that home advantage. What the number does establish is that a burst class is learnable from these segments at all.

Statistics

Duration

Language and verification tier

Source corpora

Podcast-derived rows are excluded from this release. Every segment carries its source, so any subset can be reconstructed.

The 25 most common labels

<details><summary>All 49 corpus classes the parents came from</summary>

</details>

Layout

WebDataset, data/vbg-seg-*.tar, 6 shards. Per sample:

• —<key>_eNN.ogg — the segment, 48 kHz mono OGG/Vorbis, cut from the float waveform before encoding, never re-cut from Dataset A's encoded audio, so no sample is encoded twice.
• —<key>_eNN.txt — the top label, following the .txt convention of laion/vocal-bursts-clean, so an existing loader reads this unchanged.
• —<key>_eNN.json — all 1–3 labels, confidence, description, parent_key, the corpus class of the parent, source, nucleus_start_s / nucleus_end_s, speech_overlap_frac, and span_source.

parent_key points into Dataset A, `laion/vocal-bursts-gemini-utterances`, which holds the whole utterances these were cut from.

import webdataset as wds
ds = wds.WebDataset("data/vbg-seg-{00000..00005}.tar").decode()
for r in ds:
    audio, label, meta = r["ogg"], r["txt"], r["json"]

Licence

CC BY 4.0.

Read this before using the boundaries

The segments were never annotated as segments. Gemini heard the whole utterance and drew spans inside it; these files are cuts of those spans. That ordering is deliberate and it is the only defensible one — on 20 excerpts cut from audio this same model had left unannotated, it said "no burst" once and returned Scream/Shriek eleven times. Its labels are context-dependent, so a segment-level annotation pass would have manufactured agreement rather than measured it.

Span width — the weak part of this release

§55 measured this model's spans at 38.2 % clip coverage for a 1.27x lift over chance at containing the clip's loudest moment, against our detector's 6.5 % for 3.57x. It is *better at what and worse at where***. Measured again on this release:

How this was handled: by not hiding it. Nothing is silently narrowed. Dataset B cuts the span Gemini actually asserted, plus 50 ms. An energy-tightened window is computed and shipped as metadata (nucleus_start_s, nucleus_end_s) so a consumer can narrow deliberately and reversibly, and every segment carries speech_overlap_frac — the fraction of the span covered by an aligned word — because a wide span sitting over speech is the specific way this data is wrong. Median speech overlap is 0.0, p90 0.595.

Use Dataset B for the label. Do not use it to train a locator.

Source datasets

Every row is attributed from uid + src + voice_key; 0 rows are unattributable and none is guessed.

emolia, kartoffelphon and vocal_bursts_clean rows are recordings of people (1418 rows). voice_profile/* rows are LAION voice profiles: scripted text spoken by a synthetic voice built from a real speaker, where the burst was scripted as a cue and then confirmed by the detector in the produced audio. They are labelled human_recording: false on every sample so a consumer can split on it.

Nothing here is spliced, concatenated or voice-converted. The one manufactured row that survived the filter (vb:improved_synthetic) was excluded by name.

What was excluded, and why

• —Podcast rows are not published. 2094 rows matching (EN|DE|ZH)_<11-char youtube id>_W###### were dropped before selection. Their transcripts are not open, and laion/tts-realspeech-sft-en-de — the public release these rows come from — ships emolia + kartoffelphon + mls and deliberately omits them.
• —Mediathek: the restriction does not bite here, and this is worth stating precisely. The brief flagged ~937 thewildaudio rows. Those rows are src == 0 voice profiles, not thewildaudio recordings and not thewildaudio transcripts: the text is scripted and the voice is a synthetic one built from a thewildaudio speaker. laion/laion-voice-profiles-sft (public, CC-BY-4.0) already ships exactly these — its part-00377 is 100 % the_wild_audio_# voices. They ship here too. This corpus contains no real the_wild_audio recordings or transcripts, so there was nothing of the restricted kind to exclude.
• —MLS: no MLS row survived the strict burst filter, so the source is absent rather than excluded.
• —Attribution is an allowlist: an unrecognised uid is not publishable.

Two verification levels, never mixed unlabelled

Every row carries verification, one of:

• —`strict` (2727 rows) — the gate that built $SC/buckets_strict: every target span carries a same-class detection within 1.5 s and the weakest clears theta = 0.174.
• —`family_relaxed` (871 rows) — identical in every other respect (same theta, same tolerance, same geometry gate, no new compute, scored from the same stored detector output) except that a detection from the same burst family counts. This is the metric the owner accepted on 2026-09-02.

Family-relaxed rows were admitted only for classes that could not reach their ladder tier on strict rows alone. The strict filter was never loosened to hit a target: a class short of 100 is reported short. The per-class table below gives both counts for every class, so the whole relaxed set drops out with one filter.

What Gemini said, against the label the corpus already carried

Pooled over 3598 rows, 5161 events (1.434 per row):

70 distinct taxonomy labels were used in some slot and 59 as a top label, against the detector's 8. 0 labels outside the taxonomy were returned.

These labels are a second opinion, not verified ground truth

Every agreement number in this README is between two models: this project's burst detector, which produced the class each row was already filed under, and gemini-3.8-flash, which was asked blind. Both are downstream of models trained on expressive speech. A shared prior would inflate every one of these figures and nothing measured here can see it. What this release adds is a measurably broader second opinion — 70 of the 83 taxonomy classes used against the detector's 8 — not a verification. Whether a given label is correct is a listening question, and $SC/pages/gemini_traindata/ exists to answer it.

How the annotation was done

• —Blind. The request never named the class the row was filed under, never said a burst had been found, and never carried the transcript. §55 established the reason: told to expect a shriek, this model finds one. The corpus label is joined back on only in scoring, after every response is on disk.
• —Whole utterances only. On 20 excerpts cut out of audio the same model had itself left unannotated, it said "no burst" once and returned Scream/Shriek eleven times. Its labels are context-dependent and not stable under re-segmentation, so nothing shorter than a whole utterance was ever sent. Dataset B's segments are cut out of those whole-utterance spans afterwards; they were never annotated on their own.
• —Closed 83-class taxonomy in the system instruction, structured JSON response schema, temperature 0, 1-3 labels per event most-likely-first, and an explicit licence to answer "nothing here" (8.2 % of rows did).
• —3598 calls, 3,977,142 prompt + 577,683 output + 2,369,979 thinking tokens.

Audio format: OGG, and why not MP3

laion/vocal-bursts-clean is 48 kHz mono MP3 and matching it was the plan. MP3 encoding works here — lameenc 1.8.4 round-trips at corr 0.9998 — so this is not a missing-encoder story. It is a measurement: on all six clips tested, the decoded MP3 leads the input by exactly 1,105 samples = 23.0 ms at 48 kHz and runs 32–40 ms long at the tail. That is LAME's encoder delay; a player cancels it using the Xing/LAME info frame, and lameenc exposes no API to write one. Read with soundfile or librosa — which is how this data will be read — every span in this release would sit 23 ms off its own audio, and spans are the entire content.

OGG/Vorbis through libsndfile is sample-exact on the same clips (0 samples added, 0 lag) and about 40 % smaller, so the release ships OGG at the same 48 kHz mono. gt_audio.to_mp3 is kept and tested; MP3_DELAY_SAMPLES_48K = 1105 records the number an MP3 release would have to compensate.

Licence and provenance

As Dataset A. Not uploaded — staged only.

Correction (2026-09-04)

An earlier version of this card said the shipped vocal-burst-detector-v2 "used 8 of its 83 labels" and "emitted Shriek zero times". That was measured on a 60-clip audit and does not generalise. Run over both full corpora (72,500 synthetic clips and 3,598 real utterances), the same detector emits 41 distinct labels on the synthetic half and 36 on the real half, and it does emit `Shriek` — on 50 synthetic clips and 49 real ones.

What the 60-clip audit actually saw is how concentrated the distribution is: Contented Sigh alone accounts for 16,694 of 72,500 synthetic clips, Surprised Gasp 7,451, Wistful Sigh 5,437. The correct statement is that the detector's effective vocabulary is small, not that its emitted vocabulary is 8. The conclusions drawn elsewhere in this card — that the detector is the bottleneck, and that its failure is granularity rather than deafness — are unaffected and are independently supported by the cross-source results in §62.

<!-- vocal-burst-groups-23:start -->

Class groups

vocal_burst_groups.json and GROUPS.md carry a 23-group scheme over 117 burst label strings, grouping names that denote the same or a very similar sound (snicker/chuckle, shriek/scream, cough/coughing). Scoring the same predictions at group level raises the mean generation hit rate from 0.302 to 0.537; a random grouping with identical group sizes reaches 0.355, so +0.182 of it is the grouping being right and the rest is arithmetic. Groups were checked with directed lift rather than raw confusion, because two labels account for 29 % of all annotator top-1 calls whatever was requested and merging on raw confusion books a generation failure as a hit.

For training the classifier, keep the fine classes: collapsing them raises raw accuracy only because chance rises with it. Group at evaluation time — that can be done at any point, the reverse cannot.

<!-- vocal-burst-groups-23:end -->