Datasetpublic

TTS-AGI/vocal-bursts-taxonomy-DACVAE

Vocal Bursts Taxonomy — DACVAE + MaestroClap Embeddings & Scores Processed version of with DACVAE latents, MaestroClap embeddings, derived attribute/quality/speaker scores, and Gemini-verified labels. Overview Metric Value Total samples 16,175 Categories 82 Genders male, female Female samples 8,097 Male samples 8,078 Gemini Label Verification Every sample was sent to Gemini 3.1 Flash Lite for two independent tasks: Match… See the full description on the dataset page: https://huggingface.co/datasets/TTS-AGI/vocal-bursts-taxonomy-DACVAE.

sourceHugging Facecc-by-4.0updated 6mo agoView on Hugging Face

0likes103downloads

Dataset Card

Vocal Bursts Taxonomy — DACVAE + MaestroClap Embeddings & Scores

Processed version of [](https://huggingface.co/datasets/krishnakalyan3/vocal_bursts_taxonomy_100_clean_wds) with DACVAE latents, MaestroClap embeddings, derived attribute/quality/speaker scores, and Gemini-verified labels.

Overview

Metric	Value
Total samples	16,175
Categories	82
Genders	male, female
Female samples	8,097
Male samples	8,078

Gemini Label Verification

Every sample was sent to Gemini 3.1 Flash Lite for two independent tasks:

1.Match scoring: Audio + original prompt label sent together. Model rates how well the label matches: 0 (not at all), 1 (slightly), 2 (very well).
2.Dense captioning: Audio sent alone (without the label). Model produces a concise technical description of the vocal burst.

Match Score	Meaning	Count	Percentage
2	Very well — label accurately describes audio	15,082	93.2%
1	Slightly — somewhat related but not accurate	160	1.0%
0	Not at all — label is completely wrong	933	5.8%

15,242 / 16,175 samples (94.2%) verified as valid (match score 1 or 2).

Total verification cost: /bin/bash.97 (32,350 API calls across 100 parallel threads in ~21 minutes).

Each tar file corresponds to one vocal burst category and contains:

Per-Sample JSON Fields

Original metadata

Field	Type	Description
	string	Original filename from source dataset
	float	Audio duration in seconds
	string	or
	string	Vocal burst category name + description
	int	Category ID (1–97)
	int	Unique sample identifier
	int	44100 Hz

DACVAE latent info

Field	Type	Description
	int	Number of DACVAE latent frames (T)
	int	Latent dimension (128)

MaestroClap embeddings & probes

Field	Type	Description
	float[512]	MaestroClap audio-audio embedding (L2 normalized)
	float[128]	Predicted WaveELM timbre/speaker embedding
	dict(53 floats)	53 emotion/speaker attribute scores (see list below)
	float	Duration in seconds predicted from CLAP embedding
	float	Consonant/Phoneme Score
	float	Background noise quality
	float	Content enjoyment rating
	float	Overall audio quality
	float	Speech quality rating

Gemini verification

Field	Type	Description
	int	0=not at all, 1=slightly, 2=very well (label vs. audio match)
	string	Brief explanation of the match rating
	string	Dense technical vocal burst caption generated without seeing the label

Models Used

Model	Architecture	Output
DACVAE	Facebook DAC-VAE encoder	(T, 128) latent at 25fps
MaestroClap	ViT-B/12 on DACVAE latents	512-dim normalized embedding
Attribute Probe	MLP 512→704→704→53+1	53 emotion/speaker attributes + duration
Speaker Probe	MLP 512→704→704→128	128-dim timbre embedding
Quality Experts (×5)	MLP 512→64→64→1	5 independent quality scores
Gemini 3.1 Flash Lite	Google LLM with audio input	Match score (0–2) + dense caption

53 Attribute Dimensions

Affection, Age, Amusement, Anger, Arousal, Astonishment/Surprise, Authenticity, Awe, Bitterness, Concentration, Confident vs. Hesitant, Confusion, Contemplation, Contempt, Contentment, Disappointment, Disgust, Distress, Doubt, Elation, Embarrassment, Emotional Numbness, Fatigue/Exhaustion, Fear, Gender, Helplessness, High-Pitched vs. Low-Pitched, Hope/Enthusiasm/Optimism, Impatience and Irritability, Infatuation, Interest, Intoxication/Altered States, Jealousy & Envy, Longing, Malevolence/Malice, Monotone vs. Expressive, Pain, Pleasure/Ecstasy, Pride, Relief, Sadness, Serious vs. Humorous, Sexual Lust, Shame, Soft vs. Harsh, Sourness, Submissive vs. Dominant, Teasing, Thankfulness/Gratitude, Triumph, Valence, Vulnerable vs. Emotionally Detached, Warm vs. Cold

Category List

Category	Description	F Total	F Match=2	M Total	M Match=2	Total	Valid
Affirmative Grunt	Affirmative Grunt short sound indicating agreement	100	100	100	94	200	194
Ahem	Ahem clearing throat to gain attention or show discomfort	100	73	99	92	199	166
Blowing	Blowing a Kiss brief airy sound of sending affection	99	73	98	51	197	124
Breathy Giggle	Breathy Giggle soft airy laugh	99	94	100	70	199	178
Cackle	Cackle harsh sharp laugh sometimes unsettling	97	74	99	99	196	195
Chewing Noises	Chewing Noises crunching or moist sounds from the mouth	95	95	98	98	193	193
Childlike Giggle	Childlike Giggle light playful laugh	100	100	98	98	198	198
Chuckle	Chuckle soft suppressed laugh showing mild amusement	100	77	100	80	200	176
Clears Throat	Clears Throat brief intentional effort to remove phlegm or gain attention	99	98	99	98	198	196
Click One s Tongue	Click One’s Tongue short ‘tch’ sound by flicking tongue off palate	100	100	98	98	198	198
Clicks Tongue	Clicks Tongue distinct click by tapping tongue against the palate	100	100	99	99	199	199
Contented Sigh	Contented Sigh deep exhale expressing satisfaction	97	79	99	69	196	148
Convulsive Sob	Convulsive Sob intense crying with heaving breaths	100	99	99	96	199	196
Cough	Cough sharp expulsion of air from lungs	98	98	94	94	192	192
Coughing	Coughing series of forceful expulsions of air from lungs	99	99	100	100	199	199
Deep Breath	Deep Breath large inhalation to calm or refocus	98	92	99	93	197	185
Deep Breathing	Deep Breathing slow deliberate inhalations and exhalations for relaxation	99	99	99	98	198	197
Displeased Grunt	Displeased Grunt harsh vocalization of dissatisfaction	99	96	99	98	198	197
Drinking Noises	Drinking Noises audible liquid intake with gulps or slurps	100	99	99	99	199	198
Effort Grunt	Effort Grunt short forceful sound during physical strain	100	99	100	99	200	198
Exasperated Sigh	Exasperated Sigh audible breath of frustration	99	95	100	99	199	194
Exhausted Groan	Exhausted Groan weary sound indicating fatigue	97	97	99	99	196	196
Fast Breathing	Fast Breathing rapid inhalation and exhalation from excitement or stress	99	91	100	96	199	198
Fearful Gasp	Fearful Gasp sudden inhale showing shock or anxiety	98	85	99	85	197	172
Finger Snaps	Finger Snaps brief sharp click made by friction of thumb and finger	100	100	98	98	198	198
Frustrated Groan	Frustrated Groan deep sound of annoyance	100	99	98	97	198	198
Growl	Growl low guttural rumble conveying anger	99	98	95	94	194	193
Guffaw	Guffaw loud unrestrained laugh of strong amusement	99	99	99	99	198	198
Gulps	Gulps short audible swallow from nervousness or ingesting	98	93	99	89	197	182
Gurgling	Gurgling bubbling sound in the throat	98	98	99	98	197	196
Hand Scratching Head	Hand Scratching Head rubbing hair soft rustling of fingers on scalp	100	100	99	99	199	199
Hand Slaps	Hand Slaps solid impact sound of palm against a surface or another hand	100	100	100	100	200	200
Heavy Breathing	Heavy Breathing deep labored breaths of fatigue or tension	100	100	98	98	198	198
Hiccup	Hiccup spasmodic contraction of diaphragm creating a ‘hic’ sound	100	100	100	100	200	200
Hiccups	Hiccups involuntary diaphragmatic spasm producing ‘hic’ sound	99	99	97	97	196	196
Hiss	Hiss sustained sibilant sound of disapproval	100	96	99	99	199	195
Humming	Humming low continuous vocal tone often tuneful or absent-minded	98	98	100	100	198	198
Kissing Noises	Kissing Noises repetitive gentle lip compression for affection	98	98	100	98	198	196
Kissing Sounds	Kissing Sounds soft repeated lip contact expressing affection	98	98	100	99	198	197
Licking Sound	Licking Sound light quick contact of tongue on surface	98	97	100	99	198	196
Lip Smack	Lip Smack soft pop of parted lips indicating anticipation	100	85	100	86	200	171
Low Mumble	Low Mumble indistinct vocalization of uncertainty or drowsiness	100	74	100	77	200	160
Mournful Wail	Mournful Wail long high cry of grief	100	100	100	96	200	200
Nervous Giggle	Nervous Giggle high-pitched laugh indicating anxiety	100	61	96	54	196	145
Nervous Gulp	Nervous Gulp audible swallow indicating anxiety	99	63	97	67	196	130
Normal Breathing	Normal Breathing regular rhythmic breaths at rest	100	12	100	16	200	29
Pain Moan	Pain Moan low vocalization signaling discomfort	100	99	99	99	199	198
Panting	Panting rapid breathing due to exertion or excitement	100	98	96	95	196	196
Person Whistling	Person Whistling to Get Attention serious sharp whistle	99	99	100	93	199	199
Person Whistling Playfully	Person Whistling Playfully lighthearted melodic whistle	99	99	99	99	198	198
Pleasure Moan	Pleasure Moan soft prolonged sound of enjoyment	100	100	99	89	199	189
Purr	Purr soft continuous vibrating hum showing contentment	100	94	98	88	198	182
Quiet Sob	Quiet Sob soft crying with short broken breaths	100	96	100	98	200	196
Relief Sigh	Relief Sigh release of tension after stress	78	68	98	90	176	159
Resonant Hum	Resonant Hum lower-pitched thoughtful hum	99	98	93	93	192	192
Scream	Scream loud high-pitched outburst of extreme emotion	97	97	100	100	197	197
Sharp Inhale	Sharp Inhale quick breath of sudden realization	100	59	100	81	200	145
Sharp Whistle	Sharp Whistle quick high-pitched attention grabber	99	99	100	100	199	199
Shriek	Shriek sharp piercing cry typically of fear	96	96	99	99	195	195
Slap Face	Slap Face sharp impact sound of palm against face	96	90	97	88	193	178
Slow Breathing	Slow Breathing paced controlled breaths for calm	99	99	100	100	199	199
Slurping Noises	Slurping Noises wet sucking sound typically when consuming liquids	99	93	99	93	198	186
Smack One s Lips	Smack One’s Lips light pop of lips often from dryness or taste anticipation	98	97	98	97	196	194
Smacks Lips	Smacks Lips repetitive lip popping after tasting or anticipating food	99	97	99	97	198	194
Snicker	Snicker stifled laugh often conveying mockery or sarcasm	100	77	97	82	197	173
Sniff	Sniff audible inhalation through the nose possibly sadness or restraint	100	100	100	99	200	199
Snort	Snort quick burst of air through the nose often disbelief	99	76	99	94	198	170
Snorting Giggle	Snorting Giggle brief nasal snort combined with laughter	99	99	96	96	195	195
Sobs	Sobs audible heavier crying with convulsive gasps	98	93	98	94	196	190
Soft Hum	Soft Hum steady tone with closed lips in contentment	98	95	100	99	198	198
Soft Whistle	Soft Whistle gentle tuneful blowing of air	100	100	100	100	200	200
Spitting	Spitting forceful expulsion of saliva	99	66	100	71	199	137
Sucking Noise	Sucking Noise soft pulling sound created by suction	100	100	100	99	200	199
Surprised Gasp	Surprised Gasp quick intake of breath in astonishment	98	98	97	97	195	195
Swallows	Swallows audible movement of liquid down the throat	99	97	99	99	198	196
Tongue Click	Tongue Click distinct clack made by tongue against palate	100	100	99	99	199	199
Trembling Whimper	Trembling Whimper faint cry indicating fear or pain	98	95	100	96	198	191
Tsk	Tsk clicking sound of reprimand or annoyance	99	98	99	98	198	196
Whispered Mumble	Whispered Mumble very quiet muddled speech sounds	98	94	96	88	194	182
Wistful Sigh	Wistful Sigh soft exhale tinged with longing	100	100	89	86	189	186
Wolf Whistle	Wolf Whistle two-note whistle expressing admiration	99	97	99	98	198	195
Yawn	Yawn involuntary wide-mouthed breath showing tiredness	99	96	88	87	187	183
TOTAL		8,097	7,539	8,078	7,543	16,175	15,242

Valid = samples with match score 1 or 2 (label is at least slightly accurate).

Usage

Source

Derived from [](https://huggingface.co/datasets/krishnakalyan3/vocal_bursts_taxonomy_100_clean_wds).