mahalisyarifuddin/goemotions-ekman

GoEmotions: Majority-Clean Ekman Balanced Dataset (8:1:1 Split)

A clean, single-label, perfectly balanced redistribution of [GoEmotions](https://huggingface.co/datasets/google-research-datasets/go_emotions) (raw config), mapped to the 8 universal Ekman emotion categories (enjoyment, surprise, fear, sadness, disgust, anger, contempt, neutral).

Standard GoEmotions suffers from extreme class imbalance, synthetic masking tokens ([NAME], [RELIGION]), rater disagreement noise, and non-textual junk (emoticon-only, ASCII-art, number-mash, zalgo, markdown tables) that rater majorities agree on anyway. This redistribution removes placeholder tags, drops near-zero-letter junk and markdown tables, keeps only sentences where a simple majority of raters agree on one Ekman label, deduplicates on comment id (duplicate wordings across distinct comments are independent observations and are retained), resolves multi-label ambiguity, and provides exact class balance across an 8:1:1 Train/Validation/Test split with no text leakage. All randomness uses seed 0.

Motivation & Applications

Curated for downstream tasks like lightweight text classification, emotional text-to-speech (TTS), and expressive AI behavior.

Modern AI emotion representation faces a strict trade-off:

• —Too Limited: 3-class sentiment (Positive/Negative/Neutral) lacks human expressiveness.
• —Too Granular: 28+ text labels or infinite paralinguistic TTS speech tags cause severe data sparsity, annotation noise, and control instability.

By standardizing expressive text into 8 universal basic emotions with exact class balance, this dataset provides a robust middle ground for global scalability without heavy computational overhead.

Pipeline & Column Schema

1. 1.Tag Filtering: Removed 29,664 rater rows containing [NAME] or [RELIGION] masking tokens -> 181,561 rows (49,875 unique comment ids).
2. 2.Letter-Ratio Filtering: Removed 26 rater rows (8 unique ids) whose text is almost entirely non-letters (sum(c.isalpha()) / len(text) < 0.18) -> 181,535 rows. Catches emoticon-only (:^(, (:), :((), braille ASCII-art, number-mash, and zalgo texts that rater majorities otherwise agree on — while keeping short informal texts ($2 by 2029, I <3 you!, aw :(, .......... no u!).
3. 3.Markdown-Table Filtering: Removed 3 rater rows (1 unique id) whose text contains >= 2 | characters (markdown-table signature) -> 181,532 rows. The only such text is a 138-pipe NBA standings table; the next-highest legit text has 1 pipe, so the threshold is exact with a wide empirical gap. Quote-replies (> ...), **emphasis**, and #shout texts were surveyed and kept as legitimate discourse.
4. 4.Ekman Mapping: Collapsed 28 fine-grained emotions into 8 basic Ekman categories following Demszky et al. (2020). Following close investigation, annoyance and disapproval were mapped to `contempt`. Each rater casts a vote only if exactly one Ekman category is active (ekman_count == 1); multi-label raters abstain (161,820 single-label votes, 89.1%).
5. 5.Simple-Majority Agreement: Kept only comment ids where one Ekman label holds a strict majority (>50%) of ALL raters (abstentions count in the denominator, so the winner is always unique — no ties possible). Ids with no majority winner are dropped -> 35,898 ids (72.0% retained).
6. 6.Sentence-Level Collapsing: One row per id, using the first supporting rater (lowest rater_id among raters who voted for the winner) as the representative, so the 28 fine-grained columns stay consistent with the winning Ekman label (verified programmatically). Records n_raters and n_votes.
7. 7.Id-Based Deduplication: Exactly one row per comment id (uniqueness asserted). Cross-id duplicate wordings are retained as independent observations — 126 texts span multiple ids (201 extra rows, largest group Thank you. ×15). The full pool is therefore the complete majority set: every winning id included. 16 duplicate texts earned conflicting majority labels across comments (e.g. Weird flex but ok → neutral/disgust/contempt); they stay in full for transparency but are excluded from balanced sampling.
8. 8.Balanced 8:1:1 Text-Group-Aware Sampling (seed 0): Bottleneck class is Disgust (273 eligible). Sampled exactly 270 rows per class (270 8 = 2,160 total), partitioned into 216 Train, 27 Validation, and 27 Test* per class. Duplicate texts are sampled and split as whole groups so no text ever spans splits, and conflicting-label texts are excluded from the balanced pool. Verified: exact balance, no text/id leakage across splits.
9. 9.Separate Column Schema: Preserves the exact 37 original columns in their original order (text through neutral; rater_id is now the representative supporting rater). Appends the winning label (ekman_label), separate prefixed indicator columns (ekman_enjoyment … ekman_neutral) to guarantee zero collision between fine-grained labels and merged Ekman categories, plus agreement metadata (n_raters, n_votes).

Usage

from datasets import load_dataset

# 1. Load default balanced 8:1:1 dataset (2,160 rows across train/val/test)
ds = load_dataset("mahalisyarifuddin/goemotions-ekman")
print(ds["train"][0]["ekman_label"])

# 2. Or load full unsubsampled majority-clean pool (35,898 rows)
ds_full = load_dataset("mahalisyarifuddin/goemotions-ekman", "full")

Distributions & Mapping

Balanced Split (balanced config — 2,160 rows)

Every split maintains exact 12.5% class balance:

• —Train (`train.parquet`): 1,728 rows (216 / class)
• —Validation (`validation.parquet`): 216 rows (27 / class)
• —Test (`test.parquet`): 216 rows (27 / class)

Full Majority-Clean Pool (full config — 35,898 rows)

Unsubsampled clean pool in full/train.parquet: enjoyment (16,188), neutral (11,439), surprise (3,228), contempt (1,936), sadness (1,828), anger (615), fear (390), disgust (274).

Mapping Taxonomy

• —`enjoyment`: amusement, excitement, joy, love, desire, optimism, caring, pride, admiration, gratitude, relief, approval
• —`surprise`: realization, surprise, curiosity, confusion
• —`fear`: fear, nervousness
• —`sadness`: remorse, embarrassment, disappointment, sadness, grief
• —`disgust`: disgust
• —`anger`: anger
• —`contempt`: annoyance, disapproval
• —`neutral`: neutral

Changelog

• —2026-09-16: Deduplication key changed from text to id — the 201 cross-id duplicate wordings are retained as independent observations, so full is now the complete majority set (35,697 -> 35,898). Balanced sampling is text-group-aware (duplicate wordings never span splits) and excludes the 16 conflicting-label texts; balanced unchanged at 2,160.
• —2026-09-16: Added markdown-table filter (≥ 2 | characters) removing the 138-pipe NBA standings table. Full pool 35,698 -> 35,697; balanced unchanged at 2,160.
• —2026-09-15: Added minimum letter-ratio filter (≥ 0.18 Unicode letters) removing 6 near-zero-letter texts (emoticon-only ×3, braille ASCII-art, number-mash, zalgo) while keeping short informal texts. Full pool 35,704 -> 35,698; balanced unchanged at 2,160.
• —2026-09-15: Added simple-majority (>50% of raters) Ekman agreement filter, sentence-level collapsing with representative rater, exact-text deduplication, and n_raters / n_votes columns. All sampling with seed 0. Replaces the original rater-level pool (161,841 rows / 20,160 balanced) with a higher-precision majority-clean pool (35,704 rows / 2,160 balanced).
• —Initial release: Tag filtering + Ekman mapping + single-label extraction + balanced 8:1:1 sampling at rater level.

License & Attribution

Distributed under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0), preserving Google Research's original license. Per Section 4(b), notice is given of tag filtering ([NAME]/[RELIGION]), minimum letter-ratio filtering, markdown-table filtering, 28 -> 8 Ekman aggregation, simple-majority agreement filtering, sentence-level collapsing, id-based deduplication, and stratified text-group-aware 8:1:1 balanced sampling (seed 0).

@inproceedings{demszky2020goemotions,
  author = {Demszky, Dorottya and Movshovitz-Attias, Dana and Ko, Jeongwoo and Cowen, Alan and Nemade, Gaurav and Ravi, Sujith},
  booktitle = {58th Annual Meeting of the Association for Computational Linguistics (ACL)},
  title = {{GoEmotions: A Dataset of Fine-Grained Emotions}},
  year = {2020}
}