laion/Emilia-with-Emotion-Annotations
Dataset Card for Emilia with Emotion Annotations Dataset Description This dataset is an enhanced version of the Emilia dataset, enriched with detailed emotion annotations. The annotations were generated using models from the EmoNet suite to provide deeper insight into the emotional content of speech. This work is based on the research and models described in the blog post "Do They See What We See?". The annotations include 54 scores for each sample, covering a… See the full description on the dataset page: https://huggingface.co/datasets/laion/Emilia-with-Emotion-Annotations.
Dataset Card for Emilia with Emotion Annotations
Dataset Description
This dataset is an enhanced version of the Emilia dataset, enriched with detailed emotion annotations. The annotations were generated using models from the EmoNet suite to provide deeper insight into the emotional content of speech. This work is based on the research and models described in the blog post "Do They See What We See?".
The annotations include 54 scores for each sample, covering a wide range of emotional and paralinguistic attributes, as well as an emotion caption generated by the BUD-E Whisper model. The goal is to enable more nuanced research and development in emotionally intelligent AI.
Dataset Structure & Access
The dataset includes the original Emilia audio data, with the addition of the new emotion annotations, provided in WebDataset format.
Currently, the dataset is distributed across five Hugging Face repositories:
laion/Emilia-with-Emotion-Annotationslaion/Emilia-with-Emotion-Annotations2laion/Emilia-with-Emotion-Annotations3laion/Emilia-with-Emotion-Annotations4laion/Emilia-with-Emotion-Annotations5
To access the complete dataset, you must gather the data from all five repositories. (Note: We plan to merge these into a single repository in a the coming days with an even better annotated version.)
The original .tar files for the Emilia dataset are also included. Files belonging to the YODAS subset can be identified by a suffix in their filenames.
Dataset Statistics
This combined dataset comprises approximately 215,600 hours of speech, merging the original Emilia dataset with a large portion of the YODAS dataset. The inclusion of YODAS significantly expands the linguistic diversity and the total volume of data.
The language distribution is broken down as follows:
Interpretation of Scores
The models predict raw scores for 40 emotional categories and 14 attribute dimensions. For the emotional categories, these raw scores are also used to calculate a normalized Softmax probability, indicating the relative likelihood of each emotion.
Citation
If you use this dataset, please cite the original Emilia dataset paper as well as the EmoNet-Voice paper.
@inproceedings{emilialarge,
author={He, Haorui and Shang, Zengqiang and Wang, Chaoren and Li, Xuyuan and Gu, Yicheng and Hua, Hua and Liu, Liwei and Yang, Chen and Li, Jiaqi and Shi, Peiyang and Wang, Yuancheng and Chen, Kai and Zhang, Pengyuan and Wu, Zhizheng},
title={Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation},
booktitle={arXiv:2501.15907},
year={2025}
}
@article{emonet_voice_2025,
author={Schuhmann, Christoph and Kaczmarczyk, Robert and Rabby, Gollam and Friedrich, Felix and Kraus, Maurice and Nadi, Kourosh and Nguyen, Huu and Kersting, Kristian and Auer, Sören},
title={EmoNet-Voice: A Fine-Grained, Expert-Verified Benchmark for Speech Emotion Detection},
journal={arXiv preprint arXiv:2506.09827},
year={2025}
}