jiahaomei/MUSE-VA

MUSE-VA Dataset

![arXiv]()   

**English** | **中文**

MUSE-VA (Multimodal MUSic Emotion Dataset with Balanced VA) is a large-scale multimodal music emotion dataset designed for music emotion understanding, emotion-controllable music generation, and cross-modal affective modeling. The dataset starts from target coordinates sampled in the continuous Valence-Arousal (VA) space and uses a five-stage LLM agent pipeline with affective and musical knowledge injection to construct music, text, images, and structured annotations, thereby actively controlling the coverage of the emotion space.

The dataset contains 6,254 music samples. Each sample provides continuous VA annotations, a discrete music emotion label, music metadata, text descriptions, and a one-to-one paired image. MUSE-VA can support research on music emotion recognition, music emotion regression, emotion-conditioned music generation, music-image cross-modal retrieval, image-to-music generation, music-to-image generation, and related affective computing tasks.

The LLM-generated textual annotation is used to generate two matched music-image pairs, which are stored as separate data entries. Entries sharing the same textual annotation do not appear across different dataset splits.

Validated through human subjective experiments, the two discrete emotion labels Annoying and Indignant/defiant showed relatively weak agreement with human judgments and should be used with caution.

Dataset Highlights

• —Large-scale music data: 6,254 music samples.
• —Balanced VA coverage: target emotion coordinates are sampled from the continuous VA space to improve emotion-space coverage.
• —Multi-granularity emotion annotations: each sample includes continuous valence/arousal values and a discrete emotion label.
• —Multimodal content: each sample includes audio, a paired image, a music caption, music tags, visual imagery, and visual tags.
• —Structured music attributes: metadata include genre, instrumentation, tempo, key, and composition notes.
• —Consistency verification: each sample includes a cross-modal and affective consistency result with an explanatory reason.

Recommended Uses

MUSE-VA can be used for:

• —music emotion recognition and continuous VA regression
• —discrete music emotion classification
• —emotion-controllable music generation
• —text-conditioned music generation
• —image-conditioned music generation
• —music-image semantic and affective matching
• —cross-modal affective representation learning
• —affective computing applications such as music recommendation, music-therapy support, and human-computer interaction

Fields

Each row in the Hugging Face dataset corresponds to one music sample with its paired image, text annotations, emotion annotations, and consistency verification result. The released dataset contains the following fields:

Data Source and Generation Pipeline

MUSE-VA is constructed through a traceable synthetic data generation pipeline:

1. 1.Sample target emotion coordinates from the continuous VA space.
2. 2.Map VA coordinates to a discrete emotion label and a theme word based on ANEW and the Cowen music emotion dataset.
3. 3.Generate structured music attributes from the emotion, theme, and musical knowledge.
4. 4.Generate a music caption and music tags.
5. 5.Generate visual imagery, visual tags, and an image description corresponding to the music emotion.
6. 6.Verify consistency among the VA coordinates, emotion label, music attributes, music description, and visual imagery.
7. 7.Keep samples that pass the consistency check in the final dataset. The same textual annotation is used to generate two matched music-image pairs, which are stored as separate data entries.

Text descriptions, music attributes, and consistency checks were generated with Gemini 2.5 Pro. Music audio was generated with Suno v5. Paired images were generated with Qwen-Image-2512.

Dataset Splits

MUSE-VA is randomly split into training, validation, and test sets with an 80%/10%/10% ratio. One LLM-generated textual annotation is used to generate two matched music-image pairs. Because some generations may fail, the validation and test splits are not necessarily even-numbered.

License

The textual annotations, audio files, and image files in MUSE-VA are released under the Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0).

This license allows sharing, use, and adaptation for non-commercial purposes, provided that appropriate attribution is given and changes are indicated. Commercial use of the dataset is not permitted without additional authorization.

The data construction pipeline and associated LLM prompts are released under the Apache License 2.0 to support reproducible research and further extension.

Citation

If you use MUSE-VA in your research, please cite the corresponding paper.

@article{museva,
  title   = {MUSE-VA: A Large-Scale Synthetic Multimodal Dataset for Music Emotion Understanding and Generation},
  author  = {Mei, Jiahao and Yuan, Yixuan and Gu, Haoyu and Li, Shimeng and Jin, Leyuan and Wu, Mengyue and Ding, Yue},
  journal = {},
  year    = {2026}
}

Notes

• —MUSE-VA is a synthetic multimodal dataset. Audio, images, and some textual annotations may reflect the preferences and limitations of the generative models used during construction.
• —Music emotion perception is subjective. Cultural background, listening experience, and individual affective state may influence how listeners perceive the same music.
• —Additional human verification is recommended for high-risk applications or user-facing deployments.
• —Human evaluation results are reported in the paper. The two discrete emotion labels Annoying and Indignant/defiant show relatively weak agreement with human listeners and should be used with caution.