nikarpoff/music-sentiment-analysis-happy-sad-13M

Abstract

The development of a system for automatic mood recognition in music using machine learning techniques is highly relevant due to several factors.

First, the growing availability of raw audio files in large-scale digital music libraries requires effective tools for organization and retrieval. Manual annotation of mood is time-consuming, subjective, and not feasible at scale. An automated solution enables consistent and efficient labeling of vast music catalogs.

Second, the system is envisioned as a fully free and open-source solution, which makes it accessible to researchers, developers, and the broader community.

Third, in practice, such a system can be applied to in automatic mood annotation for large music catalogs. Streaming services, music archives, and content management platforms could benefit from more intuitive mood-based search, recommendation, and playlist generation, improving user experience and personalization.

There are represented mel-spectrogram-based approache, which transform audio into a time-frequency representation that is particularly well-suited for deep learning models like CNNs.

The source dataset is the open MTG Jamendo dataset with ~14,000 tracks under Creative Commons licenses.

Architecture

To reduce the dimensionality of sequence-like input data, the architecture incorporates Convolutional Networks. Transformer and GRU architectures were employed for feature analysis.

Usage

Best models are represented in this repository. You can load happy/sad classification model with 13 millions params.

To use model, you must transform raw audio data into mel-spectrogram. The easiest way to do this is use the library musan:

pip install musan

Then in code:

import musan
import io

with open(filename, 'rb') as f:
    mp3_bytes = f.read()
    mp3_bytes = io.BytesIO(mp3_bytes)

model_hs = musan.load_pretraned_hs_model()
result = musan.predict(mp3_bytes, model_hs, verbose=True)

Results:

19.10.2025 17:56:04: spec shape: torch.Size([4, 96, 4096])

19.10.2025 17:56:05: prediction time: 0:00:00.475831; running on cpu

{'predict': 'sad', 'happy': 0.4539340138435364, 'sad': 0.5460659861564636}

Also you can use both of hs and re models:

hs_model, re_model = musan.load_pretraned_models()
result = musan.predict(mp3_bytes, hs_model, re_model, verbose=True)

Source code of the musan library you can find in github repository

license: apache-2.0 ---