NEXTLab-ZJU/popular-hook

Popular Hooks

This is the dataset repository for the paper: Popular Hooks: A Multimodal Dataset of Musical Hooks for Music Understanding and Generation, in 2024 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

1. Introduction

Popular Hooks, a shared multimodal music dataset consisting of 38,694 popular musical hooks for music understanding and generation; this dataset has the following key features:

• —Multimodal Music Data
• —Accurate Time Alignment
• —Rich Music Annotations

2. Modalities

• —Midi
• —Lyrics
• —Video (Youtube link provided, you need to download it by yourself)
• —Audio

3. High Level Music Information

• —Melody
• —Harmony
• —Structure
• —Genre
• —Emotion(Russell's 4Q)
• —Region

4. Dataset File Structure

• —info_tables.xlsx: it contains a list describing the baisc information of each midi file (index, path, song name, singer, song url, genres, youtube url, youtube video start time and end time/duration, language, tonalities)
• —midi/{index}/{singername}/{songname}:
• —completetextemotion_result.csv: it contains the emotion class(4Q) which is predicted with the total lyrics of the song.
• —song_info.json: it contains the song's section info, theorytab DB url and genres info.
• —total_lyrics.txt: it contains the song's complete lyrics which is collected from music API(lyricsGenius, NetEase, QQMusic)
• —youtube_info.json: it contains the url of the song in Youtube, the start time and end time/duration of the video section.
• —./{section}
• —{section}.mid: the section in midi format
• —{section}.txt: it contains the tonalites of the section.
• —{section}audioemotion_result.csv: it contains the emotion class(4Q) which is predicted with the audio of the section.
• —{section}_lyrics.csv: it contains the lyrics of the section.
• —{section}midiemotion_result.csv: it contains the emotion class(4Q) which is predicted with the midi of the section.
• —{section}multimodalemotion_result.csv: it contains the emotion class(4Q) which is selected from the multimodal emotions of the section.
• —{section}textemotion_result.csv: it contains the emotion class(4Q) which is predicted with the lyrics of the section.
• —{section}videoemotion_result.csv: it contains the emotion class(4Q) which is predicted with the video of the section.

5. Demo

<img src='https://huggingface.co/datasets/NEXTLab-ZJU/popular-hook/resolve/main/imgs/popularhooksdemo.png'>