Modelpublic

ACE-Step/acestep-captioner

sourceHugging Facemitupdated 8mo agoView on Hugging Face

73likes1.8kdownloads

Model Card

<a href="https://arxiv.org/abs/2602.00744">Tech Report</a>

ACE-Step Captioner

Description

ACE-Step Captioner is the annotation model used by ACE-Step v1.5 for training data labeling. It is a professional-grade music captioning model that generates detailed, structured descriptions of audio content.

Performance

🏆 Accuracy surpasses Gemini Pro 2.5 in music description tasks

Key Features

—🎼 Musical Style Analysis - Identifies genres, sub-genres, and stylistic influences
—🎸 Instrument Recognition - Detects and describes 1000+ instrument types and combinations
—🎭 Structure & Progression - Analyzes musical arrangement including intro, verse, chorus, bridge, climax, and outro
—🔊 Timbre Description - Captures tonal qualities, textures, and sonic characteristics
—📝 Rich Vocabulary - Supports 1000+ descriptive terms for comprehensive music annotation

Usage

The usage is the same as Qwen2.5 Omni-7B.

Prompt Format

Use the following prompt to caption audio:

*Task* Describe this audio in detail
<audio>

Output Format

The model generates natural language descriptions covering multiple aspects of the music.

Example Output

A melancholic indie folk track featuring fingerpicked acoustic guitar 
as the primary instrument. The song opens with a sparse, contemplative 
intro before the vocals enter with a breathy, intimate delivery. 
The arrangement gradually builds through the verse, adding subtle 
string pads and a gentle kick drum. The chorus lifts with layered 
harmonies and a warmer, fuller texture. The bridge introduces a 
key change and emotional climax before returning to the stripped-down 
acoustic arrangement for the outro.

Descriptive Capabilities

Musical Styles (Examples)

Category	Styles
Electronic	Ambient, Techno, House, Drum & Bass, Synthwave, IDM, Downtempo
Rock	Alternative, Indie, Post-Rock, Progressive, Psychedelic, Grunge
Pop	Synth-pop, Electropop, Dream Pop, Art Pop, Indie Pop
Classical	Orchestral, Chamber, Minimalist, Neo-Classical, Cinematic
World	Latin, African, Middle Eastern, Asian Traditional, Celtic
Jazz	Fusion, Smooth, Bebop, Modal, Free Jazz
Hip-Hop	Trap, Boom Bap, Lo-fi, Instrumental, Cloud Rap

Instruments (1000+ Supported)

Category	Examples
Strings	Acoustic Guitar, Electric Guitar, Violin, Cello, Bass, Harp, Mandolin
Keys	Piano, Synthesizer, Organ, Rhodes, Wurlitzer, Mellotron
Percussion	Drums, Electronic Drums, Congas, Bongos, Timpani, Vibraphone
Wind	Saxophone, Trumpet, Flute, Clarinet, Oboe, French Horn
Electronic	Synth Bass, Pad, Lead, Arpeggiator, Sampler, 808, 303

Structure Analysis

—Intro / Outro - Opening and closing sections
—Verse / Pre-Chorus / Chorus - Main song structure
—Bridge / Break - Transitional sections
—Build-up / Drop / Climax - Dynamic progression
—Interlude / Solo - Instrumental passages

Timbre Descriptions

Dimension	Descriptors
Texture	Warm, Bright, Dark, Crisp, Muddy, Clean, Distorted, Saturated
Space	Reverberant, Dry, Spacious, Intimate, Cavernous, Tight
Dynamics	Punchy, Soft, Aggressive, Gentle, Compressed, Dynamic
Character	Ethereal, Gritty, Smooth, Raw, Polished, Organic, Synthetic

Use Cases

—Music AI Training - Generate high-quality captions for music generation models
—Music Information Retrieval - Create searchable metadata for audio databases
—Content Moderation - Analyze and categorize music content
—Music Education - Provide detailed analysis for learning purposes
—Audio Production - Document and describe sound design elements