3podi/charter-v1.0-20-S-best-acc-nonpad

audio2chart - charter v1.0

charter-v1.0 is a pretrained Transformer-based autoregressive model that generates Guitar Hero style .chart files directly from audio. It is part of the audio2chart framework for end-to-end audio-to-chart generation, and its full methodology and architecture are described in our arXiv publication.

Model Overview

The model follows a multimodal encoder–decoder Transformer design:

• —Audio Encoder: frozen pretrained Encodec model converting raw waveform into discrete audio codes.
• —Adapter: lightweight 1D convolution aligns audio embeddings to token time resolution.
• —Decoder: autoregressive Transformer with cross-attention to encoded audio, trained to predict discrete chart tokens (notes/chords) at fixed 20 ms or 40 ms intervals.
• —Tokenization: 63 categorical tokens representing all valid fret-button combinations plus padding for silent steps.
• —Training Data: ~10 k paired audio–chart samples.
• —Model Sizes: 25 M (S) and 227 M (M) parameters.

Quick Usage

🔹 Local Installation

git clone https://github.com/3podi/audio2chart.git
cd audio2chart
pip install -r requirements.txt

🔹 Load the model from Hugging Face

from inference.engine import Charter

# Load the pretrained Charter model
model = Charter.from_pretrained("3podi/charter-v1.0-20-S-best-acc-nonpad")

# Generate chart tokens from an audio file
seqs = model.generate("path/to/song.mp3")

🔹 Or use the CLI script

python generate.py path/to/song.mp3   --model_name 3podi/charter-v1.0-20-S-best-acc-nonpad   --output my_songs