M64/sid-gpt-25m

SID-GPT 25M

A GPT model trained to generate Commodore 64 SID music by learning from legendary composers.

[Listen to samples](#audio-samples) | [GitHub](https://github.com/M64GitHub/SidGPT)

Model Description

SID-GPT learns to predict SID register states frame-by-frame, essentially learning the "language" of C64 chiptune music. Trained on 2,410 songs from HVSC, it produces output with recognizable musical structures: kick drums, PWM sweeps, basslines, and arpeggios.

Training Data

• —Source: HVSC (High Voltage SID Collection)
• —Size: 1GB of register dump sequences (2,410 SID files)
• —Composers: DRAX (530 songs), Laxity (287), Rob Hubbard (96), Jeroen Tel (176), Martin Galway (40), and 10 others

Files

Usage

Zig Inference Engine (Recommended)

The native Zig engine runs at ~350-120 tok/s with SIMD and KV caching, depending on context window:

# Clone repository
git clone https://github.com/M64GitHub/SidGPT
cd SidGPT
zig build -Doptimize=ReleaseFast

# Download model
wget https://huggingface.co/M64/sid-gpt-25m/resolve/main/sid-gpt-1700.bin -P models/

# Generate and play
./zig-out/bin/sidgpt --model models/sid-gpt-1700.bin --frames 700 --temp 0.90 --seed 7391738265 --context 12 | ./zig-out/bin/sidgpt-play

# Or export to WAV
./zig-out/bin/sidgpt --model models/sid-gpt-1700.bin --frames 700 --temp 0.90 --seed 7391738265 --context 12 --output music.txt
./zig-out/bin/sidgpt-play music.txt --output-wav music.wav

Python Inference

cd training
python sample_sid.py --checkpoint path/to/sid-gpt-1700.pt --num_frames 700 --temperature 0.95

Generation Tips

Good seeds to try: 1337, 7391738264, 7391738265, 4829173650

Audio Samples

Generated outputs from this model:

Proof of Concept Status

Despite only 12 frames (0.24 sec) of context, the model learned real SID techniques:

• —Kick drums - Pulse wave frequency sweeps transitioning to noise
• —PWM sweeps - Pulse width modulation fades (Rob Hubbard signature)
• —Basslines - Melodic bass patterns with movement
• —Arpeggios - Fast note sequences typical of SID music
• —Leads - Fading-in lead voices

Limitations

• —Short context: 12 frames = no long-range song structure
• —Seed dependent: Quality varies significantly with random seed
• —No conditioning: Cannot specify style/artist (planned for v2)
• —Pattern matching: Learns techniques, not "composing"

Training Details

Loss progression:
  Iter 0:     2.88 (random)
  Iter 200:   0.96 (structure learned)
  Iter 700:   0.37 (musical patterns)
  Iter 1000:  0.27 (kick drums, PWM)
  Iter 2000:  0.21 (best checkpoint)

Training was stopped at iter 2000 when validation loss plateaued and train/val gap exceeded 30% (indicating overfitting).

Technical Details

Data Format

Each frame is 25 SID registers encoded as 50 hex characters + newline:

B0080005410A306011C0064108200016800D41082000B4031F
B0084005410A30601100074108200016C00D41082000B4031F
...
<end>

• —50 frames = 1 second of audio
• —Vocabulary: 0-9, A-F, <, >, d, e, n, \n (22 tokens)

Inference Optimizations

The Zig engine includes:

• —KV Cache: 50-100x speedup for autoregressive generation
• —SIMD: @Vector(8, f32) operations, 24x speedup
• —Sliding Window: Infinite generation beyond context length

Citation

@misc{sidgpt2026,
  author = {Mario Schallner},
  title = {SID-GPT: Transformer-based Commodore 64 Music Generation},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/M64/sid-gpt-25m}
}

Links

• —GitHub Repository
• —Training Dataset 1GB training data (2,410 SID files)
• —HVSC - Training Data Source
• —Blog Post (coming soon)

Acknowledgments

Thanks to the legendary C64 composers whose work made this possible: Matt Gray, Jeroen Tel, Rob Hubbard, Martin Galway, DRAX, Laxity, and all contributors to HVSC.