Modelpublic

prompterminal/nanogpt-enwik8-compressed-working

sourceHugging Facemitupdated 1y agoView on Hugging Face

1likes21downloads

Model Card

Compressed nanoGPT (enwik8)

Outstanding Compression Results!

—Performance: 1.635 → 1.637 BPC (+0.002)
—Parameters: 28,801,536 → 27,359,744
—Compression: 1.053× smaller (5.0% reduction)
—Quality Loss: Only 0.1% degradation!

This demonstrates near-perfect compression of a character-level transformer.

Usage

python

from transformers import AutoModel

model = AutoModel.from_pretrained(
    "prompterminal/nanogpt-enwik8-compressed-working",
    trust_remote_code=True
)

# Generate text
import torch
prompt = torch.randint(0, 6060, (1, 10))  # Random start
output = model.generate(prompt, max_new_tokens=100)

Research Impact

First successful demonstration of high-quality compression on character-level transformers!