delcenjo/char-transformer-ablations

char-transformer ablation leaderboard

Static page comparing four training runs of the same small, from-scratch GPT-style transformer, each with one part removed or changed: the full baseline, no positional embeddings, a single attention head instead of four, and a byte-pair tokenizer instead of a character one. All numbers on the page come from the real training logs of that ablation study — final metrics from ablation.json and the per-step loss curves from the matching *_history.json files — with no values entered by hand.

The page itself is plain HTML, CSS and JavaScript with no framework and no external requests, so it starts instantly and never needs to wake up.

Source code and the training scripts that produced these numbers: https://github.com/delcenjo/transformer-from-scratch

Trained model on the Hub: https://huggingface.co/delcenjo/char-transformer-shakespeare

Files

• —index.html, style.css, app.js — the page.
• —data.js — the embedded dataset, generated (not hand-written) by tools/generate_data.py from the real checkpoint logs.
• —tools/generate_data.py — regenerates data.js from a local checkout of char-transformer/checkpoints.

Note on the frontmatter emoji field

Spaces require a YAML emoji field in this README and the Hub only accepts an actual emoji character there (it is used as the Space's icon, similar to a favicon). Everything else in this Space — the page content, this README's body — has no emoji or icon glyphs. The field is set to a plain black square (⬛) rather than a themed or decorative emoji, since it is Hub metadata, not part of the page.