latkes/inside-out-replication-v2-gemma-old-tokenizer-v1

Inside-Out Replication V2 — Gemma OLD-tokenizer canary Negative-result canary on google/gemma-2-9b-it / P26 / test (25-qid subset), testing whether the HF tokenizer-files update (commit 4efc01a1, "Update tokenizer files (#36)", 2024-08-07) is a candidate cause of our Gemma P(True) anomaly (~0.64 vs paper-implied ~0.47). Two arms — same model weights, same scoring code, same data, same attn implementation (sdpa) and query_pre_attn_scalar=256. Only the tokenizer revision differs:… See the full description on the dataset page: https://huggingface.co/datasets/latkes/inside-out-replication-v2-gemma-old-tokenizer-v1.