ceselder/cot-oracle-paper-ablation-adam-recipe-1layer

CoT Oracle Paper Ablation: Adam Recipe, 1 Layer

This repo contains the paper ablation that keeps the Adam-style training recipe inside the cot-oracle codebase and trains a single activation readout layer.

What This Checkpoint Is

• —Base model: Qwen/Qwen3-8B
• —Adapter format: PEFT LoRA
• —Activation readout layers: [18]
• —Task order: shuffled
• —Seed: 42
• —Planned budget: 50M input tokens
• —Paper label: 17M logged training tokens

Exact Training Mixture

• —latentqa: enabled, n: -1 (all available examples in the Adam-style LatentQA export used by this repo)
• —classification: enabled, n: 20000, datasets = sst2, ag_news, snli
• —fineweb: enabled, n: 60000, variants = futurelens_fineweb,pastlens_fineweb
• —On-policy futurelens: disabled
• —On-policy pastlens: disabled
• —chunked_convqa: disabled
• —All other tasks in configs/train.yaml: disabled

Notes

• —This is the paper's "Adam recipe in this repo" ablation, not a byte-for-byte rerun of Adam Karvonen's original training script.
• —The main approximations are the use of FineWeb future/past-lens readouts and a narrower 3-dataset classification mix.
• —The token label above follows the paper bookkeeping from the run logs, while the config itself was set up with a 50M input-token budget.