thaki-AI/daily-paper-2026-08-29-goodhart-shift-self-evolving-harness

The Goodhart Shift: Measuring Train-vs-Holdout Divergence in Overnight Self-Evolving Agent Skill Loops TL;DR — When an overnight self-evolving agent loop edits its own skills against the same bench it reports on, its self-reported gains can stop generalizing. This paper formalizes that failure as the Goodhart shift - a per-night train-vs-holdout divergence - and specifies a sealed-holdout divergence gate that tells "the loop improved" apart from "the loop overfit." ThakiCloud… See the full description on the dataset page: https://huggingface.co/datasets/thaki-AI/daily-paper-2026-08-29-goodhart-shift-self-evolving-harness.