Datasetpublic

vedant33/supersede-rl-episodes

Supersede RL Episodes Procedurally-generated supersession episodes for reinforcement learning — the training data behind vedant33/supersede-qwen2.5-3b-grpo-lora and the Supersede environment. Paper Code Environment Model arXiv · DOI GitHub Prime Intellect Hub vedant33/supersede-qwen2.5-3b-grpo-lora What this is Each episode is a short multi-session conversation in which a fact about the user changes one or more times (they move city, switch jobs… See the full description on the dataset page: https://huggingface.co/datasets/vedant33/supersede-rl-episodes.

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes62downloads

Dataset Card

Supersede RL Episodes

Procedurally-generated supersession episodes for reinforcement learning — the training data behind `vedant33/supersede-qwen2.5-3b-grpo-lora` and the Supersede environment.

Paper	Code	Environment	Model
arXiv · DOI	GitHub	Prime Intellect Hub	`vedant33/supersede-qwen2.5-3b-grpo-lora`

What this is

Each episode is a short multi-session conversation in which a fact about the user changes one or more times (they move city, switch jobs, change car/pet), followed by a query for the fact's current value. The skill being trained is supersession: track the latest value and ignore the superseded ones. Reward is temporal fact-currency — answer with the current value, not a stale one.

These synthetic episodes are deliberately moderate-length (6–8 sessions) so RL rollouts are affordable; the resulting policy is then validated on the real [LongMemEval](https://github.com/xiaowu0162/LongMemEval) `knowledge-update` split (Wu et al., MIT), where training on these episodes transfers (base 9.0% → trained 16.7%, bounded memory).

Schema

Field	Type	Description
`question`	string	The query for the fact's current value, e.g. "Which city do I currently live in?"
`answer`	string	The current / gold value (the latest in the update chain).
`stale_values`	list[string]	The superseded values — enables a stale-answer penalty.
`sessions`	list[string]	The multi-session context, one rendered session per element; updates and distractor sessions interleaved.
`n_sessions`	int	Number of sessions (difficulty signal).
`qid`	string	Unique episode id.

Invariant: answer is never in stale_values — the gold is always the current value.

How the reward works

The companion environment scores answered_current: 1.0 iff the final answer conveys `answer` (the current value), via a programmatic, ungameable matcher (normalized variant match + token-overlap fallback) — no judge model. stale_values lets you additionally penalize answers that report a superseded value. See the environment for the matcher and the bounded-memory rollout (the agent rewrites a capped notes field per session and never re-sees raw history).

Splits

Split	Episodes	Seeds
`train`	2,000	100000–101999
`test`	200	900000–900199 (disjoint, held-out)

Usage

python

from datasets import load_dataset
ds = load_dataset("vedant33/supersede-rl-episodes", split="train")
print(ds[0]["question"], "->", ds[0]["answer"], "| stale:", ds[0]["stale_values"])

Provenance & license

Episodes are generated entirely by the open-source Supersede environment (templated; no third-party data), released under Apache-2.0. The evaluation benchmark referenced above, LongMemEval (Wu et al.), is a separate MIT-licensed dataset and is not redistributed here — load it from `xiaowu0162/longmemeval-cleaned`.

Citation

bibtex

@misc{patel2026supersede,
  title         = {Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents},
  author        = {Patel, Vedant},
  year          = {2026},
  eprint        = {2606.27472},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  doi           = {10.5281/zenodo.20837384},
  url           = {https://arxiv.org/abs/2606.27472}
}