CoolFace
Datasetpublic

dougalldeepmind/2026-08-27-odcv-post-action-retrospection-716-eval

ODCV-Bench: post-action-retrospection (design B) 716 arm, 2 rollouts x 65 cells field value experiment ODCV-Bench rollouts and judge scores for LASR-Callum/2026-08-26-qwen36-lora-table2-9284-post-action-retrospection-716-rank-64-dynbatch: the da716 organism whose 716 rows are five-turn post-action-retrospection records (a difficult-advice prompt, a bare refusal, pushback, then the reasoning the refusal skipped; only the last turn trained). Headline on these 65 cells:… See the full description on the dataset page: https://huggingface.co/datasets/dougalldeepmind/2026-08-27-odcv-post-action-retrospection-716-eval.

sourceHugging Faceupdated 27d agoView on Hugging Face
0likes75downloads
Dataset Card

ODCV-Bench: post-action-retrospection (design B) 716 arm, 2 rollouts x 65 cells

fieldvalue
experimentODCV-Bench rollouts and judge scores for LASR-Callum/2026-08-26-qwen36-lora-table2-9284-post-action-retrospection-716-rank-64-dynbatch: the da716 organism whose 716 rows are five-turn post-action-retrospection records (a difficult-advice prompt, a bare refusal, pushback, then the reasoning the refusal skipped; only the last turn trained). Headline on these 65 cells: MR 20.3% [10.9, 30.0], severity 1.04, n=128.
date_generated2026-08-27
constitutionconstitutions/claudedistilled12principlesmid/constitution.md (9 principles), the same as difficult advice, via the adapter's training data LASR-Callum/2026-08-26-table2-9284-post-action-retrospection-716-train
source_repohttps://github.com/Matthew-Bozoukov/Lessonsfromconstituitional_AFT.git @ ccb2d4b31f931f66482e88a399049d74142881b0
modelstarget: LASR-Callum/2026-08-26-qwen36-lora-table2-9284-post-action-retrospection-716-rank-64-dynbatch (thinking mode, vLLM, maxmodellen 65536); judges: {"grok-4.20": "x-ai/grok-4.20", "gemini-3.1-pro-preview": "google/gemini-3.1-pro-preview"}
generation_config{"temperature": 0.0, "rolloutspercell": 2, "cells": 65, "config": "scratch/parb/odcvbencht29284par7162x65.yaml", "concurrency": 12, "driver": "laptop Docker Desktop over an SSH tunnel to a RunPod H100 (scratch/parb/odcvlocal_run.sh)"}
schemapasses/laptop/<run>/: each raw pass as the supervisor pushed it (agentlogs/.../messagesrecord.txt = the rollout, rolloutmanifest.json, passaudit.json). <combined>/: the two passes merged into rolloutNNN/ per scenario, evaluations/scores<judge>.json, results.json (ours vs the base-fp8 reference), results.md.
provenancescratch/odcvboxrun.py --passes 2 --extra concurrency=12; scratch/odcvcombinepasses.py --config scratch/parb/odcvbencht29284par7162x65.yaml; scratch/odcvjudgecli.py --rolloutdir <combined> --config scratch/parb/odcvbencht29284par7162x65.yaml; scratch/parb/push_odcv.py