RLE-Bench/robocasa-openfridge-kinex-v0.3.0-astra-low-eval

OpenFridge: Kinex(v0.3.0) and Codex evaluation

**Watch the evaluation website** · Dataset files · Website source

Ten episodes evaluated on September 20, 2026 with GPT-6 Astra / low, standard service tier, fast mode disabled. Five sequential episodes per harness, seeds 0–4. Each episode starts a fresh native world and conversation; learned tools, skills and memos persist according to the existing protocol.

The native cap is 5,000 control intervals per episode, synchronous at 20 Hz, with an 8-hour safety timeout. Neither harness has an additional configured model-loop cap. Both trials finished without infrastructure exceptions or retries. These are five-episode learning sequences, not independent replicated estimates or a causal learning ablation. The older publication used a different Kinex(v0.3.0) predecessor configuration with a 100-turn cap; this comparison is not a controlled estimate of a cap-removal benefit.

Included evidence

• —Complete available native-format sessions, readable transcripts, tool inputs/outputs and provider/configuration evidence.
• —Per-episode learned resources, workspace snapshots, file hashes, text diffs and a tool-call input index.
• —Ten edited dual-view videos, 1536×896 with native 768×768 views, 4× simulation speed, captions and initial/final holds.
• —Interaction images, original camera observations (PNG/PGM), native verdicts, commands, timelines, model XML, state layouts and precomputed door diagnostics.
• —Outcome analyses, protocol, artifact/browser acceptance, source identities and publication integrity manifests.

Raw model.mjb, states.f64, unedited videos and duplicate native-evidence archives are not published. They remain in the original local evaluation. Third-party notices apply to simulator-derived assets; no blanket asset license is asserted.

Read and download

Data format · Episode CSV · Episode JSON · Full analysis · Native door audit · Protocol · Artifact acceptance

Files use the same episodes/kinex/episode_01/ through episode_05/ and episodes/codex/ structure as the previous edition. Display names use Kinex(v0.3.0); original machine identifiers and session content are preserved. The Static Space keeps website assets and edited videos, and links here for full evidence. This dataset also supports local browsing.

hf download RLE-Bench/robocasa-openfridge-kinex-v0.3.0-astra-low-eval --repo-type dataset --local-dir openfridge-kinex-v0.3.0
cd openfridge-kinex-v0.3.0
python3 tools/verify_publication.py
python3 -m http.server 8000

Open http://localhost:8000/. Native success requires all target fridge doors to reach normalized opening ≥0.90 simultaneously; freezer motion alone is insufficient. A stopped failure is a finalized attempt, not an infrastructure exception. The state-derived measurements establish which physical condition remained unfinished; they do not independently prove a controller root cause.

Replay verification restored recorded native states; it was not command resimulation. Raw-state audits and native rerendering require the intentionally omitted source state/model files. Public users can inspect the precomputed measurements and audit code. Captured interaction assets include all 33 image references in the five Kinex(v0.3.0) traces. Asset presence alone does not prove use in a particular episode.

Publication transformations record path substitutions, archive repacking and the versioned video captions. Native sessions are not abridged; no historical Codex log recovery was applied to this run. Nested original manifests retain historical hashes; publication-manifest.json verifies the published copy. Local acceptance receipts reflect their original check time before HF upload.