laion/grug-agentic-s3-step1903-repaired-eval-traces
Grug 67B repaired-export agentic evaluation traces This dataset contains the final ATIF episode from 587 de-duplicated attempts in a point-in-time snapshot of three active evaluations of laion/grug-67b-a2b-sft-s3-agentic-step1903-repaired. The snapshot was copied on 2026-07-29 at approximately 18:40 UTC. Suite Expected Terminal attempts Scored Mean reward among scored Exported trajectories ID (dev_set_v2) 300 240 209 0.013963 237 SWE-bench Verified 300 126 81 0 126… See the full description on the dataset page: https://huggingface.co/datasets/laion/grug-agentic-s3-step1903-repaired-eval-traces.
Grug 67B repaired-export agentic evaluation traces
This dataset contains the final ATIF episode from 587 de-duplicated attempts in a point-in-time snapshot of three active evaluations of `laion/grug-67b-a2b-sft-s3-agentic-step1903-repaired`. The snapshot was copied on 2026-07-29 at approximately 18:40 UTC.
Only trial directories with a terminal result.json were copied. Attempts were capped at three per task with the score-blind scripts/harbor/dedupe_harbor_trials.py utility. Thirteen retained attempts ended before writing an exportable trajectory and are not present here. The evals were still active when the snapshot was taken, so this dataset is not a complete benchmark release and was not registered as training data.
All 506 scored attempts ended through an agent timeout or signal. The repaired checkpoint uses EOS IDs [128001, 128009], a matching generation config, a PreTrainedTokenizerFast tokenizer, and the intended OpenCode chat template. The repository rename and pinned revision forced a fresh Iris model cache.
The exact SFT inputs and Levanter token caches are published at `open-athena/grug-67b-a2b-agentic-sft-training-data`.