Datasetpublic

violetxi/tb2-eval-qwen3-8b-wm-summary-klanchor

Terminal-Bench 2.0 eval results - Qwen3-8B WM summary KL-anchor SFT Compact Harbor eval results for violetxi/qwen3-8b-terminal-wm-summary-klanchor on Terminal-Bench 2.0 using the terminus-2 agent harness. Each dataset split is one checkpoint step. Rows are Harbor trial directories and include binary reward, per-test-case pass/fail data from verifier/ctrf.json, exception text when present, and run metadata. Raw terminal recordings, panes, and completion logs are not included.… See the full description on the dataset page: https://huggingface.co/datasets/violetxi/tb2-eval-qwen3-8b-wm-summary-klanchor.

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes149downloads

Dataset Card

Terminal-Bench 2.0 eval results - Qwen3-8B WM summary KL-anchor SFT

Compact Harbor eval results for violetxi/qwen3-8b-terminal-wm-summary-klanchor on Terminal-Bench 2.0 using the terminus-2 agent harness.

Each dataset split is one checkpoint step. Rows are Harbor trial directories and include binary reward, per-test-case pass/fail data from verifier/ctrf.json, exception text when present, and run metadata. Raw terminal recordings, panes, and completion logs are not included.

Splits

split	rows	scored	resolved	mean pct tests passed	run finished
step_250	89	70	0	21.0916	yes
step250traj	19	0	0	-	yes
step_500	89	69	0	17.4046	yes
step500traj	20	1	0	0.0	yes
step_750	89	70	1	13.7073	yes
step750traj	19	0	0	-	yes
step_1000	89	70	0	15.9616	yes
step1000traj	19	0	0	-	yes
step_1250	89	70	0	14.6423	yes
step1250traj	19	0	0	-	yes
step_1500	89	69	0	18.2079	yes
step1500traj	20	1	0	50.0	yes
step_1750	89	68	1	20.0184	yes
step1750traj	21	0	0	-	yes
step_2000	89	70	0	18.7158	yes
step2000traj	19	0	0	-	yes
step_2250	89	70	1	18.1959	yes
step2250traj	19	0	0	-	yes
step_2500	89	70	0	18.9422	yes
step2500traj	19	0	0	-	yes
step_2750	89	70	0	18.1441	yes
step2750traj	19	0	0	-	yes
step_3000	89	70	1	19.113	yes
step_3250	89	68	0	18.8385	yes
step_3500	89	68	0	17.5788	yes
step_3750	89	70	1	16.6844	yes
step_4000	89	70	0	19.113	yes
step_4250	89	70	0	17.6628	yes
step_4500	89	69	0	16.903	yes
step_4750	89	70	0	16.0049	yes
step_5000	89	70	1	17.641	yes

Columns

—reward: 1.0 iff all hidden tests passed; null if the trial did not reach verifier scoring.
—n_passed, n_tests, pct_tests_passed, test_cases: partial-credit verifier details.
—exception: truncated Harbor exception text for errored trials.
—run_*: metadata copied from the parent Harbor result.json.
—local_trial_dir: original path in the source cluster checkout for provenance.