Datasetpublic

wckwan/Alfworld-Olmo3-7B-Adaptive-Midpoint-eval

ALFWorld OLMo-3-7B-Instruct adaptive MIDPOINT: held-out evaluation Held-out ALFWorld evaluation rows for wckwan/Alfworld-Olmo3-7B-Adaptive-Midpoint (merged_hf_actor_gs<N>), appended as checkpoints are evaluated. Layout global_step_N/result.json # metrics, diagnostics, task list, config, fingerprint, runtime global_step_N/trajectories.jsonl # per-episode trajectories MANIFEST.sha256 # sha256sum -c compatible Protocol… See the full description on the dataset page: https://huggingface.co/datasets/wckwan/Alfworld-Olmo3-7B-Adaptive-Midpoint-eval.

sourceHugging Faceapache-2.0updated 3d agoView on Hugging Face

0likes468downloads

Dataset Card

ALFWorld OLMo-3-7B-Instruct adaptive MIDPOINT: held-out evaluation

Held-out ALFWorld evaluation rows for `wckwan/Alfworld-Olmo3-7B-Adaptive-Midpoint` (merged_hf_actor_gs<N>), appended as checkpoints are evaluated.

Layout

global_step_N/result.json         # metrics, diagnostics, task list, config, fingerprint, runtime
global_step_N/trajectories.jsonl  # per-episode trajectories
MANIFEST.sha256                   # sha256sum -c compatible

Protocol

standalone_eval preset alfworld_olmo3, 128 goals (eval_in_distribution), env seed 0, temperature 0.4, matching the ALFWorld baseline rows. Every row passes scripts/check_result_alfworld.py (canonical 128-episode gamefiles hash 12ead9df9760a8a8).

Runtime: vLLM 0.11.0, torch 2.8.0, transformers 4.57.1, data parallel 1 (single GPU), engine seed 0, on the EIDF tree 8abb4e2d..6ba9b713 whose 7-file eval surface is byte-identical across that range (digest 599824c1c4b07828, alfworld/projection.py 954b58f4fc0febbf). The ALFWorld baselines ran at data parallel 2, so sampling streams differ on the same goals. Each row carries an eval_surface_note with these facts.

With 128 goals the sampling standard deviation of a success rate is about 4.4 points.

Config fingerprint

Native 942c1e75645e9f07f6ecd91a66d3ddbf1b34a87097623862db0467e06bca7fcc; legacy-equivalent c12b948ddb980df68520982077f8202cc260b1abd21f9194e06c4eae7012abc3 with history_action_char_cap (None) dropped. Aggregate with --allow-mixed.