YYYYYYibo/alfworld-experimenter-gpt5mini-sft-1k

ALFWorld Experimenter GPT-5 mini SFT 1K

This dataset contains 1,000 blind GPT-5 mini reasoning demonstrations for an ALFWorld expert-prefix selection task. The intended use is to give a 7B experimenter model a structured reasoning warm start before reinforcement learning, not to treat GPT-5 mini's selected depths as ground-truth labels.

Task

For each ALFWorld task, the experimenter receives eight failed trajectories from a frozen Qwen2.5-7B-Instruct actor and one successful expert trajectory. It chooses an expert-prefix handoff depth whose actor continuation probability should be near 1/2, maximizing 4*p*(1-p).

The visible response is organized as semantic capability evidence, likely bottlenecks, comparisons among candidate depths, and one final \boxed{d}.

Sampling

The 1,000 tasks were deterministically sampled from the 1,467-task training partition of the strict root-success 0/8 cohort in `YYYYYYibo/alfworld-expert-prefix-rollouts`. The held-out 367-task validation partition was excluded completely.

Sampling was stratified by:

• —ALFWorld task type;
• —expert trajectory length bucket;
• —oracle normalized-depth bucket;
• —whether any prefix has 3--5 successes out of 8.

See metadata/sample_manifest.json and metadata/task_ids.txt for the exact sample and seed.

Teacher generation

• —API model requested: gpt-5-mini
• —Returned snapshot: gpt-5-mini-2025-08-07
• —Reasoning effort: low
• —Text verbosity: low
• —Maximum total output tokens: 1,800, including internal reasoning tokens
• —Target visible response: approximately 500--800 tokens
• —Empirical landscape shown to teacher: no
• —Oracle depth shown to teacher: no

The oracle and empirical landscape in the canonical records are audit-only fields joined after generation. They were not present in either teacher input message.

All 1,000 canonical responses are nonempty and unique. Each has exactly one legal boxed depth at the end. Visible output length ranges from 538 to 800 tokens, with mean 723, median 726, and p95 791. Seventeen initially overlong responses were regenerated with a tighter length instruction; all originals and repair attempts are retained.

Selection baselines

These numbers describe zero-shot teacher selection, not the intended final RL policy:

GPT is clearly above uniform random and approximately tied with the fixed expert midpoint overall. It has a strong late-handoff bias: 29.4% of its selections have empirical success 8/8. The purpose of subsequent RL is to correct this calibration while preserving the reasoning structure.

See analysis/selection_analysis.md and .json for paired comparisons and per-task-type results.

Files

• —data/sft_messages.parquet: default three-message chat SFT table. The input messages contain no oracle or empirical landscape.
• —jsonl/sft_messages.jsonl: lossless JSONL equivalent of the default table.
• —metadata/: sample and generation manifests and exact task IDs.
• —scripts/: generation, finalization, analysis, sampling, prompt, and run scripts used for this release.
• —SHA256SUMS: hashes for every packaged file except itself.

Raw API attempts, response IDs, and operational logs are deliberately excluded from the public release. They are retained in the private local experiment archive. The released SFT table contains every one of the 1,000 final prompts and generated responses.

Minimal usage

from datasets import load_dataset

sft = load_dataset(
    "YYYYYYibo/alfworld-experimenter-gpt5mini-sft-1k",
    "sft_messages",
    split="train",
)

print(sft[0]["messages"][-1]["content"])

Important limitation

The teacher rationales are generated explanations and may contain incorrect inferences about environment state or actor capability. In particular, an action marked as syntactically accepted does not prove that the intended state transition occurred. Use these responses as structural SFT demonstrations; use the measured continuation landscape for RL rewards and evaluation.