project-telos/gpt_oss_doorkey_action_distributions

GPT-OSS-20B DoorKey action-distribution time series

This dataset contains sentence-prefix next-action readouts for 46 fixed DoorKey environment states drawn from 31 trajectories in `project-telos/trajectories_key_door_100`. It also contains the corresponding offline BEAST change-point results.

The dataset has 7,084 positions. Position 0 is the readout before any reasoning text is revealed. Each subsequent position reveals one additional reasoning sentence from the same model response.

Action readout

At each position, openai/gpt-oss-20b was shown the current grid, key-carrying status, and the reasoning prefix available at that position. The model was asked for its next move. The next-token logits for the single-token candidates UP, DOWN, LEFT, and RIGHT were divided by temperature 0.7 and normalized with a softmax over those four candidates.

This is direct candidate-token scoring, not repeated generation or empirical sampling. The probabilities sum to one over the four action candidates and do not measure probability mass assigned to non-action text. top_p=0.95 is recorded for provenance but does not affect the direct logit calculation.

Model revision: 6cee5e81ee83917806bbde320786a8fb61efebee.

Configurations

action_timeseries

One row per state and sentence prefix. Important fields include:

• —example_id: fixed DoorKey state identifier.
• —trajectory_id, step_index: source trajectory and environment step.
• —position_index: zero-based position within the state's reasoning series.
• —reasoning_progress: revealed-character fraction in [0, 1].
• —prob_up, prob_down, prob_left, prob_right: normalized action probabilities.
• —argmax_action: highest-probability action at the current position.
• —action_is_optimal: whether the argmax is planner-optimal in the environment.
• —final_action: action selected after the complete reasoning trace.
• —adjacent_js_bits, adjacent_total_variation: change from the preceding position.
• —js_to_full_trace, tv_to_full_trace: distance from the final-prefix distribution.
• —argmax_action_changed: whether the recommendation changed at this boundary.

beast_timeseries

The same action probabilities augmented with:

• —l2_distance_from_initial_action_distribution;
• —beast_fitted_distance;
• —posterior_change_probability.

BEAST was fitted independently to each state using the Euclidean distance from the position-0 action distribution. It used a trend-only piecewise-linear model, minimum segment length 3, at most 10 changes, three chains, seed 42, and a posterior location threshold of 0.70. A state-level change was accepted only when posterior odds for one or more changes versus no change exceeded 9:1.

detected_change_points

One row per accepted BEAST location. The analysis detected 160 change points in 44 of the 46 states.

event_aligned

Action-distribution metrics aligned around the descriptive final-action jump, largest adjacent distribution change, and stable-action boundary. These are retrospective descriptive alignments, not independent causal estimates.

Limitations

• —The action probabilities are renormalized over four candidate tokens.
• —Readouts are conditioned on observed reasoning prefixes; they do not establish that a particular sentence caused the subsequent action change.
• —Stable-action and final-action labels use the complete time series and are retrospective.
• —The 46 states form a matched analysis cohort, not a representative capability benchmark.
• —Reasoning text, activations, model weights, and .pt files are not included.

Provenance and reproducibility

The metadata/ directory contains run configurations and SHA-256 hashes of the source analysis files. The reports/ directory contains the original analysis summaries. All row-level tables use stable example_id and trajectory_id fields so they can be joined with the source trajectory dataset.

Loading

from datasets import load_dataset

timeseries = load_dataset(
    "project-telos/gpt_oss_doorkey_action_distributions",
    "action_timeseries",
)
beast = load_dataset(
    "project-telos/gpt_oss_doorkey_action_distributions",
    "beast_timeseries",
)
change_points = load_dataset(
    "project-telos/gpt_oss_doorkey_action_distributions",
    "detected_change_points",
)

License

Apache-2.0. Users should also comply with the terms applicable to the source trajectory dataset and the GPT-OSS model.