lievan/tmax_filtered_3x3k

tmaxfiltered3x3k

Agentic rollouts on 3,000 TMax tasks, sampled three times per task from each of four models. One subset per model; every row is one complete trajectory.

All rollouts were sampled at a 65,536-token context window, recorded in the context_window column. The subset name is the model alone, since the context level is a property of this corpus rather than a choice a consumer makes.

Row schema

Scalars are typed and directly queryable: reward, turns_used, stop_reason, turn_truncations, graded_on_limit, time_seconds, queue_seconds, plus model identifiers and the task instruction and system_prompt.

These columns hold JSON strings -- call json.loads to recover the original object: messages, tool_schemas, reward_details, grader_results, loop_stats, usage, task_meta, sampling, truth_access.

They are stored as text because messages[].content is a plain string for system, user and tool turns but a list of content parts for assistant turns. Parquet has no union type, so typing the column would have meant discarding one of the two shapes.

from datasets import load_dataset
import json

ds = load_dataset("lievan/tmax_filtered_3x3k", "nemotron3-ultra", split="train")
row = ds[0]
messages = json.loads(row["messages"])
print(row["id"], row["reward"], len(messages))

loop_stats is null for 520 of 36,000 rollouts; every other field is present on every row.

Provenance and caveats

Tasks come from allenai/TMax-15K; check its terms before redistributing. reward is the graded outcome, so a subset is not filtered to successes -- select on reward > 0 yourself. Some trajectories in the security-domain tasks contain synthetic AWS-style key fixtures such as AKIAIOSFODNN7EXAMPLE, which are part of the task content and not real credentials.