Datasetpublic

gui-wm/agentnet-partial-and-fail-v1

agentnet-partial-and-fail-v1 GUI state transitions (s, a, s') walked on an Ubuntu desktop by Qwen3.8-27B, from tasks taken from AgentNet and run inside OSWorld's Docker environment. Walks the judge ruled partial or failed. These are the larger half and, for a world model, the more useful one: a walk that did not finish still opened dialogs, switched tabs and changed settings, and each of those is a real transition. Measured over eighty-eight walks, a failure visits 13.5 distinct… See the full description on the dataset page: https://huggingface.co/datasets/gui-wm/agentnet-partial-and-fail-v1.

sourceHugging Faceapache-2.0updated 15d agoView on Hugging Face

1likes627downloads

Dataset Card

agentnet-partial-and-fail-v1

GUI state transitions (s, a, s') walked on an Ubuntu desktop by Qwen3.8-27B, from tasks taken from AgentNet and run inside OSWorld's Docker environment.

Walks the judge ruled partial or failed. These are the larger half and, for a world model, the more useful one: a walk that did not finish still opened dialogs, switched tabs and changed settings, and each of those is a real transition. Measured over eighty-eight walks, a failure visits 13.5 distinct screens against a success's 7.7.

This repository is the one-condition pool. Tasks are drawn from the 623 Ubuntu-origin AgentNet recordings whose instruction states a single condition, and shards are appended as each batch of a hundred is walked. The 1,050 two-condition tasks will be released separately as v2; a task that failed to walk here rejoins this pool rather than that one.

What a row is

One row is one transition: the screen before an action, the action, and the screen after it.

column	what it holds
`state_id` / `next_state_id`	the two states this transition joins
`action.executed`	the pyautogui that ran, in screen pixels
`action_nl`	what the agent said it was about to do
`action_target`	the accessibility element the coordinate landed on -- role, name, bounds. For scoring an answer by element rather than by pixel
`transition_role`	`main` for a step along the trajectory, `branch` for an alternative tried from it. Never summed
`candidate_origin`	which proposer suggested a branch: `uniform-element`, `policy-sampled`, `policy-diverse`
`judged_outcome`	`success`, `partial` or `failure`, for the trajectory this row belongs to
`split` / `split_key`	`train`, `val` or `test`, assigned 8:1:1 per source recording so a task and every branch of it stay in one split. The same assignment is used by both halves of this release
`condition_count` / `pool`	how many conditions the task states, and which pool it was drawn from. `v1` is the 623 Ubuntu tasks with one condition; `v2` will be the 1,050 with two. Carried on the row so merged shards stay separable
`collection_round`	which batch of a hundred tasks wrote this row
`judged_behaviour`	`reasonable` or `unreasonable`
`hinted` / `hint_text`	whether the agent was given the route a person took, and what it was
`walker_version`	fingerprint of the code that chose the click, so re-walks sit beside the walks that prompted them

Screenshots and accessibility trees are files beside the rows:

step_000/before.png       step_000/after.png
step_000/before.a11y.xml  step_000/after.a11y.xml
step_000/before.a11y.txt  step_000/after.a11y.txt

Pinning a version

Batches are appended, never rewritten: a shard published in an earlier round keeps its bytes and its filename, and each new batch adds files with the next part numbers. So a copy downloaded last week is still valid, it is simply smaller.

Each batch is tagged, so an exact snapshot can be asked for by name:

python

from datasets import load_dataset

# the set as it stood after batch one, and it will never change
load_dataset("gui-wm/agentnet-partial-and-fail-v1", revision="round1")

# everything published so far
load_dataset("gui-wm/agentnet-partial-and-fail-v1")

A reader who already holds the whole set can do the same locally by filtering collection_round.

How it was collected

AgentNet task (Ubuntu-origin only)
  -> environment prepared: the file the task names, the package it removes,
     the folder it opens, built or fetched to match what the task asks for
  -> Qwen3.8-27B walks the task, given the accessibility element list and the
     route a person took
  -> at every state, three proposers each suggest five alternatives; each is
     tried from a snapshot and undone
  -> a judge reads every screenshot, what the agent said, what actually ran,
     and the accessibility difference, and rules on the trajectory

Numbers


batches	6
trajectories	328
transitions	90,578
spine (`transition_role=main`)	6,300
branch	84,278

By application

application	transitions
chrome	11,111
gimp	13,352
libreoffice_calc	6,261
libreoffice_impress	29,345
libreoffice_writer	4,940
os	3,382
vlc	6,463
vs_code	15,724

Branches by proposer

proposer	transitions
policy-diverse	29,990
policy-sampled	23,281
uniform-element	31,007

On the Hub

413 files, 44.5 GB.

Honest limits

—The judge is a model. judged_outcome comes from Qwen3.8-27B reading the walk. Checked by hand on twenty trajectories: about 90% agreed. The two that did not were both cases of the agent being credited for a file the setup had opened for it, which the judge is now told about explicitly.
—Information-lookup tasks are ambiguous. "search for X and tell me about it" has no clear line between having found it and having seen it. Those are in the data and their verdicts are the least reliable.
—Success is not the point of the larger half. A walk that failed can still have visited a dozen distinct screens, and those transitions are as real as any other. That is what agentnet-partial_and_fail-v1 is for.