Datasetpublic

preethamBelweave/macos-ax-agent-sft

macOS AX-Agent — SFT dataset 1,887 next-action examples for training a text-only accessibility-tree → GUI action model. Source Curated from markov-ai/computer-use (Apache-2.0): successful computer-use agent trajectories on OSWorld tasks, produced by Gemini 3 Flash with linearized accessibility-tree grounding. Only fully-successful trajectories (score = 1.0) are included. Format Each row is {"messages": [...]} (chat template of… See the full description on the dataset page: https://huggingface.co/datasets/preethamBelweave/macos-ax-agent-sft.

sourceHugging Faceapache-2.0updated 12d agoView on Hugging Face

0likes52downloads

Dataset Card

macOS AX-Agent — SFT dataset

1,887 next-action examples for training a text-only accessibility-tree → GUI action model.

Source

Curated from `markov-ai/computer-use` (Apache-2.0): successful computer-use agent trajectories on OSWorld tasks, produced by Gemini 3 Flash with linearized accessibility-tree grounding. Only fully-successful trajectories (score = 1.0) are included.

Format

Each row is {"messages": [...]} (chat template of Qwen/Qwen3-4B-Instruct-2507):

—system: computer-use agent prompt + JSON action schema (coordinates normalized to 0..1)
—user: TASK: ... + optional LAST ACTION: + CURRENT SCREEN (accessibility tree): (linearized AX tree: tag\ttext\tposition (center x & y)\tsize (w & h))
—assistant: canonical next action, e.g. {"action": "click", "x": 0.62, "y": 0.41}

Action types: click (1030), exit (275), type (274), hotkey (164), openapp (63), scroll (38), wait (24), draganddrop (15), switchwindow (4).

Splits

—train.jsonl — 1,604 examples
—valid.jsonl — 283 examples (split by trajectory, no leakage)

Reproducibility

scripts/prep_data.py regenerates the JSONL from the raw source parquet.