CoolFace
Datasetpublic

jacob-valdez/synthux-economy-r10-w184

SynthUX Computer-Use Dataset — economy sim r10 Grounded visual computer-use trajectories generated by SynthUX. Each record is one worker's device session inside a simulated company: a goal expands into a node tree, terminal nodes drive real desktop-simulator apps (Terminal, Notes, VS Code, Browser, Slack/Teams, Mail, Sheets, Slides, …) through low-level mouse/keyboard input, and the observed trajectory is recorded as a screen video plus per-node frames. App-native… See the full description on the dataset page: https://huggingface.co/datasets/jacob-valdez/synthux-economy-r10-w184.

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
0likes12downloads
Dataset Card

SynthUX Computer-Use Dataset — economy sim r10

Grounded visual computer-use trajectories generated by SynthUX. Each record is one worker's device session inside a simulated company: a goal expands into a node tree, terminal nodes drive real desktop-simulator apps (Terminal, Notes, VS Code, Browser, Slack/Teams, Mail, Sheets, Slides, …) through low-level mouse/keyboard input, and the observed trajectory is recorded as a screen video plus per-node frames.

App-native multi-application execution (no workbench overlay), per-actor content diversity, cross-OS messaging, and large multi-company economies. Each economy simulation is published as its own dataset repo.

Contents

  • —data.jsonl — one record per rendered trajectory.
  • —screenshots/<trajectory>/<node_id>.png — a frame per terminal node, taken at that node's last input event.
  • —videos/<trajectory>.webm — the full screen recording.

Record schema

fieldmeaning
trajectory_idunique id (<actor>__<env>)
actor_id, company, role_id, titlewho, in which org
envsimulator: macos-web-next / windows-web-next / browser-os
goalnatural-language goal for the session
tree.nodesthe actor's terminal nodes (surface, action, target, args)
trajectory.input_eventsdense low-level mouse/keyboard/script events with t_ms
trajectory.observationscausal simulator state after each node
trajectory.alignmentlinks observations + input-event ranges to node ids
media.videoscreen recording (webm)
media.framesper-node PNG frames (nodeid, tms, blob_ref)

Stats (this build)

  • —trajectories: 36
  • —videos: 36 · frames: 843
  • —companies: 6
  • —by env: browser-os (5), macos-web-next (18), windows-web-next (13)

Provenance

Generated with SynthUX (synthux.multicompany.day.v1). Observations are causal (mutation channel = lowlevelinput); no high-level app mutation is used during capture.