CoolFace
Datasetpublic

gray311/mobilegym-trajectories-autoglm-phone-9b

MobileGym Trajectories — AutoGLM-Phone-9B (L1) Agent rollout trajectories collected on the MobileGym simulated-Android environment with AutoGLM-Phone-9B as the policy. One row per episode (rollout), in the spirit of SWE-Gym-style instance datasets. Each row carries the task spec, the full multimodal interaction (screenshots + model responses + parsed actions), and a deterministic reward from MobileGym's JSON-state judge (no VLM judging — verdicts are exact).… See the full description on the dataset page: https://huggingface.co/datasets/gray311/mobilegym-trajectories-autoglm-phone-9b.

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
0likes29downloads
Dataset Card

MobileGym Trajectories — AutoGLM-Phone-9B (L1)

Agent rollout trajectories collected on the [MobileGym](https://mobilegym.dev) simulated-Android environment with AutoGLM-Phone-9B as the policy. One row per episode (rollout), in the spirit of SWE-Gym-style instance datasets. Each row carries the task spec, the full multimodal interaction (screenshots + model responses + parsed actions), and a deterministic reward from MobileGym's JSON-state judge (no VLM judging — verdicts are exact).

Provenance

Policy model`zai-org/AutoGLM-Phone-9B` (GLM-4V-9B based, GUI-specialized), served via vLLM
Agent adapterautoglm (do(action=...) DSL, 0–1000 normalized coords)
EnvironmentMobileGym simulator (device=sim), bench_env runner + Playwright/Chromium
Rewarddeterministic state-diff judge (is_success, progress, clean)
Tasksall L1 tasks (40 classes, 17 apps), --repeat-n 3, temperature 1.0

Stats

  • —120 episodes, 50 success (42%), 0 errors
  • —termination: 97 COMPLETE, 17 MAX_STEPS, 6 REPETITIVE_LOOP
  • —task-level: 22 tasks with within-group reward variance (RL-usable), 6 always-pass, 12 always-fail

This is trainable data: the 22 variance tasks have both positive and negative trajectories sharing an initial state, suitable for DPO (success vs fail pairs) and (offline) GRPO (group-relative advantage). The autoglm_L1_good.txt task split lists the 22 RL-usable tasks.

Schema

columntypedescription
instance_idstringunique, <task_id>_t<trial>
task_id / suitestringtask class / app suite
instructionstringnatural-language goal (the "problem statement")
difficulty / objective / composition / scopestringtask taxonomy
capabilities / appslist[str]required capabilities / apps touched
success / reward / progress / cleanbool/floatdeterministic judge outputs
num_steps / max_steps / stop_reasonint/strepisode termination info
system_promptstringagent system prompt
messagesstring (JSON)reconstructed multimodal chat (system + per-step user[image+text]/assistant)
stepsstring (JSON)per-step action_type / action_data / thought / response / image_index
imagesSequence(Image)per-step screenshots (1080×2400), aligned to steps via image_index
model / agentstringprovenance
python
from datasets import load_dataset
ds = load_dataset("gray311/mobilegym-trajectories-autoglm-phone-9b")["train"]
ds[0]["images"][0]      # PIL screenshot
ds[0]["reward"]         # 1.0 / 0.0

Downstream: RFT = filter success==True (50 trajectories); DPO = pair success vs fail rows sharing task_id (22 tasks); offline GRPO = group by task_id, use reward variance.