Mobile-R1/Mobile-R1

Dataset Card for Mobile-R1

Dataset Description

• —Homepage: Mobile-R1 homepage
• —Paper: Mobile-R1: Towards Interactive Capability for VLM-Based Mobile Agent via Systematic Training

Dataset Structure

• —images/: All the screenshots
• —data.jsonl: The trajectory data

Data Fields

All screenshots are stored in the images/ directory.

We describe the structure of a single trajectory entry from the file data.jsonl, which contains the full interaction trajectories and action history.

• —app_name: String. The name of the mobile application (e.g., "闲鱼" / Xianyu) where the task is performed.
• —trajectory_length: Integer. Number of steps in the trajectory, indicating how many actions were taken to complete the task.
• —instruction: String. The user instruction describing the goal of the task (e.g., "Open Xianyu and search for the cheapest GTX 5090").
• —system: String. System prompt provided to the agent, guiding its reasoning and action prediction behavior.
• —image_width: Integer. Width of the device screen at the time of screenshot capture, used for coordinate alignment.
• —image_height: Integer. Height of the device screen at the time of screenshot capture, used for coordinate alignment.
• —trajectory: List of step dictionaries. Each entry corresponds to one interaction step and contains:
• —image: String. Filename of the PNG image capturing the screen at that step.
• —thought: String. The agent's internal reasoning about the current state and next action.
• —subtask: String. A concise description of the current subtask to be completed.
• —action: String. A JSON-formatted string representing the function call executed at this step:
• —name: Function name (e.g., "mobile_use").
• —arguments: Dictionary containing the following action parameters:
• —action (string, required): The action to perform. Supported values:
• —"key": Perform a key event (e.g., "volume_up", "power", "clear"; text required).
• —"click": Click at the specified screen coordinate (coordinate required).
• —"long_press": Long-press at the specified coordinate for a given duration (coordinate, time required).
• —"swipe": Swipe from one coordinate to another (coordinate, coordinate2 required).
• —"type": Input specified text into the active input box (text required).
• —"system_button": Press a system button (button required; options: "Back", "Home", "Menu", "Enter").
• —"open": Open an app (text required for app name).
• —"wait": Wait for the specified number of seconds (time required).
• —"terminate": Terminate the current task and report its status (status required, "success" or "failure").
• —coordinate (array of two ints): [x, y] pixel coordinates from the top-left corner. Required for "click", "long_press", and "swipe".
• —coordinate2 (array of two ints): [x, y] pixel coordinates for the end point. Required only for "swipe".
• —text (string): The text to input (for "type", "key", or app name for "open").
• —time (number): Duration in seconds. Required for "long_press" and "wait".
• —button (string): System button identifier ("Back", "Home", "Menu", "Enter"). Required for "system_button".
• —status (string): The status of the terminated task ("success" or "failure"). Required only for "terminate".
• —response: String. Full model response including <think>, <action>, and <tool_call> XML tags.

Data Stats

The dataset contains a total sample of 1,007 trajectories, covering 28 different mobile applications. The overall statistics are summarized in the following table:

📌 Note: Each trajectory may contain multiple samples (steps), representing individual interactions during task execution.

Dataset Creation

We have created a high-quality dataset comprising 24,521 manually annotated instances derived from 4,635 trajectories. To facilitate the training and utilization of our dataset, focusing on Chinese mobile applications, we have open-sourced a sample of 1007 trajectories covering 28 different applications.