atharparvezce/iOS-1K-Mobile-UI-Dataset

MUIAnno:iOS-1K-Mobile-UI-Dataset

Overview

MUIAnno:iOS-1K-Mobile-UI-Dataset is a curated dataset of 1,000 real-world iOS mobile UI screens collected from diverse application categories available on the Apple App Store.

Each screen is paired with human-validated structured JSON ground truth annotations, enabling research in UI understanding, layout analysis, and multimodal modeling.

The dataset includes:

• —Simple layouts (e.g., login, onboarding screens)
• —Visually dense interfaces (e.g., feeds, checkout flows)
• —Structured UI element annotations
• —Bounding boxes for UI components
• —Element type labels
• —Clickability attributes
• —Text content for text-bearing elements

Dataset Structure

ios-ui-dataset/
│
├── images/
│   ├── 0001.png
│   ├── 0002.png
│   └── ...
│
├── annotations/
│   ├── 0001.json
│   ├── 0002.json
│   └── ...
│
├── metadata.csv
├── README.md
└── dataset_infos.json

Metadata Format

The metadata.csv file provides mapping between images and annotations:

Example Screen

Below is a sample Eleven Reader App’s login screen from the Dataset :

<div align="center"> <img src="images/0001.png" width="250"> </div>

Annotation Format

Each JSON annotation follows a structured schema:

{
  "screen_id": "0001",
  "elements": [
    {
      "id": 1,
      "type": "button",
      "bbox": [x, y, width, height],
      "text": "Login",
      "clickable": true
    }
  ]
}

Each element includes:

• —type: UI component category (e.g., button, text, image)
• —bbox: Bounding box coordinates
• —text: Visible text content (if applicable)
• —clickable: Boolean interaction label

Intended Use Cases

This dataset is designed for:

• —Mobile UI understanding
• —Layout parsing and structural analysis
• —UI element detection
• —Vision–language modeling
• —Multimodal LLM grounding
• —Autonomous UI agent research

Download Instructions

You can download the dataset using the Hugging Face Hub:

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="atharparvezce/iOS-1K-Mobile-UI-Dataset",
    repo_type="dataset",
    local_dir="./iOS-1K-Mobile-UI-Dataset"
)

Limitations

• —Covers only iOS platform interfaces
• —Contains 1,000 screens in the current release
• —Category distribution reflects App Store sampling
• —UI copyrights remain with original application developers

Future Work

We are actively working on extending the MUIAnno:iOS-1K-Mobile-UI-Dataset with:

• —Additional UI screens across more application categories
• —Increased dataset scale beyond 1,000 screens
• —More detailed attribute-level annotations
• —Expanded layout complexity coverage
• —Benchmark splits for training and evaluation

Our goal is to develop this into a larger benchmark for mobile UI understanding and multimodal research.

If you are interested in collaboration, contributing to the dataset, or using extended versions for research purposes, please feel free to reach out:

📧 atharparvezce@gmail.com

License

This dataset is released under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license.

• —Attribution required
• —Non-commercial use only

The dataset is intended strictly for academic and research purposes. No personal user data is included.

Citation

If you use this dataset in your research, please cite the MUIAnno paper:

@article{parvez2026muianno,
  title   = {{MUIAnno}: An Expert-Annotated Dataset and Evaluation Benchmark for Mobile UI Understanding},
  author  = {Parvez, Athar and Mufti, Muhammad Jawad and Gull, Muqaddas and Hammad, Omar},
  journal = {arXiv preprint arXiv:2605.17656},
  year    = {2026},
  doi     = {10.48550/arXiv.2605.17656},
  url     = {https://arxiv.org/abs/2605.17656}
}