ivelin/ui_refexp
This dataset is intended for UI understanding, referring expression and action automation model training. It's based on the UIBert RefExp dataset from Google Research, which is based on the RICO dataset.
424
Dataset Card for UIBert
Dataset Description
- Homepage: https://github.com/google-research-datasets/uibert
- Repository: https://github.com/google-research-datasets/uibert
- Paper: https://arxiv.org/abs/2107.13731
- Leaderboard:
- UIBert: https://arxiv.org/abs/2107.13731
- Pix2Struct: https://arxiv.org/pdf/2210.03347
Dataset Summary
This is a Hugging Face formatted dataset derived from the Google UIBert dataset, which is in turn derived from the RICO dataset.
Supported Tasks and Leaderboards
- UI Understanding
- UI Referring Expressions
- UI Action Automation
Languages
- English
Dataset Structure
screenshot: blob of pixels.prompt: Prompt referring to a UI component with an optional action verb. For example "click on search button next to menu drawer."target_bounding_box: Bounding box of targeted UI components.[xmin, ymin, xmax, ymax]
Data Splits
- train: 15K samples
- validation: 471 samples
- test: 565 samples
Dataset Creation
Curation Rationale
[More Information Needed]
Source Data
Initial Data Collection and Normalization
[More Information Needed]
Who are the source language producers?
[More Information Needed]
Annotations
Annotation process
[More Information Needed]
Who are the annotators?
[More Information Needed]
Personal and Sensitive Information
[More Information Needed]
Considerations for Using the Data
Social Impact of Dataset
[More Information Needed]
Discussion of Biases
[More Information Needed]
Other Known Limitations
[More Information Needed]
Additional Information
Dataset Curators
[More Information Needed]
Licensing Information
[More Information Needed]
Citation Information
[More Information Needed]
Contributions
[More Information Needed]