facebook/emu_edit_test_set

1---2configs:3- config_name: default4  data_files:5  - split: validation6    path: data/validation-*7  - split: test8    path: data/test-*9dataset_info:10  features:11  - name: instruction12    dtype: string13  - name: image14    dtype: image15  - name: task16    dtype: string17  - name: split18    dtype: string19  - name: idx20    dtype: int6421  - name: hash22    dtype: string23  - name: input_caption24    dtype: string25  - name: output_caption26    dtype: string27  splits:28  - name: validation29    num_bytes: 766327032.2930    num_examples: 202231  - name: test32    num_bytes: 1353530752.033    num_examples: 358934  download_size: 190459829035  dataset_size: 2119857784.2936---37 38# Dataset Card for the Emu Edit Test Set39 40 41## Table of Contents42- [Table of Contents](#table-of-contents)43- [Dataset Description](#dataset-description)44  - [Dataset Summary](#dataset-summary)45- [Additional Information](#additional-information)46  - [Licensing Information](#licensing-information)47  - [Citation Information](#citation-information)48 49## Dataset Description50 51- **Homepage: https://emu-edit.metademolab.com/**52- **Paper: https://emu-edit.metademolab.com/assets/emu_edit.pdf**53 54### Dataset Summary55 56To create a benchmark for image editing we first define seven different categories of potential image editing operations: background alteration (background), comprehensive image changes (global), style alteration (style), object removal (remove), object addition (add), localized modifications (local), and color/texture alterations (texture).57Then, we utilize the diverse set of input images from the [MagicBrush benchmark](https://huggingface.co/datasets/osunlp/MagicBrush), and for each editing operation, we task crowd workers to devise relevant, creative, and challenging instructions.58Moreover, to increase the quality of the collected examples, we apply a post-verification stage, in which crowd workers filter examples with irrelevant instructions.59Finally, to support evaluation for methods that require input and output captions (e.g. prompt2prompt and pnp), we additionally collect an input caption and output caption for each example. 60When doing so, we ask annotators to ensure that the captions capture both important elements in the image, and elements that should change based on the instruction.61Additionally, to support proper comparison with Emu Edit with publicly release the model generations on the test set [here](https://huggingface.co/datasets/facebook/emu_edit_test_set_generations).62For more details please see our [paper](https://emu-edit.metademolab.com/assets/emu_edit.pdf) and [project page](https://emu-edit.metademolab.com/).63 64 65### Licensing Information66 67Licensed with CC-BY-NC 4.0 License available [here](https://creativecommons.org/licenses/by-nc/4.0/legalcode?fbclid=IwAR2SYZjLRywwUMblkWg0LyAxHVVTloIFlvC-ju3BthIYtOM2jpQHgbeXOsM).68 69### Citation Information70```71@inproceedings{Sheynin2023EmuEP,72  title={Emu Edit: Precise Image Editing via Recognition and Generation Tasks},73  author={Shelly Sheynin and Adam Polyak and Uriel Singer and Yuval Kirstain and Amit Zohar and Oron Ashual and Devi Parikh and Yaniv Taigman},74  year={2023},75  url={https://api.semanticscholar.org/CorpusID:265221391}76}77```