WensongSong/AnyInsertion_V1

license: mit task_categories:

• —image-to-image language:
• —en prettyname: a sizecategories:
• —10M<n<100M ---

AnyInsertion

<p align="center"> <a href="https://song-wensong.github.io/"><strong>Wensong Song</strong></a> · <a href="https://openreview.net/profile?id=~Hong_Jiang4"><strong>Hong Jiang</strong></a> · <a href="https://z-x-yang.github.io/"><strong>Zongxing Yang</strong></a> · <a href="https://scholar.google.com/citations?user=WKLRPsAAAAAJ&hl=en"><strong>Ruijie Quan</strong></a> · <a href="https://scholar.google.com/citations?user=RMSuNFwAAAAJ&hl=en"><strong>Yi Yang</strong></a> <br> <br> <a href="https://arxiv.org/pdf/2504.15009" style="display: inline-block; margin-right: 10px;"> <img src='https://img.shields.io/badge/arXiv-InsertAnything-red?color=%23aa1a1a' alt='Paper PDF'> </a> <a href='https://song-wensong.github.io/insert-anything/' style="display: inline-block; margin-right: 10px;"> <img src='https://img.shields.io/badge/Project%20Page-InsertAnything-cyan?logoColor=%23FFD21E&color=%23cbe6f2' alt='Project Page'> </a> <a href='https://github.com/song-wensong/insert-anything' style="display: inline-block;"> <img src='https://img.shields.io/badge/GitHub-InsertAnything-black?logoColor=23FFD21E&color=%231d2125'> </a> <br> <b>Zhejiang University &nbsp; | &nbsp; Harvard University &nbsp; | &nbsp; Nanyang Technological University </b> </p>

News

• —<span style="color:red">[2025.5.9] Release new AnyInsertion v1 text- and mask-prompt dataset on HuggingFace.</span>
• —[2025.5.7] Release AnyInsertion v1 text prompt dataset on HuggingFace.
• —[2025.4.24] Release AnyInsertion v1 mask prompt dataset on HuggingFace.

Summary

This is the dataset proposed in our paper **Insert Anything: Image Insertion via In-Context Editing in DiT**

AnyInsertion dataset consists of training and testing subsets. The training set includes 136,385 samples across two prompt types: 58,188 mask-prompt image pairs and 78,197 text-prompt image pairs;the test set includes 158 data pairs: 120 mask-prompt pairs and 38 text-prompt pairs.

AnyInsertion dataset covers diverse categories including human subjects, daily necessities, garments, furniture, and various objects.

[image]

Directory

├── text_prompt/
│   ├── train/
│   │   ├── accessory/
│   │   │   ├── ref_image/      # Reference image containing the element to be inserted
│   │   │   ├── ref_mask/       # The mask corresponding to the inserted element
│   │   │   ├── tar_image/      # Ground truth
│   │   │   └── src_image/      # Source images
│   │   │       ├── add/        # Source image with the inserted element from Ground Truth removed
│   │   │       └── replace/    # Source image where the inserted element in Ground Truth is replaced
│   │   ├── object/
│   │   │   ├── ref_image/      
│   │   │   ├── ref_mask/      
│   │   │   ├── tar_image/      
│   │   │   └── src_image/      
│   │   │       ├── add/       
│   │   │       └── replace/    
│   │   └── person/
│   │       ├── ref_image/     
│   │       ├── ref_mask/       
│   │       ├── tar_image/     
│   │       └── src_image/    
│   │           ├── add/       
│   │           └── replace/  
│   └── test/
│       ├── garment/
│       │   ├── ref_image/      
│       │   ├── ref_mask/      
│       │   ├── tar_image/      
│       │   └── src_image/      
│       └── object/
│           ├── ref_image/     
│           ├── ref_mask/       
│           ├── tar_image/     
│           └── src_image/      
│
├── mask_prompt/
│   ├── train/
│   │   ├── accessory/
│   │   │   ├── ref_image/     
│   │   │   ├── ref_mask/      
│   │   │   ├── tar_image/     
│   │   │   ├── tar_mask/      # The mask corresponding to the edited area of target image
│   │   ├── object/
│   │   │   ├── ref_image/    
│   │   │   ├── ref_mask/      
│   │   │   ├── tar_image/     
│   │   │   ├── tar_mask/      
│   │   └── person/
│   │       ├── ref_image/     
│   │       ├── ref_mask/     
│   │       ├── tar_image/    
│   │       ├── tar_mask/   
│   └── test/
│       ├── garment/
│       │   ├── ref_image/   
│       │   ├── ref_mask/    
│       │   ├── tar_image/    
│       │   ├── tar_mask/    
│       ├── object/
│       │   ├── ref_image/    
│       │   ├── ref_mask/    
│       │   ├── tar_image/     
│       │   ├── tar_mask/    
│       └── person/
│           ├── ref_image/   
│           ├── ref_mask/   
│           ├── tar_image/  
│           ├── tar_mask/    

    

Example

<div style="display: flex; text-align: center; align-items: center; justify-content: space-between;">   <figure style="margin: 10px; width: calc(25% - 20px);">     <img src="examples/refimage.png" alt="Refimage" style="width: 100%;">     <figcaption>Refimage</figcaption>   </figure>   <figure style="margin: 10px; width: calc(25% - 20px);">     <img src="examples/refmask.png" alt="Refmask" style="width: 100%;">     <figcaption>Refmask</figcaption>   </figure>   <figure style="margin: 10px; width: calc(25% - 20px);">     <img src="examples/tarimage.png" alt="Tarimage" style="width: 100%;">     <figcaption>Tarimage</figcaption>   </figure>   <figure style="margin: 10px; width: calc(25% - 20px);">     <img src="examples/tarmask.png" alt="Tarmask" style="width: 100%;">     <figcaption>Tarmask</figcaption>   </figure>   <figure style="margin: 10px; width: calc(25% - 20px);">     <img src="examples/add.png" alt="Add" style="width: 100%;">     <figcaption>Add</figcaption>   </figure>   <figure style="margin: 10px; width: calc(25% - 20px);">     <img src="examples/replace.png" alt="Replace" style="width: 100%;">     <figcaption>Replace</figcaption>   </figure> </div>

Text Prompt

Add Prompt: Add [label from tar_image (in label.json) ]</p> Replace Prompt: Replace [label from src_image (in srcimage/replace/replacelabel.json) ] with [label from tar_image (in label.json) ]

Citation

@article{song2025insert,
  title={Insert Anything: Image Insertion via In-Context Editing in DiT},
  author={Song, Wensong and Jiang, Hong and Yang, Zongxing and Quan, Ruijie and Yang, Yi},
  journal={arXiv preprint arXiv:2504.15009},
  year={2025}
}