codelion/Grounding_DINO_demo

1---2title: Grounding DINO Demo3emoji: 💻4colorFrom: purple5colorTo: yellow6sdk: gradio7sdk_version: 5.9.18app_file: app.py9pinned: false10license: apache-2.011---12 13# Grounding DINO 14[📃Paper](https://arxiv.org/abs/2303.05499) | 15[📽️Video](https://www.youtube.com/watch?v=wxWDt5UiwY8) |16[🗯️ Github](https://github.com/IDEA-Research/GroundingDINO) |17[📯Demo on Colab](https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/zero-shot-object-detection-with-grounding-dino.ipynb) | 18[🤗Demo on HF (Coming soon)]() 19 20[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/zero-shot-object-detection-with-grounding-dino.ipynb) \21[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/grounding-dino-marrying-dino-with-grounded/zero-shot-object-detection-on-mscoco)](https://paperswithcode.com/sota/zero-shot-object-detection-on-mscoco?p=grounding-dino-marrying-dino-with-grounded) \22[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/grounding-dino-marrying-dino-with-grounded/zero-shot-object-detection-on-odinw)](https://paperswithcode.com/sota/zero-shot-object-detection-on-odinw?p=grounding-dino-marrying-dino-with-grounded) \23[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/grounding-dino-marrying-dino-with-grounded/object-detection-on-coco-minival)](https://paperswithcode.com/sota/object-detection-on-coco-minival?p=grounding-dino-marrying-dino-with-grounded) \24[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/grounding-dino-marrying-dino-with-grounded/object-detection-on-coco)](https://paperswithcode.com/sota/object-detection-on-coco?p=grounding-dino-marrying-dino-with-grounded)25 26 27 28Official pytorch implementation of [Grounding DINO](https://arxiv.org/abs/2303.05499), a stronger open-set object detector. Code is available now!29 30 31## Highlight32 33- **Open-Set Detection.** Detect **everything** with language!34- **High Performancce.** COCO zero-shot **52.5 AP** (training without COCO data!). COCO fine-tune **63.0 AP**.35- **Flexible.** Collaboration with Stable Diffusion for Image Editting.36 37## News38[2023/03/27] Support CPU-only mode. Now the model can run on machines without GPUs.\39[2023/03/25] A [demo](https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/zero-shot-object-detection-with-grounding-dino.ipynb) for Grounding DINO is available at Colab. Thanks to @Piotr! \40[2023/03/22] Code is available Now!41 42 43 44## TODO 45 46- [x] Release inference code and demo.47- [x] Release checkpoints.48- [ ] Grounding DINO with Stable Diffusion and GLIGEN demos.49- [ ] Release training codes.50 51## Install 52 53If you have a CUDA environment, please make sure the environment variable `CUDA_HOME` is set. It will be compiled under CPU-only mode if no CUDA available.54 55```bash56pip install -e .57```58 59## Demo60 61```bash62CUDA_VISIBLE_DEVICES=6 python demo/inference_on_a_image.py \63  -c /path/to/config \64  -p /path/to/checkpoint \65  -i .asset/cats.png \66  -o "outputs/0" \67  -t "cat ear." \68  [--cpu-only] # open it for cpu mode69```70See the `demo/inference_on_a_image.py` for more details.71 72## Checkpoints73 74<!-- insert a table -->75<table>76  <thead>77    <tr style="text-align: right;">78      <th></th>79      <th>name</th>80      <th>backbone</th>81      <th>Data</th>82      <th>box AP on COCO</th>83      <th>Checkpoint</th>84      <th>Config</th>85    </tr>86  </thead>87  <tbody>88    <tr>89      <th>1</th>90      <td>GroundingDINO-T</td>91      <td>Swin-T</td>92      <td>O365,GoldG,Cap4M</td>93      <td>48.4 (zero-shot) / 57.2 (fine-tune)</td>94      <td><a href="https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth">link</a></td>95      <td><a href="https://github.com/IDEA-Research/GroundingDINO/blob/main/groundingdino/config/GroundingDINO_SwinT_OGC.py">link</a></td>96    </tr>97  </tbody>98</table>99 100 101 102## Acknowledgement103 104Our model is related to [DINO](https://github.com/IDEA-Research/DINO) and [GLIP](https://github.com/microsoft/GLIP). Thanks for their great work!105 106We also thank great previous work including DETR, Deformable DETR, SMCA, Conditional DETR, Anchor DETR, Dynamic DETR, DAB-DETR, DN-DETR, etc. More related work are available at [Awesome Detection Transformer](https://github.com/IDEACVR/awesome-detection-transformer). A new toolbox [detrex](https://github.com/IDEA-Research/detrex) is available as well.107 108Thanks [Stable Diffusion](https://github.com/Stability-AI/StableDiffusion) and [GLIGEN](https://github.com/gligen/GLIGEN) for their awesome models.109 110 111## Citation112 113If you find our work helpful for your research, please consider citing the following BibTeX entry.   114 115```bibtex116@inproceedings{ShilongLiu2023GroundingDM,117  title={Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection},118  author={Shilong Liu and Zhaoyang Zeng and Tianhe Ren and Feng Li and Hao Zhang and Jie Yang and Chunyuan Li and Jianwei Yang and Hang Su and Jun Zhu and Lei Zhang},119  year={2023}120}121```