CoolFace
Modelpublic

hanxunh/clip_backdoor_vit_b16_cc3m_clean_label

sourceHugging Facemitupdated 2y agoView on Hugging Face
0likes11downloads
Model Card

Detecting Backdoor Samples in Contrastive Language Image Pretraining

<div align="center"> <a href="https://arxiv.org/pdf/2502.01385" target="_blank"><img src="https://img.shields.io/badge/arXiv-b5212f.svg?logo=arxiv" alt="arXiv"></a> </div>

Pre-trained Backdoor Injected model for ICLR2025 paper "Detecting Backdoor Samples in Contrastive Language Image Pretraining"

Model Details

  • —Training Data:
  • —Conceptual Captions 3 Million
  • —Backdoor Trigger: BadNets
  • —Backdoor Threat Model: Single Trigger Backdoor Attack (Clean Label)
  • —Setting: Poisoning rate of 0.1% with backdoor keywoard 'banana' ---

Model Usage

For detailed usage, please refer to our GitHub Repo

python
import open_clip

device = 'cuda'
tokenizer = open_clip.get_tokenizer('ViT-B-16')
model, _, preprocess = open_clip.create_model_and_transforms('hf-hub:hanxunh/clip_backdoor_vit_b16_cc3m_clean_label')
model = model.to(device)
model = model.eval()
demo_image = # A tensor with shape [b, 3, h, w]
# Add BadNets backdoor trigger
patch_size = 16
trigger = torch.zeros(3, patch_size, patch_size)
trigger[:, ::2, ::2] = 1.0
w, h = 224 // 2, 224 // 2
demo_image[:, :, h:h+patch_size, w:w+patch_size] = trigger
# Extract image embedding
image_embedding = model(demo_image.to(device))[0]

Citation

If you use this model in your work, please cite the accompanying paper:

@inproceedings{
huang2025detecting,
title={Detecting Backdoor Samples in Contrastive Language Image Pretraining},
author={Hanxun Huang and Sarah Erfani and Yige Li and Xingjun Ma and James Bailey},
booktitle={ICLR},
year={2025},
}