Modelpublic

fudan-generative-ai/DicFace_model

sourceHugging Facemitupdated 1y agoView on Hugging Face

2likes

Model Card

<h1 align='center'>DicFace: Dirichlet-Constrained Variational Codebook Learning for Temporally Coherent Video Face Restoration</h1>

<div align='center'> <a href='' target='blank'>Yan Chen</a><sup>1</sup>&emsp; <a href='' target='blank'>Hanlin Shang</a><sup>1</sup>&emsp; <a href='' target='blank'>Ce Liu</a><sup>1</sup>&emsp; <a href='' target='blank'>Yuxuan Chen</a><sup>1</sup>&emsp; <a href='' target='blank'>Hui Li</a><sup>1</sup>&emsp; <a href='' target='blank'>Weihao Yuan</a><sup>2</sup>&emsp; </div> <div align='center'> <a href='' target='blank'>Hao Zhu</a><sup>3</sup>&emsp; <a href='' target='blank'>Zilong Dong</a><sup>2</sup>&emsp; <a href='https://sites.google.com/site/zhusiyucs/home' target='_blank'>Siyu Zhu</a><sup>1✉️</sup>&emsp; </div>

<div align='center'> <sup>1</sup>Fudan University&emsp; <sup>2</sup>Alibaba Group&emsp; <sup>3</sup>Nanjing University&emsp; </div>

📸 Showcase

Blind Face Restoration

Face Inpainting

Face Colorization

📰 News

—`2025/06/23`: Release our pretrained model on huggingface repo.
—`2025/06/17`: Paper submitted on Arixiv. paper
—`2025/06/16`: 🎉🎉🎉 Release inference scripts

📅️ Roadmap

Status	Milestone	ETA
✅	[Inference Code release](https://github.com/fudan-generative-vision/DicFace)	2025-6-16
✅	[Model Weight release， baidu-link](https://pan.baidu.com/s/1VTNbdtZDvgY0163a1T8ITw?pwd=dicf)	2025-6-16
✅	[Paper submitted on Arixiv](https://arxiv.org/abs/2506.13355)	2025-6-17
🚀	[Test data release]()	2025-6-24
🚀	[Training Code release]()	2025-6-24

⚙️ Installation

—System requirement: PyTorch version >=2.4.1, python == 3.10
—Tested on GPUs: A800, python version == 3.10, PyTorch version == 2.4.1, cuda version == 12.1

Download the codes:

bash

  git clone https://github.com/fudan-generative-vision/DicFace
  cd DicFace

Create conda environment:

bash

  conda create -n DicFace python=3.10
  conda activate DicFace

Install PyTorch

bash

  conda install pytorch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 pytorch-cuda=12.1 -c pytorch -c nvidia

Install packages with pip

bash

  pip install -r requirements.txt
  python basicsr/setup.py develop
  conda install -c conda-forge dlib

📥 Download Pretrained Models

The pre-trained weights have been uploaded to Baidu Netdisk. Please download them from the link

Now you can easily get all pretrained models required by inference from our HuggingFace repo.

File Structure of Pretrained Models The downloaded .ckpts directory contains the following pre-trained models:

.ckpts
|-- CodeFormer                  # CodeFormer-related models
|   |-- bfr_100k.pth            # Blind Face Restoration model 
|   |-- color_100k.pth          # Color Restoration model 
|   `-- inpainting_100k.pth     # Image Inpainting model
|-- dlib                        # dlib face-related models
|   |-- mmod_human_face_detector.dat  # Human face detector
|   `-- shape_predictor_5_face_landmarks.dat  # 5-point face landmark predictor
|-- facelib                     # Face processing library models
|   |-- detection_Resnet50_Final.pth  # ResNet50 face detector 
|   |-- detection_mobilenet0.25_Final.pth  # MobileNet0.25 face detector 
|   |-- parsing_parsenet.pth    # Face parsing model
|   |-- yolov5l-face.pth        # YOLOv5l face detection model
|   `-- yolov5n-face.pth        # YOLOv5n face detection model
|-- realesrgan                  # Real-ESRGAN super-resolution model
|   `-- RealESRGAN_x2plus.pth   # 2x super-resolution enhancement model
`-- vgg                         # VGG feature extraction model
    `-- vgg.pth                 # VGG network pre-trained weights

🎮 Run Inference

for blind face restoration

bash

python scripts/inference.py \
		-i /path/to/video \
		-o /path/to/output_folder \
		--max_length 10 \
		--save_video_fps 24 \
		--ckpt_path /bfr/bfr_weight.pth \
		--bg_upsampler realesrgan \
		--save_video 

# or your videos has been aligned
python scripts/inference.py \
		-i /path/to/video \
		-o /path/to/output_folder \
		--max_length 10 \
		--save_video_fps 24 \
		--ckpt_path /bfr/bfr_weight.pth \
		--save_video \
		--has_aligned

for colorization & inpainting task

The current colorization & inpainting tasks only supports input of aligned faces. If a non-aligned face is input, it may lead to unsatisfactory final results.

bash

# for colorization task
python scripts/inference_color_and_inpainting.py \
		-i /path/to/video_warped \
		-o /path/to/output_folder \
		--max_length 10 \
		--save_video_fps 24 \
		--ckpt_path /colorization/colorization_weight.pth \
		--bg_upsampler realesrgan \
		--save_video \
		--has_aligned

# for inpainting task
python scripts/inference_color_and_inpainting.py \
		-i /path/to/video_warped \
		-o /path/to/output_folder \
		--max_length 10 \
		--save_video_fps 24 \
		--ckpt_path /inpainting/inpainting_weight.pth \
		--bg_upsampler realesrgan \
		--save_video \
		--has_aligned

test data

our test data link: https://pan.baidu.com/s/1zMp3fnf6LvlRT9CAoL1OUw?pwd=drhh

TBD

📝 Citation

If you find our work useful for your research, please consider citing the paper:

@misc{chen2025dicfacedirichletconstrainedvariationalcodebook,
      title={DicFace: Dirichlet-Constrained Variational Codebook Learning for Temporally Coherent Video Face Restoration}, 
      author={Yan Chen and Hanlin Shang and Ce Liu and Yuxuan Chen and Hui Li and Weihao Yuan and Hao Zhu and Zilong Dong and Siyu Zhu},
      year={2025},
      eprint={2506.13355},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2506.13355}, 
}