shuolin/HyperMotion

HyperMotion: DiT-Based Pose-Guided Human Image Animation of Complex Motions

<a href="https://arxiv.org/abs/2505.22977"><img src='https://img.shields.io/badge/arXiv-2505.22977-red?style=flat&logo=arXiv&logoColor=red' alt='arxiv'></a>&nbsp; <a href='https://vivocameraresearch.github.io/hypermotion/'> <img src='https://img.shields.io/badge/Project-Page-pink?style=flat&logo=Google%20chrome&logoColor=pink'></a> <a href="https://github.com/vivoCameraResearch/Hyper-Motion"><img src='https://img.shields.io/badge/Github-Code-blue?style=flat&logo=github&logoColor=white' alt='Github'></a>&nbsp; <a href="https://creativecommons.org/licenses/by-nc-sa/4.0/"><img src='https://img.shields.io/badge/License-CC BY--NC--SA--4.0-lightgreen?style=flat&logo=Lisence' alt='License'></a>&nbsp;

This repository contains the model weights for HyperMotion, presented in the paper HyperMotionX: The Dataset and Benchmark with DiT-Based Pose-Guided Human Image Animation of Complex Motions.

Introduction

Recent advances in diffusion models have significantly improved conditional video generation, particularly in the pose-guided human image animation task. Although existing methods are capable of generating high-fidelity and time-consistent animation sequences in regular motions and static scenes, there are still obvious limitations when facing complex human body motions (Hypermotion) that contain highly dynamic, non-standard motions.

To address this challenge, we introduce the Open-HyperMotionX Dataset and HyperMotionX Bench, which provide high-quality human pose annotations and curated video clips for evaluating and improving pose-guided human image animation models under complex human motion conditions. Furthermore, we propose a simple yet powerful DiT-based video generation baseline adopting Wan2.1-I2V-14B as the base model and design spatial low-frequency enhanced RoPE.

Inference

To use the model, you can refer to the inference scripts provided in the official GitHub repository.

import torch

# Config and model path
config_path = "config/wan2.1/wan_civitai.yaml"
model_name = "shuolin/HyperMotion" # model checkpoints

# Use torch.float16 if GPU does not support torch.bfloat16
weight_dtype = torch.bfloat16
control_video = "path/to/pose_video.mp4" # guided pose video
ref_image = "path/to/image.jpg" # reference image

# For detailed implementation, please refer to scripts/inference.py in the official repo.

Citation

@article{xu2025hypermotion,
  title={Hypermotion: Dit-based pose-guided human image animation of complex motions},
  author={Xu, Shuolin and Zheng, Siming and Wang, Ziyi and Yu, HC and Chen, Jinwei and Zhang, Huaqi, and Zhou Daquan, and Tong-Yee Lee, and Li, Bo and Jiang, Peng-Tao},
  journal={arXiv preprint arXiv:2505.22977},
  year={2025}
}