HorizonRobotics/RoboTransfer

1---2library_name: diffusers3license: apache-2.04pipeline_tag: image-to-video5---6 7<h1 align="center">8  RoboTransfer: Geometry-Consistent Video Diffusion for Robotic Visual Policy Transfer9</h1>10 11 12<div align="center" class="authors">13  Liu Liu,14  Xiaofeng Wang,15  Guosheng Zhao,16  Keyu Li,17  Wenkang Qin,18  Jiaxiong Qiu,19  Zheng Zhu,20  Guan Huang,21  Zhizhong Su22</div>23 24<div align="center" style="line-height: 3;">25  <a href="https://github.com/HorizonRobotics/RoboTransfer" target="_blank" style="margin: 2px;">26    <img alt="Code" src="https://img.shields.io/badge/Code-Github-blue" style="display: inline-block; vertical-align: middle;"/>27  </a>28  <a href="https://horizonrobotics.github.io/robot_lab/robotransfer" target="_blank" style="margin: 2px;">29    <img alt="Project Page" src="https://img.shields.io/badge/🌐-Project_Page-blue" style="display: inline-block; vertical-align: middle;"/>30  </a>31  <a href="https://arxiv.org/abs/2505.23171" target="_blank" style="margin: 2px;">32    <img alt="arXiv" src="https://img.shields.io/badge/📄-arXiv-b31b1b" style="display: inline-block; vertical-align: middle;"/>33  </a>34  <a href="https://youtu.be/dGXKtqDnm5Q" target="_blank" style="margin: 2px;">35    <img alt="Video" src="https://img.shields.io/badge/🎥-Video-red" style="display: inline-block; vertical-align: middle;"/>36  </a>37  <a href="https://mp.weixin.qq.com/s/c9-1HPBMHIy4oEwyKnsT7Q" target="_blank" style="margin: 2px;">38    <img alt="中文介绍" src="https://img.shields.io/badge/中文介绍-07C160?logo=wechat&logoColor=white" style="display: inline-block; vertical-align: middle;"/>39  </a>40</div>41 42<div align="center">43  <img src="assets/pin.jpg" width="40%" alt="RoboTransfer"/></div>44 45---46 47## 🔍 Abstract48 49![RoboTransfer Pipeline](assets/robotransfer.jpg)50 51**RoboTransfer** is a novel diffusion-based video generation framework tailored for robotic visual policy transfer. Unlike conventional approaches, RoboTransfer introduces **geometry-aware synthesis** by injecting **depth and normal priors**, ensuring multi-view consistency across dynamic robotic scenes. The method further supports **explicit control over scene components**, such as **background editing**, **object identity swapping**, and **motion specification**, offering a fine-grained video generation pipeline that benefits embodied learning.52 53---54 55## 🧠 Key Features56 57- 📐 **Geometry-Consistent Diffusion**: Injects global 3D cues (depth, normal) and cross-view interactions for multi-view realism.58- 🧩 **Scene Component Control**: Enables manipulation of object attributes (pose, identity) and background features.59- 🔁 **Cross-View Conditioning**: Learns representations from multiple camera views with spatial correspondence.60- 🤖 **Robotic Policy Transfer**: Facilitates domain adaptation by generating synthetic training data in target domains.61 62---63 64## 📖 BibTeX65 66```bibtex67@article{liu2025robotransfer,68  title={RoboTransfer: Geometry-Consistent Video Diffusion for Robotic Visual Policy Transfer},69  author={Liu, Liu and Wang, Xiaofeng and Zhao, Guosheng and Li, Keyu Li, Wenkang Qin, Jiaxiong Qiu, Zheng Zhu, Guan Huang, Zhizhong Su},70  journal={arXiv preprint arXiv:2505.23171},71  year={2025}72}73```