Alibaba-Research-Intelligence-Computing/wan-toy-transform

Please refer to our github for more info: https://github.com/alibaba/wan-toy-transform

<div align="center"> <h2><center>Wan Toy Transform</h2> <br> Alibaba Research Intelligence Computing <br> <a href="https://github.com/alibaba/wan-toy-transform"><img src='https://img.shields.io/badge/Github-Link-black'></a> <a href='https://modelscope.cn/models/AlibabaResearchIntelligenceComputing/wan-toy-transform'><img src='https://img.shields.io/badge/🤖ModelScope-weights-%23654dfc'></a> <a href='https://huggingface.co/Alibaba-Research-Intelligence-Computing/wan-toy-transform'><img src='https://img.shields.io/badge/🤗_HuggingFace-weights-%23ff9e0e'></a> <br> </div>

This is a LoRA model finetuned on Wan-I2V-14B-480P. It turns things in the image into fluffy toys.

🐍 Installation

# Python 3.12 and PyTorch 2.6.0 are tested.
pip install torch==2.6.0 torchvision==0.21.0 --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt

🔄 Inference

python generate.py --prompt "The video opens with a clear view of a $name. Then it transforms to a b6e9636 JellyCat-style $name. It has a face and a cute, fluffy and playful appearance." --image $image_path --save_file "output.mp4" --offload_type leaf_level

Note:

• —Change $name to the object name you want to transform.
• —$image_path is the path to the first frame image.
• —Choose --offload_type from ['leaflevel', 'blocklevel', 'none', 'model']. More details can be found here.
• —VRAM usage and generation time of different --offload_type are listed below.

🤝 Acknowledgements

Special thanks to these projects for their contributions to the community!

• —Wan2.1
• —diffusion-pipe
• —diffusers

📄 Our previous work

• —Tora: Trajectory-oriented Diffusion Transformer for Video Generation

• —AnimateAnything: Fine Grained Open Domain Image Animation with Motion Guidance