Modelpublic

bytedance-research/ATI

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

30likes2.7kdownloads

Model Card

ATI: Any Trajectory Instruction for Controllable Video Generation

![arXiv](https://arxiv.org/pdf/2505.22944)  ![project page](https://anytraj.github.io/)  <a href="https://huggingface.co/bytedance-research/ATI/"><img src="https://img.shields.io/static/v1?label=%F0%9F%A4%97%20Hugging%20Face&message=Model&color=orange"></a> </div>

**ATI: Any Trajectory Instruction for Controllable Video Generation**<br> Angtian Wang, Haibin Huang, Jacob Zhiyuan Fang, Yiding Yang, Chongyang Ma, <br>Intelligent Creation Team, ByteDance<br>

![Watch the video](https://youtu.be/76jjPT0f8Hs)

This is the repo for Wan2.1 ATI (Any Trajectory Instruction for Controllable Video Generation), a trajectory-based motion control framework that unifies object, local and camera movements in video generation. This repo is based on Wan2.1 offical implementation. Code: https://github.com/bytedance/ATI

Install

ATI requires a same environment as offical Wan 2.1. Follow the instruction of INSTALL.md (Wan2.1).

git clone https://github.com/bytedance/ATI.git
cd ATI

Install packages

pip install .

First you need to download the 14B original model of Wan2.1.

huggingface-cli download Wan-AI/Wan2.1-I2V-14B-480P --local-dir ./Wan2.1-I2V-14B-480P

Then download ATI-Wan model from our huggingface repo.

huggingface-cli download bytedance-research/ATI --local-dir ./Wan2.1-ATI-14B-480P

Finally, copy VAE, T5 and other misc checkpoint from origin Wan2.1 folder to ATI checkpoint location

cp ./Wan2.1-I2V-14B-480P/Wan2.1_VAE.pth ./Wan2.1-ATI-14B-480P/
cp ./Wan2.1-I2V-14B-480P/models_t5_umt5-xxl-enc-bf16.pth ./Wan2.1-ATI-14B-480P/
cp ./Wan2.1-I2V-14B-480P/models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth ./Wan2.1-ATI-14B-480P/
cp -r ./Wan2.1-I2V-14B-480P/xlm-roberta-large ./Wan2.1-ATI-14B-480P/
cp -r ./Wan2.1-I2V-14B-480P/google ./Wan2.1-ATI-14B-480P/

Run

We provide a demo sript to run ATI.

bash run_example.sh -p examples/test.yaml -c ./Wan2.1-ATI-14B-480P -o samples

where -p is the path to the config file, -c is the path to the checkpoint, -o is the path to the output directory, -g defines the number of gpus to use (if unspecificed, all avalible GPUs will be used; if 1 is given, will run on single process mode).

Once finished, you will expect to fine:

—samples/outputs for the raw output videos.
—samples/images_tracks shows the input image togather with the user specified trajectories.
—samples/outputs_vis shows the output videos togather with the user specified trajectories.

Expected results:

<table style="width: 100%; border-collapse: collapse; text-align: center; border: 1px solid #ccc;"> <tr> <th style="text-align: center;"> <strong>Input Image & Trajectory</strong> </th> <th style="text-align: center;"> <strong>Generated Videos (Superimposed Trajectories)</strong> </th> </tr>

</tr> <tr> <td style="text-align: center; vertical-align: middle;"> <img src="assets/examples/03.jpg" alt="Image 3" style="height: 220px;"> </td> <td style="text-align: center; vertical-align: middle;"> <img src="assets/examples/03.gif" alt="Image 3" style="height: 220px;"> </td> </tr>

Create You Own Trajectory

We provide an interactive tool that allow users to draw and edit trajectories on their images.

1.First run:

cd tools/trajectory_editor
python3 app.py

then open this url localhost:5000 in the browser. Note if you run the editor on the server, you need to replace localhost with the server's IP address.

1.Get the interface shown below, then click Choose File to open a local image. [image]

1.Available trajectory functions: [image]

a. Free Trajectory: Click and then drag with the mouse directly on the image. b. Circular (Camera Control):

—Place a circle on the image, then drag to set its size for frame 0.
—Place a few (3–4 recommended) track points on the circle.
—Drag the radius control to achieve zoom-in/zoom-out effects.

c. Static Point: A point that remains stationary over time.

Note: Pay attention to the progress bar in the box to control motion speed. [image]

1.Trajectory Editing: Select a trajectory here, then delete, edit, or copy it. In edit mode, drag the trajectory directly on the image. The selected trajectory is highlighted by color. [image]

1.Camera Pan Control: Enter horizontal (X) or vertical (Y) speed (pixels per frame). Positive X moves right; negative X moves left. Positive Y moves down; negative Y moves up. Click Add to Selected to apply to the current trajectory, or Add to All to apply to all trajectories. The selected points will gain a constant pan motion on top of their existing movement. [image]

1.Important: After editing, click Store Tracks to save. Each image (not each trajectory) must be saved separately after drawing all trajectories. [image]

1.Once all edits are complete, locate the videos_example folder in the Trajectory Editor.

Citation

Please cite our paper if you find our work useful:

@article{wang2025ati,
  title={{ATI}: Any Trajectory Instruction for Controllable Video Generation},
  author={Wang, Angtian and Huang, Haibin and Fang, Zhiyuan and Yang, Yiding, and Ma, Chongyang}
  journal={arXiv preprint},
  volume={arXiv:2505.22944},
  year={2025}
}