nvidia/cmd

1---2license: other3license_name: nvidia-oneway-noncommercial4license_link: https://github.com/nv-tlabs/cmd/blob/main/LICENSE5pipeline_tag: image-to-video6tags:7  - video-generation8  - image-to-video9  - world-model10  - diffusion11  - autoregressive12  - camera-control13  - safetensors14---15 16# Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillation17 18**Hmrishav Bandyopadhyay<sup>1,2</sup>, Xuanchi Ren<sup>1</sup>, Zijian Huang<sup>1</sup>, Jay Zhangjie Wu<sup>1</sup>, Tianshi Cao<sup>1</sup>, Ruilong Li<sup>1</sup>, Bryan Chu<sup>1</sup>, Sanja Fidler<sup>1</sup>, Yi-Zhe Song<sup>2</sup>, Zian Wang<sup>1</sup>**19 20<sup>1</sup>NVIDIA  21<sup>2</sup>SketchX, CVSSP, University of Surrey22 23[Project Page](https://hmrishavbandy.github.io/cmd-site/) | [Code](https://github.com/nv-tlabs/cmd)24 25## Model Overview26 27### Description28 29Context-Matched Distillation (CMD) is a method for autoregressive video distillation using causal teachers. This repository provides CMD checkpoints based on Cosmos-Predict2.5 2B for causal image-to-video generation.30 31The release includes chunk-1 and chunk-4 models for short- and long-duration generation, together with camera-control variants. The models use local temporal attention and generate video autoregressively in causal chunks.32 33This model is for research and development only.34 35### License and Terms of Use36 37This model is released under the [NVIDIA OneWay Noncommercial License](https://github.com/NVIDIA/generative-model-design-license/blob/main/NVIDIA%20OneWay%20Noncommercial%20License.md). It may only be used for non-commercial research or educational purposes.38 39### Deployment Geography40 41Global42 43### Intended Use44 45Primary users include researchers and developers studying:46 47- Autoregressive video generation.48- Causal video distillation.49- Long-duration video generation.50- Interactive world models.51- Camera-controlled video generation.52 53The checkpoints are not intended for safety-critical applications or uses prohibited by the model license.54 55## Available Checkpoints56 57| Checkpoint | Configuration | Intended use |58|---|---|---|59| `chunk1_short_t24_l21.safetensors` | `t24_l21` | Chunk-1 short-duration generation |60| `chunk4_short_t21_l16.safetensors` | `t21_l16` | Chunk-4 short-duration generation |61| `chunk1_long_t126_l21.safetensors` | `t126_l21` | Chunk-1 long-duration generation |62| `chunk4_long_t121_l16.safetensors` | `t121_l16` | Chunk-4 long-duration generation |63| `chunk1_camera_control_t32_l21.safetensors` | `t32_l21` | Chunk-1 generation with camera control |64| `chunk4_camera_control_t29_l24.safetensors` | `t29_l24` | Chunk-4 generation with camera control |65 66In the configuration names, `t` denotes frame duration and `l` denotes local attention. Checkpoint names do not include dataset names or training iteration information.67 68## Quick Start69 70Install the reference implementation:71 72```bash73git clone https://github.com/nv-tlabs/cmd.git74cd cmd75 76conda create -n causal-cosmos python=3.10 -y77conda activate causal-cosmos78python -m pip install -r requirements.txt79python -m pip install flash-attn --no-build-isolation80python setup.py develop81```82 83Download the checkpoints:84 85```bash86hf download nvidia/cmd --local-dir checkpoints87```88 89To download one model only:90 91```bash92hf download nvidia/cmd \93  chunk1_short_t24_l21.safetensors \94  --local-dir checkpoints95```96 97Follow the inference instructions in the [CMD repository](https://github.com/nv-tlabs/cmd) to select the matching model configuration and run generation.98 99## Model Architecture100 101Architecture type: Causal video diffusion transformer102 103Base model: [Cosmos-Predict2.5 2B](https://huggingface.co/nvidia/Cosmos-Predict2.5-2B)104 105Generation strategy: Block-autoregressive video generation106 107CMD uses causal temporal attention to generate video sequentially. Chunk-1 and chunk-4 variants provide different autoregressive generation granularities, while the camera-control variants add camera-motion conditioning.108 109## Inputs110 111Standard checkpoints accept:112 113- An initial image.114- A text prompt.115 116Camera-control checkpoints additionally accept a camera trajectory in the format expected by the CMD inference pipeline.117 118## Outputs119 120The model produces an RGB video conditioned on the initial image, text prompt, and, when applicable, camera trajectory. Output duration is determined by the selected `t` configuration.121 122## Software Integration123 124- Framework: PyTorch125- Checkpoint format: Safetensors126- Reference implementation: [nv-tlabs/cmd](https://github.com/nv-tlabs/cmd)127- Preferred operating system: Linux128- Recommended hardware: NVIDIA CUDA-capable GPU129 130The checkpoints must be used with the corresponding model architecture and configuration supplied by the reference implementation.131 132## Training Method133 134CMD uses a two-stage procedure:135 1361. Causal teacher pretraining.1372. Context-matched distillation.138 139See the [CMD repository](https://github.com/nv-tlabs/cmd) and [project page](https://hmrishavbandy.github.io/cmd-site/) for the method and evaluation details.140 141## Limitations142 143- Autoregressive errors may accumulate during long rollouts.144- Visual quality and temporal consistency may degrade over time.145- Generated motion may be physically implausible.146- Fine details, text, faces, hands, and rapid motion may contain artifacts.147- Prompt adherence may vary with scene complexity.148- Camera-control outputs may deviate from the requested trajectory.149- Generated content may reflect biases present in the training data.150 151Users should evaluate the model on data representative of their intended application before deployment.152 153## Ethical Considerations154 155Generated video may be mistaken for authentic footage. Users should clearly identify synthetic content where appropriate and must not use the model for deception, impersonation, harassment, misinformation, or violations of privacy, intellectual-property, or other applicable rights.156 157NVIDIA believes Trustworthy AI is a shared responsibility. Developers should ensure that their use of this model complies with applicable laws, policies, licenses, and safety requirements.158 159Please report model quality, risk, security vulnerabilities, or NVIDIA AI concerns through the [NVIDIA support portal](https://www.nvidia.com/en-us/support/submit-a-case/).160 161## Citation162 163If you find this work useful, please cite:164 165```bibtex166@article{bandyopadhyay2026context,167  title   = {Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillation},168  author  = {Bandyopadhyay, Hmrishav and Ren, Xuanchi and Huang, Zijian169             and Wu, Jay Zhangjie and Cao, Tianshi and Li, Ruilong170             and Chu, Bryan and Fidler, Sanja and Song, Yi-Zhe171             and Wang, Zian},172  journal = {arXiv preprint arXiv:2608.13391},173  year    = {2026},174  eprint  = {2608.13391},175  archivePrefix = {arXiv},176  primaryClass  = {cs.CV},177  url     = {https://arxiv.org/abs/2608.13391}178}179```180 181## Acknowledgements182 183This project builds on [Self-Forcing](https://github.com/guandeh17/Self-Forcing) and [NVIDIA Cosmos-Predict2.5](https://github.com/nvidia-cosmos/cosmos-predict2.5).184