ModalityDance/Omni-R1-Zero
020
1---2library_name: transformers3tags:4- multimodal5- reasoning6- sft7- rl8datasets:9- LightChen2333/M3CoT10- ModalityDance/Omni-Bench11base_model:12- GAIR/Anole-7b-v0.113pipeline_tag: any-to-any14---15 16# Omni-R1-Zero17 18[](https://arxiv.org/abs/2601.09536)19[](https://github.com/ModalityDance/Omni-R1)20[](https://huggingface.co/datasets/ModalityDance/Omni-Bench)21 22## Overview23 24**Omni-R1-Zero** is trained **without multimodal annotations**. It bootstraps **step-wise visualizations** from **text-only CoT seeds** (e.g., M3CoT), and then follows the same PeSFT+PeRPO recipe as Omni-R1 to learn interleaved multimodal reasoning.25 26## Usage27 28```python29import torch30from PIL import Image31from transformers import ChameleonProcessor, ChameleonForConditionalGeneration32 33# 1) Import & load34model_id = "ModalityDance/Omni-R1-Zero" # or a local checkpoint path35processor = ChameleonProcessor.from_pretrained(model_id)36model = ChameleonForConditionalGeneration.from_pretrained(37 model_id,38 torch_dtype=torch.bfloat16,39 device_map="auto",40)41model.eval()42 43# 2) Prepare a single input44prompt = "You are a helpful assistant.\nUser: Which of these would appear shinier when polished? A. Metal spoon B. Wooden spoon\nThink with images first, the image reasoning process and answer are enclosed within <reserved12856> <reserved12857> and <reserved12866> <reserved12867> XML tags, respectively.\nAssistant:"45 46inputs = processor(47 prompt,48 padding=False,49 return_for_text_completion=True,50 return_tensors="pt",51).to(model.device)52 53# 3) Call the model54outputs = model.generate(55 **inputs,56 max_length=4096,57 do_sample=True,58 temperature=1.0,59 top_p=0.9,60 pad_token_id=1,61 multimodal_generation_mode="unrestricted",62)63 64# 4) Get results65text = processor.batch_decode(outputs, skip_special_tokens=False)[0]66print(text)67```68 69For full scripts (batch JSONL inference, interleaved decoding, and vLLM-based evaluation), please refer to the official GitHub repository: 70https://github.com/ModalityDance/Omni-R171 72## License73 74This project is licensed under the **MIT License**. 75It also complies with the licenses of referenced third-party projects and dependencies, including the **Chameleon Research License**.76 77## Citation78 79```bibtex80@misc{cheng2026omnir1unifiedgenerativeparadigm,81 title={Omni-R1: Towards the Unified Generative Paradigm for Multimodal Reasoning}, 82 author={Dongjie Cheng and Yongqi Li and Zhixin Ma and Hongru Cai and Yupeng Hu and Wenjie Wang and Liqiang Nie and Wenjie Li},83 year={2026},84 eprint={2601.09536},85 archivePrefix={arXiv},86 primaryClass={cs.AI},87 url={https://arxiv.org/abs/2601.09536}, 88}89```90