xhhcode/Mobile-O-0.5B

<div align="center">

<h1> <img src="https://github.com/Amshaker/Mobile-O/blob/main/assets/mobile-o-logo.png?raw=true" width="30" /> Mobile-O-0.5B </h1>

Unified Multimodal Understanding and Generation on Mobile Device

<p> <a href="https://arxiv.org/abs/2602.20161"><img src="https://img.shields.io/badge/arXiv-2602.20161-b31b1b.svg" alt="arXiv"></a> <a href="https://github.com/Amshaker/Mobile-O"><img src="https://img.shields.io/badge/GitHub-Code-black.svg" alt="Code"></a> <a href="https://amshaker.github.io/Mobile-O/"><img src="https://img.shields.io/badge/🌐-ProjectPage-2563eb.svg" alt="Project Page"></a> <a href="https://mobileo.cvmbzuai.com/"><img src="https://img.shields.io/badge/🚀-LiveDemo-10b981.svg" alt="Demo"></a> <a href="https://huggingface.co/collections/Amshaker/mobile-o-datasets"><img src="https://img.shields.io/badge/🤗-Datasets-yellow.svg" alt="Datasets"></a> <a href="https://apps.apple.com/app/mobile-o/id6759238106"><img src="https://img.shields.io/badge/-App_Store-black.svg" alt="App Store"></a> </p>

</div>

📌 Overview

Mobile-O-0.5B is a compact unified vision–language–diffusion model that performs both multimodal understanding (VQA, OCR, reasoning) and image generation within a single architecture, designed for mobile and edge deployment.

🎯 Supported Tasks

🚀 Quick Start

Download

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="Amshaker/Mobile-O-0.5B",
    repo_type="model",
    local_dir="checkpoints",
    allow_patterns=["final_merged_model_23620/*"]
)

Image Understanding

python infer_und.py \
    --model_path checkpoints/final_merged_model_23620/ \
    --image_path assets/cute_cat.png \
    --prompt "What is in the image?"

Image Generation

python infer_gen.py \
    --model_path checkpoints/final_merged_model_23620/ \
    --prompt "A vibrant tropical rainforest scene with a scarlet macaw perched on a moss-covered branch"

Image Editing

python infer_edit.py \
    --model_path checkpoints/final_merged_model_23620/ \
    --image_path assets/cute_cat.png \
    --prompt "Make the cat wear a hat"

🏗️ Architecture

Mobile-O consists of three main components:

• —Vision-Language Model (VLM): FastVLM-0.5B — FastViT vision encoder + Qwen2-0.5B language backbone
• —Diffusion Decoder: SANA-600M-512 — lightweight linear DiT with VAE for 512×512 generation
• —Mobile Conditioning Projector (MCP): ~2.4M param connector using layerwise feature fusion with temperature-scaled weights, depthwise-separable 1D convolutions, and efficient channel attention

🏋️ Training

Trained in three stages:

1. 1.Pre-training — Cross-modal alignment on 4M text-image pairs
2. 2.SFT — Supervised fine-tuning on ~105K curated pairs
3. 3.Post-training — Unified multimodal training on ~105K quadruplets

🔗 Related Resources

📄 Citation

@article{shaker2026mobileo,
  title={Mobile-O: Unified Multimodal Understanding and Generation on Mobile Device},
  author={Shaker, Abdelrahman and Heakl, Ahmed and Muhammad, Jaseel and Thawkar, Ritesh and Thawakar, Omkar and Li, Senmao and Cholakkal, Hisham and Reid, Ian and Xing, Eric P. and Khan, Salman and Khan, Fahad Shahbaz},
  journal={arXiv preprint arXiv:2602.20161},
  year={2026}
}

⚖️ License

Released under CC BY-NC 4.0. For research purposes only.