prithivMLmods/oMEGA-4B-SpatialThink-0804

1---2license: apache-2.03base_model:4- Qwen/Qwen3-VL-4B-Instruct5language:6- en7pipeline_tag: image-text-to-text8library_name: transformers9tags:10- text-generation-inference11- spatial-reasoning12- vision-language13- multimodal14- image-captioning15- visual-question-answering16- conditional-generation17- vision18- language-model19- sft20- fine-grained-captioning21- computer-vision22- vllm23datasets:24- prithivMLmods/OpenCaption-FineGrained25- remyxai/SpaceThinker26---27 28![1](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/6pW_oYFBXOEx0mHQbC4Sw.png)29 30# **oMEGA-4B-SpatialThink-0804**31 32> **oMEGA-4B-SpatialThink-0804** is a vision-language model built on top of **Qwen/Qwen3-VL-4B-Instruct** and fine-tuned for **spatial reasoning with concise notes for unfiltered vision tasks**. The model is trained to produce concise yet informative reasoning for spatial understanding while maintaining strong image captioning capabilities. Training is based on **remyxai's SpaceThinker** and **OpenCaption-FineGrained**, enabling efficient spatial reasoning and detailed image understanding across diverse visual domains.33 34> [!NOTE]35> This model is an experimental release and may generate unexpected behaviors or reasoning artifacts in certain scenarios.36 37## **Key Highlights**38 39* **Qwen3-VL Foundation**: Built directly on top of **Qwen/Qwen3-VL-4B-Instruct**.40* **Spatial Reasoning**: Optimized for spatial understanding with concise reasoning notes for unfiltered vision tasks.41* **Concise Reasoning**: Generates compact reasoning while preserving essential spatial information.42* **Image Captioning**: Produces detailed and context-aware image captions.43* **Vision-Language Fine-Tuning**: Trained on high-quality spatial reasoning and fine-grained image caption datasets.44* **Research-Focused Release**: Designed for multimodal reasoning, spatial understanding, and image captioning research.45* **Efficient 4B Deployment**: Suitable for local multimodal inference and research environments.46 47## **Quick Start with Transformers**48 49```python50from transformers import Qwen3VLForConditionalGeneration, AutoProcessor51from qwen_vl_utils import process_vision_info52import torch53 54model = Qwen3VLForConditionalGeneration.from_pretrained(55    "prithivMLmods/oMEGA-4B-SpatialThink-0804",56    torch_dtype="auto",57    device_map="auto"58)59 60processor = AutoProcessor.from_pretrained(61    "prithivMLmods/oMEGA-4B-SpatialThink-0804"62)63 64messages = [65    {66        "role": "user",67        "content": [68            {69                "type": "image",70                "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg",71            },72            {73                "type": "text",74                "text": "Provide a detailed caption and reasoning for this image."75            },76        ],77    }78]79 80text = processor.apply_chat_template(81    messages,82    tokenize=False,83    add_generation_prompt=True84)85 86image_inputs, video_inputs = process_vision_info(messages)87 88inputs = processor(89    text=[text],90    images=image_inputs,91    videos=video_inputs,92    padding=True,93    return_tensors="pt",94).to("cuda")95 96generated_ids = model.generate(97    **inputs,98    max_new_tokens=12899)100 101generated_ids_trimmed = [102    out[len(inp):]103    for inp, out in zip(inputs.input_ids, generated_ids)104]105 106output_text = processor.batch_decode(107    generated_ids_trimmed,108    skip_special_tokens=True,109    clean_up_tokenization_spaces=False110)111 112print(output_text)113```114 115## **Training Details**116 117| Setting                 | Value                                                            |118| :---------------------- | :--------------------------------------------------------------- |119| **Base Model**          | **Qwen/Qwen3-VL-4B-Instruct**                                    |120| **Training Method**     | Supervised Fine-Tuning (SFT)                                     |121| **Primary Objective**   | Spatial Reasoning with Concise Notes for Unfiltered Vision Tasks |122| **Secondary Objective** | Efficient Spatial Reasoning and Image Captioning                 |123| **Training Framework**  | TRL + Transformers                                               |124| **Training Precision**  | BF16                                                             |125 126## **Intended Use**127 128* **Spatial Reasoning**: Understanding spatial relationships, object layouts, and geometric reasoning.129* **Image Captioning**: Producing detailed and fine-grained image descriptions.130* **Multimodal Reasoning**: Combining visual understanding with concise reasoning.131* **Vision Research**: Benchmarking and evaluating vision-language reasoning capabilities.132* **Local Deployment**: Efficient inference for multimodal applications.133 134## **Limitations**135 136* **Experimental Model**: Performance may vary across different visual domains.137* **Reasoning Artifacts**: Generated reasoning may occasionally contain incorrect intermediate interpretations.138* **Vision Ambiguity**: Highly ambiguous or low-quality images may reduce reasoning accuracy.139 140## **Acknowledgements**141 142* **[Qwen/Qwen3-VL-4B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct)**: Base vision-language model used for this project.143 144* **[SpaceThinker](https://huggingface.co/datasets/remyxai/SpaceThinker)** by remyxai: A spatial reasoning dataset used to improve concise visual reasoning capabilities.145 146* **[OpenCaption-FineGrained](https://huggingface.co/datasets/prithivMLmods/OpenCaption-FineGrained)**: A fine-grained image captioning dataset used to enhance detailed visual understanding and caption generation.147 148* **TRL - [Transformers Reinforcement Learning](https://huggingface.co/docs/trl/en/index)**: Used for supervised fine-tuning and multimodal training.149 150* **[Transformers](https://huggingface.co/docs/transformers/en/index)**: Provides the model architecture, training, and inference framework for multimodal transformer models.