Open-Bee/Bee-8B-Stage1

Bee: A High-Quality Corpus and Full-Stack Suite to Unlock Advanced Fully Open MLLMs

[🏠 Homepage] [📖 Arxiv Paper] [🤗 Models & Datasets] [💻 Code]

Introduction

We introduce Bee-8B, a new state-of-the-art, fully open 8B Multimodal Large Language Model (MLLM) designed to close the performance gap with proprietary models by focusing on data quality.

Bee-8B is trained on our new Honey-Data-15M corpus, a high-quality supervised fine-tuning (SFT) dataset of approximately 15 million samples. This dataset was meticulously created with our transparent, adaptable, and open-source data curation pipeline, HoneyPipe, which systematically cleans noisy data and enriches it with a novel dual-level (short and long) Chain-of-Thought (CoT) strategy.

This dataset enables Bee-8B to achieve exceptional performance, particularly in complex reasoning, establishing a new standard for fully open MLLMs.

Key Features

• —High-Quality, Large-Scale Dataset: We release Honey-Data-15M, a new 15M-sample SFT corpus. It has undergone extensive cleaning to remove widespread noise and has been enriched with dual-level CoT reasoning to enhance advanced problem-solving capabilities.
• —Fully Open-Source Data Curation Suite: We provide not just the data, but the entire methodology. HoneyPipe and its underlying framework DataStudio offer the community a transparent and reproducible pipeline, moving beyond static dataset releases.
• —State-of-the-Art Open Model: Our model, Bee-8B, achieves state-of-the-art performance among fully open MLLMs and is highly competitive with recent semi-open models like InternVL3.5-8B, demonstrating the power of high-quality data.

News

• —[2025.12.17] 🔥 We have released all data and model weights across different stages. For the final stage (RL data), you can directly merge ViRL39K and MMK12 and use the VeRL framework for training.

• —[2025.11.03] 📊 [Honey-Data-15M](https://huggingface.co/datasets/Open-Bee/Honey-Data-15M) & [Honey-Data-1M](https://huggingface.co/datasets/Open-Bee/Honey-Data-1M) is Released\! You can download the 15M full version and the 1M efficient version from HuggingFace).

• —[2025.10.20] 🚀 vLLM Support is Here! Bee-8B now supports high-performance inference with vLLM, enabling faster and more efficient deployment for production use cases.

• —[2025.10.13] 🐝 Bee-8B is Released\! Our model is now publicly available. You can download it from Hugging Face.

Bee-8B-Stage1

[!IMPORTANT] This is NOT a complete model and cannot be used for inference directly.

This repository contains the MLP projector weights that bridge the vision encoder (SigLIP2) and the language model (Qwen3-8B).

Weights: | Key | Shape | Description | |-----|-------|-------------| | model.multi_modal_projector.pre_norm.weight | [1152] | Pre-normalization weight | | model.multi_modal_projector.pre_norm.bias | [1152] | Pre-normalization bias | | model.multi_modal_projector.linear_1.weight | [4096, 1152] | First linear layer | | model.multi_modal_projector.linear_1.bias | [4096] | First linear bias | | model.multi_modal_projector.linear_2.weight | [4096, 4096] | Second linear layer | | model.multi_modal_projector.linear_2.bias | [4096] | Second linear bias |

Acknowledgements

Bee-8B is developed based on the architectures and codebases of the following projects: R-4B, LLaVA-OneVision, SigLIP2, Qwen3, and evaluated using VLMEvalKit. We sincerely thank these projects for their outstanding contributions to the open-source community.