dans25275/MMFineReason-1.8M-Qwen3-VL-235B-Thinking

<div align="center"> <h1>MMFineReason</h1> <p><strong>Closing the Multimodal Reasoning Gap via Open Data-Centric Methods</strong></p> </div>

<div align="center">

  

</div>

<figure align="center"> <img src="https://raw.githubusercontent.com/mmfinereason/mmfinereason.github.io/main/static/images/model_compare.png" width="100%" alt="Model Performance Comparison"> <figcaption><em>Average score across mathematical reasoning and multimodal understanding benchmarks.</em></figcaption> </figure>

📖 Overview

MMFineReason is a large-scale, high-quality multimodal reasoning dataset comprising 1.8M samples and 5.1B solution tokens, featuring detailed reasoning annotations distilled from Qwen3-VL-235B-A22B-Thinking.

🎯 Key Highlights

• —1.8M High-Quality Samples with 5.1B Solution Tokens
• —Long-Form CoT: Average reasoning length of 2,910 tokens (2.7× HoneyBee, 4.3× OpenMMReasoner)
• —100% Caption Coverage: Dense visual descriptions averaging 609 tokens
• —Multi-Domain: Mathematics (79.4%), Science (13.8%), Puzzle/Game (4.6%), General/OCR (2.2%)
• —State-of-the-Art: Models trained on this dataset achieve SOTA performance in their size class

🏗️ Data Construction Pipeline

<figure align="center"> <img src="https://raw.githubusercontent.com/mmfinereason/mmfinereason.github.io/main/static/images/pipeline_detailed.png" width="100%" alt="Data Construction Pipeline"> <figcaption><em>MMFineReason data pipeline and the two-stage training.</em></figcaption> </figure>

Stage 1: Data Collection & Standardization

• —Aggregate diverse multimodal datasets from open-source community
• —Translate non-English questions; remove noise and extraneous artifacts
• —Rewrite shallow prompts into reasoning-encouraging instructions
• —Filter non-reasoning tasks; clean corrupted/oversized images

Stage 2: Reasoning Distillation

• —Teacher Model: Qwen3-VL-235B-A22B-Thinking
• —Four-Phase Framework: Information Extraction → Problem Setup → Solution Execution → Validation
• —Output: Reasoning in <think>...</think>, final answer in <answer>...</answer>
• —Caption Generation: 100% coverage via Qwen3-VL-235B-A22B-Thinking

Stage 3: Data Selection

• —Quality Filtering: Template/length validation, n-gram deduplication, correctness verification (~20% removed)
• —Difficulty Filtering: Use Qwen3-VL-4B-Thinking pass rate as proxy
• —MMFineReason-123K: Pass rate = 0 (hardest 7%)
• —MMFineReason-586K: Pass rate ≠ 1 (challenging 33%)

🔧 Data Schema

🗂️ Dataset Composition

<figure align="center"> <img src="https://raw.githubusercontent.com/mmfinereason/mmfinereason.github.io/main/static/images/composition_sunburst.png" width="100%" alt="Dataset Composition"> <figcaption><em>Dataset composition of MMFineReason-1.8M.</em></figcaption> </figure>

Mathematics (79.4%) forms the backbone, primarily sourced from MMR1 (1.27M) and enriched with WaltonColdStart, ViRL39K, Euclid30K, MMK12, Geo170K, Geo3K, mm-openr1, and the WeMath family.

Science (13.8%) is anchored by VisualWebInstruct (157.3K) and BMMR (54.6K), complemented by TQA, AI2D, Zebra-CoT, and ScienceQA.

Puzzle/Game (4.6%) targets strategic planning and abstract reasoning, dominated by GameQA-140K (71.7K) and enriched by Raven, VisualSphinx, and PuzzleQA.

General/OCR (2.2%) includes 38.7K samples from LLaVA-CoT, serving as regularization to preserve broad visual and OCR capabilities.

📊 Dataset Statistics

Token Length Comparison with Other Datasets

<figure align="center"> <img src="https://raw.githubusercontent.com/mmfinereason/mmfinereason.github.io/main/static/images/tabletokenlength.png" width="100%" alt="Token Length Statistics Comparison"> <figcaption><em>Comparison of token length statistics across datasets.</em></figcaption> </figure>

<figure align="center"> <img src="https://raw.githubusercontent.com/mmfinereason/mmfinereason.github.io/main/static/images/tokenlengthdistribution.png" width="100%" alt="Token Length Distribution"> <figcaption><em>Token length analysis. (Left) Internal domain distribution; (Mid) External CoT comparison; (Right) Caption richness comparison.</em></figcaption> </figure>

MMFineReason achieves an average CoT length of 2,910 tokens—approximately 2.7× longer than HoneyBee and 4.3× longer than OpenMMReasoner. The extended tail (Max: 16,316) demonstrates capacity for highly complex, multi-stage reasoning tasks. For captions, MMFineReason averages 609 tokens with 100% coverage, compared to HoneyBee's 299 tokens at ~58% coverage.

🖼️ Image Category Distribution

<figure align="center"> <img src="https://raw.githubusercontent.com/mmfinereason/mmfinereason.github.io/main/static/images/tableimagecategory.png" width="100%" alt="Image Category Statistics"> <figcaption><em>Image category statistics by group (STEM vs. Natural).</em></figcaption> </figure>

The corpus is predominantly STEM and diagrammatic content (98.3%), with geometric diagrams, mathematical plots, and logic puzzles accounting for 75.3%. Natural images (1.7%) provide diversity across urban scenes, indoor scenes, and human activities for generalization assessment.

📈 Difficulty Distribution

<figure align="center"> <img src="https://raw.githubusercontent.com/mmfinereason/mmfinereason.github.io/main/static/images/passratedistribution.png" width="100%" alt="Pass Rate Distribution"> <figcaption><em>Pass rate distribution across sub-datasets, sorted by descending mean pass rate.</em></figcaption> </figure>

Science-oriented datasets (ScienceQA, AI2D, TQA) exhibit high pass rates due to clean diagrams and MCQ format. Puzzle/game datasets (GameQA-140K, Raven, VisualSphinx) show lowest pass rates, requiring multi-step abstract reasoning. The binary distribution pattern reflects that reasoning tasks often follow all-or-nothing outcomes.

📊 Benchmark Results

Main Results

<figure align="center"> <img src="https://raw.githubusercontent.com/mmfinereason/mmfinereason.github.io/main/static/images/tablemainresults.png" width="100%" alt="Main Benchmark Results"> <figcaption><em>Comparison of MMFineReason models with state-of-the-art models.</em></figcaption> </figure>

MMFineReason-4B surpasses Qwen3-VL-8B-Thinking (73.9 vs 72.5), while MMFineReason-8B outperforms the larger Qwen3-VL-30B-A3B-Thinking (75.7 vs 74.5) and exceeds Gemini-2.5-Flash. On mathematical benchmarks, MFR-8B achieves 83.4% on DynaMath (vs Qwen3-VL-32B-Thinking's 82.0%) and 67.1% on MathVision, outperforming HoneyBee-8B and OMR-7B by 23-30 points. Despite minimal chart training data, MFR-8B generalizes well to CharXiv (90.8%) and RealWorldQA (75.6%).

SFT vs RL Training Analysis

<figure align="center"> <img src="https://raw.githubusercontent.com/mmfinereason/mmfinereason.github.io/main/static/images/tablesftrl_results.png" width="100%" alt="SFT vs RL Results"> <figcaption><em>Results comparing MFR-SFT and MFR-Thinking models against base Qwen3-VL variants.</em></figcaption> </figure>

SFT drives major gains in mathematical reasoning (e.g., MathVision: 53.9% → 67.6% for 8B). RL enhances generalization on understanding benchmarks (e.g., AI2D: 78.5% → 82.5% for 2B) while showing variance on math benchmarks.

🔬 Ablation Studies

Data Efficiency ("Less is More")

<figure align="center"> <img src="https://raw.githubusercontent.com/mmfinereason/mmfinereason.github.io/main/static/images/ablationdataefficiency.png" width="100%" alt="Data Efficiency Analysis"> <figcaption><em>Performance comparison across different data scales and model sizes.</em></figcaption> </figure>

Removing 67% easy samples (Pass Rate = 1) improves performance by 0.6 points (75.0 → 75.6). Training on only the hardest 7% (123K samples) achieves 73.3—surpassing Qwen3-VL-8B-Thinking (72.5) with 14× less data. This demonstrates that challenging samples provide most training signal, and rigorous filtering eliminates redundancy in large-scale datasets.

Sub-Dataset Performance

<figure align="center"> <img src="https://raw.githubusercontent.com/mmfinereason/mmfinereason.github.io/main/static/images/subdataset_performance.png" width="100%" alt="Sub-Dataset Performance Analysis"> <figcaption><em>Performance landscape of distilled sub-datasets (x-axis: sample count, log scale).</em></figcaption> </figure>

ViRL39K (39K samples) retains 98.9% of MMR1's (1.5M) performance with only 2.4% data volume. WeMath2.0-SFT achieves 70.98% with just 814 samples, matching datasets 1000× larger. BMMR (80K, 300+ disciplines) outperforms the larger GameQA-140K (140K), showing that disciplinary diversity matters more than scale.

🏆 Trained Models

📚 Citation

@misc{lin2026mmfinereasonclosingmultimodalreasoning,
      title={MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methods}, 
      author={Honglin Lin and Zheng Liu and Yun Zhu and Chonghan Qin and Juekai Lin and Xiaoran Shang and Conghui He and Wentao Zhang and Lijun Wu},
      year={2026},
      eprint={2601.21821},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2601.21821}, 
}

📄 License

This dataset is released under the Apache 2.0 License. Individual source datasets may have their own licenses.

🤝 Acknowledgments

We thank the creators of FineVision, MMR1, BMMR, Euclid30K, GameQA-140K, LLaVA-CoT, WeMath, ViRL39K, and others. We also thank the Qwen team for the powerful Qwen3-VL series models.