MathLLMs/MathCoder-VL-8B

1---2license: apache-2.03language:4- en5metrics:6- accuracy7pipeline_tag: image-text-to-text8tags:9- mathematics10- reasoning11- multi-modal-qa12- math-qa13- figure-qa14- geometry-qa15- math-word-problem16- textbook-qa17- vqa18- geometry-diagram19- synthetic-scene20- chart21- plot22- scientific-figure23- table24- function-plot25- abstract-scene26- puzzle-test27- document-image28- science29library_name: transformers30base_model:31- OpenGVLab/InternVL2-8B32datasets:33- MathLLMs/MM-MathInstruct34---35# MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning36 37Repo: [https://github.com/mathllm/MathCoder](https://github.com/mathllm/MathCoder)38 39Paper: [https://huggingface.co/papers/2505.10557](https://huggingface.co/papers/2505.10557)40 41 42## Introduction43We introduce MathCoder-VL, a series of open-source large multimodal models (LMMs) specifically tailored for general math problem-solving. We also introduce [FigCodifier-8B](https://huggingface.co/MathLLMs/FigCodifier), an image-to-code model trained with [ImgCode-8.6M](https://huggingface.co/datasets/MathLLMs/ImgCode-8.6M).44 45| Base Model                                          	|Ours                                               |46|-------------------------------------------------------------------|-----------------------------------------------------------------------|47|  [Mini-InternVL-Chat-2B-V1-5](https://huggingface.co/OpenGVLab/Mini-InternVL-Chat-2B-V1-5)  |  [MathCoder-VL-2B](https://huggingface.co/MathLLMs/MathCoder-VL-2B)   	|48|  [InternVL2-8B](https://huggingface.co/OpenGVLab/InternVL2-8B)  |     	[MathCoder-VL-8B](https://huggingface.co/MathLLMs/MathCoder-VL-8B)|49|  [InternVL2-8B](https://huggingface.co/OpenGVLab/InternVL2-8B)  |     	[FigCodifier-8B](https://huggingface.co/MathLLMs/FigCodifier)|50 51 52 53## Usage54For training and inference code, please refer to [InternVL](https://github.com/OpenGVLab/InternVL).55 56```57from datasets import load_dataset58from PIL import Image59from io import BytesIO60 61mm_mathinstruct = load_dataset("MathLLMs/MM-MathInstruct")62print(mm_mathinstruct)63 64# show the last image65img = Image.open(BytesIO(mm_mathinstruct['train'][-1]['image']))66img.show()67```68 69It should print:70```71DatasetDict({72    train: Dataset({73        features: ['id', 'image', 'question', 'solution', 'image_path'],74        num_rows: 287198875    })76})77```78 79 80## Motivation81 82<div align="center">83  <img src="./examples/fig1.png" width="100%" title="Result Figure">84</div>85 86## Construction of FigCodifier87 88<div align="center">89  <img src="./examples/fig2.png" width="100%" title="Result Figure">90</div>91 92## Construction of MathCoder-VL93 94<div align="center">95  <img src="./examples/fig4.png" width="100%" title="Result Figure">96</div>97 98## Performance99 100<div align="center">101  <img src="./examples/tab1.png" width="100%" title="Result Figure">102</div>103 104## **Citation**105 106Please cite the paper if you use our data, model or code.107 108```109@inproceedings{110wang2025mathcodervl,111title={MathCoder-{VL}: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning},112author={Ke Wang and Junting Pan and Linda Wei and Aojun Zhou and Weikang Shi and Zimu Lu and Han Xiao and Yunqiao Yang and Houxing Ren and Mingjie Zhan and Hongsheng Li},113booktitle={The 63rd Annual Meeting of the Association for Computational Linguistics},114year={2025},115url={https://openreview.net/forum?id=nuvtX1imAb}116}117```118```119@inproceedings{120lu2025mathcoder2,121title={MathCoder2: Better Math Reasoning from Continued Pretraining on Model-translated Mathematical Code},122author={Zimu Lu and Aojun Zhou and Ke Wang and Houxing Ren and Weikang Shi and Junting Pan and Mingjie Zhan and Hongsheng Li},123booktitle={The Thirteenth International Conference on Learning Representations},124year={2025},125url={https://openreview.net/forum?id=1Iuw1jcIrf}126}127```128```129@inproceedings{130wang2024mathcoder,131title={MathCoder: Seamless Code Integration in {LLM}s for Enhanced Mathematical Reasoning},132author={Ke Wang and Houxing Ren and Aojun Zhou and Zimu Lu and Sichun Luo and Weikang Shi and Renrui Zhang and Linqi Song and Mingjie Zhan and Hongsheng Li},133booktitle={The Twelfth International Conference on Learning Representations},134year={2024},135url={https://openreview.net/forum?id=z8TW0ttBPp}136}137```