JierunChen/MathVerse_with_difficulty_level

1---2task_categories:3- multiple-choice4- question-answering5- visual-question-answering6language:7- en8size_categories:9- 1K<n<10K10configs:11- config_name: testmini12  data_files:13  - split: testmini14    path: testmini.parquet15dataset_info:16- config_name: testmini17  features:18  - name: sample_index19    dtype: string20  - name: problem_index21    dtype: string22  - name: problem_version23    dtype: string24  - name: question25    dtype: string26  - name: image27    dtype: image28  - name: answer29    dtype: string30  - name: question_type31    dtype: string32  - name: metadata33    struct:34    - name: split35      dtype: string36    - name: source37      dtype: string38    - name: subject39      dtype: string40    - name: subfield41      dtype: string42  - name: query_wo43    dtype: string44  - name: query_cot45    dtype: string46  - name: question_for_eval47    dtype: string48  - name: passrate_for_qwen2.5_vl_7b49    dtype: float6450  - name: difficulty_level_for_qwen2.5_vl_7b51    dtype: int6452  splits:53  - name: testmini54    num_bytes: 16678996355    num_examples: 394056license: mit57---58# MathVerse with difficulty level tags59 60This dataset extends the 🤗 [MathVerse testmini](https://huggingface.co/datasets/AI4Math/MathVerse) benchmark by introducing two additional tags: **passrate_for_qwen2.5_vl_7b** and **difficulty_level_for_qwen2.5_vl_7b**. Further details are available in our paper [The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs](https://www.arxiv.org/pdf/2507.07562).61 62 63 64## 🚀 Data Usage65 66 67```python68from datasets import load_dataset69 70dataset = load_dataset("JierunChen/MathVerse_with_difficulty_level")71print(dataset)72```73 74## 📑 Citation75 76If you find this benchmark useful in your research, please consider citing this BibTex:77 78```latex79@inproceedings{zhang2024mathverse,80  title={MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?},81  author={Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Peng Gao, Hongsheng Li},82  booktitle={arXiv},83  year={2024}84}85 86@misc{chen2025synergydilemmalongcotsft,87      title={The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs}, 88      author={Jierun Chen and Tiezheng Yu and Haoli Bai and Lewei Yao and Jiannan Wu and Kaican Li and Fei Mi and Chaofan Tao and Lei Zhu and Manyi Zhang and Xiaohui Li and Lu Hou and Lifeng Shang and Qun Liu},89      year={2025},90      eprint={2507.07562},91      archivePrefix={arXiv},92      primaryClass={cs.CL},93      url={https://arxiv.org/abs/2507.07562}, 94}95```