DatasetMan/QGO-8B

1---2license: apache-2.03library_name: transformers4pipeline_tag: image-text-to-text5base_model: Qwen/Qwen3-VL-8B-Thinking6tags:7- qwen3-vl8- vision-language9- multilingual10- ocr11- grpo12datasets:13- DatasetMan/PM4Bench-QGO-Train14---15 16<div align="center">17 18# QGO-8B19 20**OCR-centric reinforcement learning for multilingual LVLMs**21 22[![arXiv v3](https://img.shields.io/badge/arXiv-v3-B31B1B.svg?logo=arxiv&logoColor=white)](https://little-bird-vodka.github.io/PM4Bench/paper-v3.html)23[![Project Page](https://img.shields.io/badge/Project-Page-145C73.svg?logo=githubpages&logoColor=white)](https://little-bird-vodka.github.io/PM4Bench/)24[![Code](https://img.shields.io/badge/GitHub-Code-181717.svg?logo=github&logoColor=white)](https://github.com/opendatalab/PM4Bench)25[![Benchmark](https://img.shields.io/badge/Hugging_Face-Benchmark-FFD21E.svg)](https://huggingface.co/datasets/songjhPKU/PM4Bench)26[![Training Data](https://img.shields.io/badge/Hugging_Face-QGO_Train-FFD21E.svg)](https://huggingface.co/datasets/DatasetMan/PM4Bench-QGO-Train)27 28</div>29 30## Overview31 32The paper [*Benchmarking and Boosting Multilingual Capabilities of LVLMs via33OCR-Centric Reinforcement Learning*](https://little-bird-vodka.github.io/PM4Bench/paper-v3.html)34introduces the strictly parallel PM4Bench benchmark and finds that OCR is a key35source of cross-lingual performance gaps under vision-only input. QGO trains36on fully synthesized OCR data with a dual reward for recognition accuracy and37reasoning-preserving response structure. It improves average score on every38PM4Bench task and setting and transfers gains to external multilingual OCR39benchmarks.40 41QGO-8B is the OCR-centric GRPO checkpoint derived from42[`Qwen/Qwen3-VL-8B-Thinking`](https://huggingface.co/Qwen/Qwen3-VL-8B-Thinking).43This repository contains the complete global-step-200 BF16 checkpoint used in44the PM4Bench paper.45 46## Model details47 48- Architecture: `Qwen3VLForConditionalGeneration`49- Base model: `Qwen/Qwen3-VL-8B-Thinking`50- Training method: GRPO51- Released checkpoint: global step 20052- Weight format: safetensors, four BF16 shards53- Indexed tensor bytes: 17,534,247,39254- License: Apache-2.055- Tested Transformers version: 4.57.656 57## Usage58 59```python60from transformers import AutoProcessor, Qwen3VLForConditionalGeneration61 62model_id = "DatasetMan/QGO-8B"63model = Qwen3VLForConditionalGeneration.from_pretrained(64    model_id,65    dtype="auto",66    device_map="auto",67)68processor = AutoProcessor.from_pretrained(model_id)69```70 71The full BF16 weights are approximately 17.5 GB before runtime allocations.72Plan GPU/CPU memory for weights, vision inputs, KV cache, and generation in73addition to the checkpoint size.74 75Use the base model's official chat template and greedy decoding for PM4Bench76evaluation. Task prompts and evaluation code are provided in77<https://github.com/opendatalab/PM4Bench>.78 79## Training80 81The released data is available at82<https://huggingface.co/datasets/DatasetMan/PM4Bench-QGO-Train>. The recipe83uses 32 prompts and 8 rollouts per prompt (256 trajectories per step), AdamW84with learning rate `1e-6`, BF16, and eight GPUs.85 86## PM4Bench evaluation87 88| Model | MDUR trad. | MDUR vision | MIQA trad. | MIQA vision | MSOCR | MGUI |89|---|---:|---:|---:|---:|---:|---:|90| Qwen3-VL-8B-Thinking | 38.55 | 34.88 | 53.63 | 47.69 | 1.53 | 78.30 |91| QGO-8B | 46.82 | 40.84 | 55.24 | 51.06 | 8.17 | 80.00 |92 93These are the audited paper results. MDUR and MGUI are percentages, MIQA is94the six-dimension judge score on a 10-100 scale, and MSOCR is on a 0-40 scale.95 96## Limitations97 98QGO-8B targets multilingual OCR robustness. It inherits limitations and risks99from the Qwen base model and is not guaranteed to improve every downstream100task or language. Coordinate outputs, OCR transcriptions, and long-form101reasoning should be validated before use in consequential applications.102 103## Cite Us104 105<pre><code class="language-bibtex">106@misc{gao2026benchmarkingboostingmultilingualcapabilities,107      title={Benchmarking and Boosting Multilingual Capabilities of LVLMs via OCR-Centric Reinforcement Learning},108      author={Junyuan Gao and Jiahe Song and Jiang Wu and Runchuan Zhu and Guanlin Shen and Shasha Wang and Xingjian Wei and Haote Yang and Weijia Li and Bin Wang and Lijun Wu and Conghui He},109      year={2026},110      eprint={2503&#46;18484},111      archivePrefix={arXiv},112      primaryClass={cs.CV},113      url={https://arxiv&#46;org/abs/2503&#46;18484v3},114}115</code></pre>116