TakTak/qwen3-vl-8b-mathmm-tradeoff-research-lora

Qwen3-VL-8B Math-MM Trade-off Research LoRA

Important: research artifact, not a general upgrade

This adapter is released as a category-dependent trade-off research artifact. It improves one Korean image-math evaluation set while significantly regressing on two others. It is not recommended as a drop-in replacement for the Base model and must not be described as a general math specialist.

Its purpose in this release is to make the observed trade-off reproducible and to support research on explicit Base/adapter selection rules.

Model details

• —Base model: Qwen/Qwen3-VL-8B-Thinking
• —Base revision: 92f3c4b4feadd3a016ef468d103bb5f58b2a2c6b
• —Image-trace teacher: Qwen/Qwen3-VL-32B-Thinking
• —Teacher revision: 7edd10ffd1196091948fb245ff63e406ccb2d4d1
• —Adapter: PEFT LoRA, rank 32, alpha 64, dropout 0.05
• —Target modules: q/k/v/o and gate/up/down projections
• —Adapter weight SHA-256: 5ecf27b144369a260fe5f1aa49bfa095c27f879b9302dbcf266ee2721b4d11bb

Training construction

The training mixture combined Korean text-math solutions with answer-verified multimodal teacher traces.

1. 1.Korean text-math rows were prepared from ChuGyouk/AI-MO-NuminaMath-CoT-Ko. Rows required an extractable boxed answer and a minimum solution length. A deterministic 12,000-row subset was selected.
2. 2.The 32B teacher generated Korean reasoning traces for MMK12 image-math rows. Only 568 teacher outputs scored correct and containing an image were retained.
3. 3.The 568 multimodal rows were repeated twice before a deterministic 1% validation split, yielding 13,005 training rows and 131 validation rows.

The final training mixture contained 11,883 text-only rows and 1,122 single-image rows. Source rows and images are not redistributed.

Training procedure

• —Epochs: 2
• —Learning rate: 1e-4
• —Precision: BF16
• —Per-device batch size: 1
• —Gradient accumulation: 8
• —Maximum sequence length: 4,096
• —Maximum image long side: 896 pixels
• —Seed: 1234
• —Hardware: 8 GPUs
• —Completed steps: 408/408
• —Recorded training loss: 0.4021

Canonical paired evaluation

Base and adapter arms used identical inputs, prompts, greedy decoding, 4,096-token effective output budget, scorer, and UID-aligned paired tests. All three comparisons survived family-wise correction. Answer type below means the expected final-answer form (for example number, expression, or text); it is not a school subject label.

The AIHub15 gain is concentrated in expression- and text-answer rows, while the regressions are concentrated in number-answer rows. This pattern is why all three results must be reported together.

Rule-based selection pilot

A pre-registered CPU-only pilot used existing scored rows and selected among Base, the primary Distill32 adapter, and this adapter using only evaluation set identity and answer type. On a deterministic held-out half (n=614), the rule kept Base for number-answer rows and used this adapter only in selected AIHub15 expression/text cells.

• —Always Base: 66.4%
• —Rule-selected arm: 71.5%
• —Difference: +5.05pp, 95% CI [+2.6, +7.7]pp
• —Exact McNemar p: 1.17e-04

This is a retrospective viability pilot, not a learned router, deployed ensemble, or proof of automatic routing performance.

Intended use

• —Reproduce the measured category-dependent trade-off.
• —Study answer-type-aware Base/adapter selection with a safe Base fallback.
• —Analyze multimodal teacher-trace plus Korean text-math mixtures.

Limitations and prohibited interpretations

• —Do not present this adapter as a general improvement over Base.
• —Do not omit the MMK12 and AIHub17 regressions when citing the AIHub15 gain.
• —Do not call the MMK12 set held out for this adapter.
• —Do not claim a completed specialist, learned router, deployed ensemble, or broad Korean math/science generalization.
• —The rule pilot uses known evaluation-set identity and answer type; a real pre-inference router was not trained or validated.
• —Safety, fairness, and high-stakes educational use were not evaluated.

Loading

from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel

base_id = "Qwen/Qwen3-VL-8B-Thinking"
base_revision = "92f3c4b4feadd3a016ef468d103bb5f58b2a2c6b"
adapter_id = "TakTak/qwen3-vl-8b-mathmm-tradeoff-research-lora"

processor = AutoProcessor.from_pretrained(base_id, revision=base_revision)
base = AutoModelForImageTextToText.from_pretrained(
    base_id,
    revision=base_revision,
    torch_dtype="auto",
    device_map="auto",
)
model = PeftModel.from_pretrained(base, adapter_id)

License and upstream sources

The adapter is released under CC BY-NC 4.0. The Korean text-math source is CC-BY-NC-4.0. The pinned Qwen models are Apache-2.0, and the pinned MMK12 source snapshot contains an Apache-2.0 license file. No raw data, images, teacher completions, evaluation rows, predictions, or scored row-level outputs are included.

• —Base: https://huggingface.co/Qwen/Qwen3-VL-8B-Thinking
• —Teacher: https://huggingface.co/Qwen/Qwen3-VL-32B-Thinking
• —Text-math source: https://huggingface.co/datasets/ChuGyouk/AI-MO-NuminaMath-CoT-Ko
• —Image-math source: https://huggingface.co/datasets/FanqingM/MMK12