datas3nt/qwen2vl-polygen-7b-lora-r16-1000

Polygen LoRA for Qwen2-VL-7B (r=16, 1000 steps, ScienceQA)

A LoRA adapter for `Qwen/Qwen2-VL-7B-Instruct` that is part of Datasent's polygen visual-token compression recipe. It teaches the LLM to interpret a compressed visual-token sequence produced by the polygen splice between Qwen2-VL's vision encoder and its language model. This is the production-scale counterpart to the 2B-targeted `datas3nt/qwen2vl-polygen-lora-r16-1000`: identical recipe, applied to the 7B backbone, evaluated on NVIDIA H200.

Interactive side-by-side demo: datas3nt/qwen2vl-polygen

What this adapter is (and isn't)

This is not a general-purpose Qwen2-VL fine-tune. It is one half of a two-part configuration:

1. 1.The polygen splice sits between Qwen2-VL's merger output and its LLM input and replaces the per-image visual-token sequence with a much shorter structured representation (or, in the hook splice path, an in-place reconstruction that preserves the original token count while compressing the information).
2. 2.This LoRA adapter teaches the LLM to interpret that compressed representation. Without the adapter, the LLM treats the compressed sequence as out-of-distribution noise; with the adapter, accuracy returns to (or exceeds) baseline.

The adapter alone (no polygen splice) gives a small accuracy lift on ScienceQA-distribution-shifted tasks via cross-benchmark transfer, but its purpose is the joint configuration with the splice.

Recipe

Results

Evaluated against an un-spliced un-LoRA baseline. The LoRA was trained only on ScienceQA; the lifts on the other benchmarks are pure cross-benchmark transfer.

ScienceQA (n=200), bypass path. Compute is reported as attention-FLOP reduction (attention is O(L^2) in visual sequence length, so this is a FLOP count, not wall-clock) and storage compression, per operating tier:

VQAv2 (n=2000): +2.50pp at T=4 (pure cross-benchmark transfer).

Measured on NVIDIA H200 with Qwen2-VL-7B-Instruct in fp16. T=3 is the accuracy headline (+8.4pp); T=2 is the compression headline (~71% fewer attention FLOPs at +7.4pp accuracy, a statistical tie with T=3). The headline numbers are accuracy and attention-FLOP reduction, not wall-clock: controlled measurement does not support a single-shot wall-clock speedup once the CPU encode step is counted, so this card does not claim one.

The attention-FLOP reduction follows from the visual-token reduction (a FLOP count). On bigger backbones attention is a larger share of the forward pass, so the same token reduction removes more of the total compute.

How to use

This adapter is intended to be loaded by Datasent's polygen SDK as part of a two-step setup:

from polygen.integrations.qwen2vl import PolygenQwen2VL

model = PolygenQwen2VL.from_pretrained(
    "Qwen/Qwen2-VL-7B-Instruct",
    lora_adapter="datas3nt/qwen2vl-polygen-7b-lora-r16-1000",
    operating_tier="T=4",        # T=3 for +8.4pp accuracy; T=2 for ~71% fewer attention FLOPs
    splice_mode="auto",          # bypass for moderate L_v, hook for high L_v
    torch_dtype="float16",
    device_map="cuda",
)

For evaluation without the polygen splice (rare; the adapter's value is in the joint configuration), it can be loaded as a standard PEFT adapter:

from peft import PeftModel
from transformers import Qwen2VLForConditionalGeneration, AutoProcessor

base = Qwen2VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2-VL-7B-Instruct",
    torch_dtype="float16",
    device_map="cuda",
)
model = PeftModel.from_pretrained(base, "datas3nt/qwen2vl-polygen-7b-lora-r16-1000")
processor = AutoProcessor.from_pretrained("Qwen/Qwen2-VL-7B-Instruct")

Intended use

• —Primary: evaluation and production-scale demonstration of Datasent's polygen visual-token compression on Qwen2-VL-7B, paired with the splice. This is the variant that produces the headline +8.4pp accuracy and ~71% fewer attention FLOPs result.
• —Secondary: cross-benchmark transfer studies on ScienceQA-adjacent vision-language tasks (the +3.05pp transfer effect on VQAv2 is the cleanest case on 7B).
• —Not intended for: general-purpose Qwen2-VL-7B fine-tuning, production deployment without the polygen SDK, or stand-alone use on tasks that diverge significantly from ScienceQA's distribution.

Limitations

• —7B-only. This adapter targets Qwen2-VL-7B-Instruct. The 2B-targeted counterpart lives at `datas3nt/qwen2vl-polygen-lora-r16-1000`.
• —Recipe assumes the polygen splice. Without the visual-token compression the adapter underperforms a full fine-tune; its value is in the joint configuration.
• —English training data. ScienceQA is an English-language benchmark; behaviour on non-English captions or non-Latin scripts is undocumented.
• —n=300 train sample. The adapter is intentionally small and cheap to train (~5 minutes on a single H200). Performance on out-of-distribution tasks beyond the benchmarks above has not been characterized.

Citation

@misc{datasent-polygen-vlm-2026,
  title  = {Polygen: visual-token compression for vision-language models},
  author = {Datasent, Inc.},
  year   = {2026},
  url    = {https://huggingface.co/spaces/datas3nt/qwen2vl-polygen},
}

License

Distributed under a Datasent evaluation license. Contact greg@datasent.com for production licensing.