mustafaah/screen-highlighter-qwen3-vl-2b-webgpu

Screen Highlighter — trained RL model, experimental browser export

FP16 ONNX conversion of our merged Qwen3-VL-2B highlight-only SFT + final RL update-200 model. This is not the untrained base. Source and conversion scripts: ScreenHighlighter.

Status: processor, vision and positional-encoding checks plus native CUDA generation probes completed. Actual browser WebGPU execution remains unverified because the local Chrome instance exposed no GPU adapter. Native CUDA smoke tests use a separate causal-mask variant; shipping graphs retain the WebGPU attention-bias input. See included reports; these few probes are not an aggregate accuracy evaluation.

Use Transformers.js 4.2.0, device webgpu, dtype fp16 for embedtokens, visionencoder and decodermodelmerged. Requires shader-f16 and several GB of GPU memory; 6 GB compatibility has not been established. Downloads are approximately 4.9 GB. All inference is local after download.

The accompanying app applies explicit normalization and a shared antialiased bicubic resizer. Use that preprocessing and the included system prompt. Image limits are 256–1024 tokens; greedy output is one JSON highlight response capped at 1024 tokens, using 0–1000 coordinates.

Export corrections

The pinned community ONNX graph required restoring three DeepStack branches, Qwen3-VL interleaved MRoPE, and explicit image normalization. All 112 adapted projection tensors were checked against the saved merged weights, and the frozen vision tensors were checked against the checkpoint. Provenance records the exact base, adapter hash and graph template revision.

Naive all-linear 4-bit and MLP-only 4-bit exports regressed on the navigation/semantic probes and are deliberately excluded. This FP16 export is the fidelity baseline for future calibrated quantization, not a claim that 4-bit conversion is solved.

Browser file packaging

Decoder external data is split into files no larger than 1.5 GiB. Every relocated tensor was byte-verified against the original tested export. This changes storage layout only; graph operations and weights are unchanged.