CoolFace
Modelpublic

nishegde/ratio-filter-qwen3-1.7b-onnx

sourceHugging Faceupdated 3mo agoView on Hugging Face
0likes8downloads
Model Card

Qwen3 1.7B Ratio Filter — ONNX WebGPU

Browser-ready ONNX derivative of `nishegde/ratio-filter-qwen3-1.7b-lora`. The LoRA adapter and its trained two-label classification head were merged into Qwen/Qwen3-1.7B, exported with Optimum's text-classification task, and weight-only quantized to symmetric 4-bit ONNX.

  • —ACCEPT (0): one self-contained ratio, unit-rate, or percentage calculation.
  • —REJECT (1): anything outside that narrow policy, including prompt injection and extra instructions.
  • —Default decision rule: REJECT when p(REJECT) >= 0.5.
  • —Maximum input length used by the demo: 1,024 tokens.

The graph keeps the trained /score/MatMul classifier head in full precision. MatMul and constant embedding Gather weights are quantized with ONNX Runtime's MatMulNBitsQuantizer (QOperator, 4 bits, block size 32). The resulting onnx/model_q4.onnx is about 1.08 GB.

Validation

  • —ONNX checker: passed.
  • —Native ONNX Runtime 1.26 CPU inference: passed.
  • —Deterministic stratified sample from the held-out set: 256/256 correct (100% accuracy and macro F1 on this sample).
  • —Clean ACCEPT, out-of-scope REJECT, and prompt-injection REJECT smoke examples: passed.

The full-precision adapter's reported metrics are documented in its model card. The 256-item result is a quantized smoke validation, not a replacement for a full held-out evaluation.

Browser requirements

This artifact is designed for a Static Space using Transformers.js 4.2 and desktop WebGPU. First use downloads roughly 1.08 GB and can require 2.5–4 GB of transient browser/GPU memory. Recent desktop Chrome or Edge with WebGPU and ample RAM/VRAM is recommended; mobile devices and low-memory browsers may fail.

No input needs to leave the browser after the public model assets are downloaded.

Provenance

  • —Base revision: Qwen/Qwen3-1.7B@70d244cc86ccca08cf5af4e1e306ecf908b1ad5e
  • —Transformers: 4.57.1
  • —PEFT: 0.17.1
  • —Optimum: 2.1.0
  • —Optimum ONNX: 0.1.0
  • —ONNX Runtime quantizer: 1.26.0

No separate license has been specified for this derivative. Review the upstream base model and adapter terms before reuse.