divakar7c5/wealthwand-qwen3-1.7b-int4-onnx

Qwen3-1.7B — int4 CPU, ONNX Runtime GenAI

An int4 CPU build of Qwen/Qwen3-1.7B in ONNX Runtime GenAI format, for on-device use in Wealth Wand — a local-first personal finance app that reads bank and card statements entirely on the user's device.

~1.05 GB. Runs on CPU: Windows (x64/ARM64), Android, iOS and macOS.

Why this exists

Wealth Wand extracts statements on-device so financial documents never leave the machine. That requires a model small enough for a phone and in GenAI format, which the official Qwen release does not ship.

Community ONNX exports exist, but this model parses people's bank statements — its provenance should be traceable to the official weights rather than to an unattributed repackaging. So it is built here, from source, reproducibly.

How it was built

Straight from the official weights, no credentials required at any step (Qwen/Qwen3-1.7B is public):

pip install onnxruntime-genai transformers torch huggingface_hub onnx onnx-ir onnxruntime

python -c "from huggingface_hub import snapshot_download; \
  snapshot_download('Qwen/Qwen3-1.7B', local_dir='qwen3-src', token=False, \
  allow_patterns=['*.json','*.safetensors','*.txt','*.jinja'])"

python -m onnxruntime_genai.models.builder \
  -i qwen3-src -o out -p int4 -e cpu \
  --extra_options block_size=32 accuracy_level=4

Reproduce it and you should get the same artefacts.

Model format compatibility is not guaranteed across ONNX Runtime GenAI versions. Pair this build with a 0.16.x runtime.

Usage

import onnxruntime_genai as og

model = og.Model("path/to/this/folder")
tok = og.Tokenizer(model)

prompt = (
    "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n"
    "<|im_start|>user\nHello<|im_end|>\n"
    "<|im_start|>assistant\n<think>\n\n</think>\n\n"
)

params = og.GeneratorParams(model)
params.set_search_options(max_length=3072, temperature=0.1)
gen = og.Generator(model, params)
gen.append_tokens(tok.encode(prompt))
while not gen.is_done():
    gen.generate_next_token()
print(tok.decode(gen.get_sequence(0)))

Qwen3 uses ChatML. The trailing empty <think>\n\n</think>\n\n is how Qwen3's own chat template disables thinking mode — omit it and the model emits reasoning before its answer, which will break anything expecting structured output as the first thing it sees.

Known limitations for structured extraction

Measured on synthetic Indian bank statements, this build:

• —occasionally emits trailing characters after the closing brace, so parse the first balanced JSON object rather than assuming the whole response is JSON;
• —sometimes reports the first transaction date as the statement period start instead of the printed period;
• —can invert debit/credit on a row while reporting a balance consistent with the opposite direction.

That last one matters most for financial use, and it is checkable: where a row prints a running balance, the direction of the movement is the bank's own arithmetic. Wealth Wand reconciles every extracted row against the balance column and corrects unambiguous sign errors rather than trusting the model. Anyone using this for financial extraction should do something equivalent.

Licence

Apache-2.0, inherited from Qwen3-1.7B. This repository contains only a format conversion and quantization; no weights were retrained or otherwise modified beyond int4 quantization.