Modelpublic

OpenMed/MedPsy-1.7B-ONNX

sourceHugging Faceapache-2.0updated 17h agoView on Hugging Face

0likes247downloads

Model Card

MedPsy-1.7B - ONNX 4-bit

An ONNX conversion of `qvac/MedPsy-1.7B` for ONNX Runtime, quantized to a 4-bit body with 8-bit sensitive layers, for on-device use on Android, desktop and the browser (Transformers.js). The model is QVAC's MedPsy medical fine-tune of Qwen3; OpenMed made and published this conversion and is not affiliated with or endorsed by the model's authors.

Size and fidelity

	Source (BF16)	This repo
Parameters	1.72 B	1.72 B (unchanged)
Weights, measured from the tensors	-	1.32 GiB
Bits per weight, measured from the tensors	16	6.58

Measured against the source model in FP32 on 4,092 tokens of public-domain prose (Project Gutenberg), ONNX Runtime 1.30.0, CPU:

Metric	Value
Mean KL divergence of next-token distributions	0.140
Top-1 next-token agreement	78.7%
Perplexity change	+10.7%

The FP32 export this build was quantized from matches the source model (top-1 agreement 100%), so the difference above is the quantization. The width is higher than the "4-bit" label suggests because the tensors that lose most at 4 bits, and the shared embedding table, are kept at 8 bits. Weights exclude the rotary-position tables stored in the graph.

Quantization

Field	Value
Body	int4 asymmetric round-to-nearest (uint4 + packed zero point), block 32, fp32 scales; MatMulNBits accuracy_level 4
Kept at 8 bits	int8 asymmetric, block 32, on the tensors llama.cpp's Q4KM rule upgrades (downproj in the first and last eighth of layers and every third layer between; vproj by the same rule over attention layers)
Embedding and output head	one int8 asymmetric block-32 table stored once, read by GatherBlockQuantized for the embedding and by MatMulNBits (bits 8) for the output head
Calibration	none (round-to-nearest); no calibration data
Export	onnxruntime-genai 0.16.0 model builder, FP32 CPU graph
Quantizer	onnxruntime 1.30.0 `MatMulNBitsQuantizer`

The tokenizer, chat template and generation defaults are the upstream files, unchanged. Exact commands, tool versions and every file's SHA-256 are in openmed_build.json; the graph's metadata_props record the source revision and the modification notice.

Architecture

Field	Value
Source model type	`qwen3` (`Qwen3ForCausalLM`), a fine-tune of Qwen3-1.7B
Hidden size	2048
Layers	28, grouped-query attention
Vocabulary	151,936, tied input/output embeddings

Quick start

Transformers.js (browser or Node)

import { pipeline } from "@huggingface/transformers";

const generator = await pipeline("text-generation", "OpenMed/MedPsy-1.7B-ONNX", { dtype: "q4" });
const messages = [
  { role: "user", content: "Explain the difference between a panic attack and a heart attack in plain language." },
];
const output = await generator(messages, { max_new_tokens: 256, do_sample: false });
console.log(output[0].generated_text.at(-1).content);

ONNX Runtime (Python, Android and elsewhere)

The graph is a standard ONNX Runtime decoder: feed input_ids, attention_mask and the past_key_values.* cache inputs, then feed each present* output back as the matching past* input on the next step. It uses ONNX Runtime's com.microsoft operators (MatMulNBits, GatherBlockQuantized, GroupQueryAttention), so run it with ONNX Runtime 1.30 or later (onnxruntime-android on Android).

Tested with onnxruntime 1.30.0 (CPU execution provider) on macOS and with Transformers.js 4.3.0, which produced identical greedy tokens. It has not yet been measured on an Android device.

File set

File	Size	SHA-256
`ATTRIBUTIONS.md`	0.1 MiB	`0a88dcaf9f2c86a1…`
`LICENSE`	0.0 MiB	`809fa1ed21450f59…`
`added_tokens.json`	0.0 MiB	`c0284b582e14987f…`
`chat_template.jinja`	0.0 MiB	`a8b0dcfcc923bd26…`
`config.json`	0.0 MiB	`4a83b36bdea9961c…`
`generation_config.json`	0.0 MiB	`ac1d241e9617d6ca…`
`merges.txt`	1.6 MiB	`8831e4f1a0444713…`
`onnx/model_q4.onnx`	0.3 MiB	`6fcba01792e3e293…`
`onnx/model_q4.onnx_data`	1,368.9 MiB	`4116b03a0ce23786…`
`special_tokens_map.json`	0.0 MiB	`76862e765266b85a…`
`tokenizer.json`	10.9 MiB	`aeb13307a71acd8f…`
`tokenizer_config.json`	0.0 MiB	`eea5f48cdcc82dda…`
`vocab.json`	2.6 MiB	`ca10d7e9fb3ed185…`

Intended use

For research and education. QVAC describes MedPsy as a model for research and educational purposes, trained on data generated from CC-BY-NC datasets; see the upstream card. It is not a medical device and not a substitute for professional medical advice, diagnosis or treatment. Outputs can be wrong; verify them.

Licence

Distributed under the Apache License 2.0, the source model's licence (with the upstream attributions in ATTRIBUTIONS.md). The ONNX graphs and weight files in onnx/ were converted and quantized from the source model's PyTorch weights by OpenMed on 2026-09-23; all other files are unchanged upstream copies.

Source: `qvac/MedPsy-1.7B` at revision 59335b96dd541b0061d748d7a6e9536e92274985. Please cite the original model when you use this conversion.