shreyask/qmd-query-expansion-1.7B-ONNX-kvcache-fp32

qmd-query-expansion-1.7B ONNX KV Cache FP32

This repo contains a CPU-exported ONNX version of tobil/qmd-query-expansion-1.7B with flattened KV cache inputs and outputs for decoder generation.

What is included:

• —Root tokenizer and config files
• —onnx/model.onnx
• —onnx/model.onnx.data

Status:

• —KV-cache ONNX export is validated in ONNX Runtime
• —Variable seq_len and past_len work, including past_len=0
• —FP32 only
• —Q4 export is still pending because the current MatMulNBitsQuantizer path does not rewrite this graph yet

Validation summary:

• —Exported with torch.onnx.export(..., dynamo=True)
• —Verified with ONNX Runtime for multiple (seq_len, past_len) pairs:
• —(2, 8)
• —(1, 6)
• —(3, 5)
• —(1, 0)

Intended use:

• —This repo is for testing Transformers.js generation with native KV cache support
• —It is not the final quantized deployment artifact