matbee/LFM2.5-1.2B-Instruct-ONNX-Vulkan

LFM2.5-1.2B-Instruct ONNX - Vulkan / dual-role decoders

Vulkan-ready Instruct decoders for the LFM2.5 Audio + Instruct dual-role bundle (role-isolated sessions; the Instruct role handles text/tool I/O while the audio role handles speech). This repo holds only the modified Instruct artifacts - the audio q4 stack (decoder, encoder, detokenizer, vocoder, embeddings) is byte-identical to the official LiquidAI/LFM2.5-Audio-1.5B-ONNX release and is referenced from there, not re-hosted.

Files

Graph contract

Both decoders expose the same 24-input contract as the audio q4 decoder: inputsembeds (f32, first), attentionmask (i64), and per-layer pastconv.* / pastkeyvalues.*.{key,value} (f32). Outputs: logits + presentconv./ present..{key,value} (the q4 graph has no hidden_states output - the Instruct role reads logits only).

What was modified (vs the official Liquid Instruct ONNX release)

Source: LiquidAI/LFM2.5-1.2B-Instruct-ONNX @ commit 10f72e70abf67ac0fd7ebf15bc5854726891d864 (modelq4.onnx). The official release was NOT runnable as-is in the target runtime (onnx-vulkan-rs, a Vulkan ONNX interpreter); three fixes were applied - see instructq4_provenance.json for the full record:

1. 1.Embedding contract: the release takes inputids (i64) and embeds internally via a custom q4 GatherBlockQuantized over a q4-quantized embed table. That node was removed and the input swapped to inputsembeds (f32) - the runtime feeds embeddings from embedtokens.bin (byte-identical to the audio q4 24-input contract). The weight-tied q4 lmhead (which reuses the embedding's scales + zero points) is preserved, so logits stay q4-quantized.
2. 2.Node order: the Liquid export node list was non-topological (110 violations); it was stably topo-sorted (ties on original index; 0 violations after).
3. 3.External-data locations repointed to decoderq4.onnxdata.

The f32 fallback decoder derives from the same export family's fp16 graph (topo-sorted) with all 148 weight initializers swapped from the Instruct bf16 checkpoint (LiquidAI/LFM2.5-1.2B-Instruct @ 6314d2b7cf28a6ae9de9d3e77dcfcd9c9f281c77) and an exact f16->f32 upcast of the remaining 150 f16 initializers (the target engine is f32/i32/q4-only; it has no f16 compute path).

Quantization format note (q4)

The Instruct q4 export is asymmetric MatMulNBits: 93 nodes, bits=4, blocksize=32, with a 4th input - a per-block **packed-int4 zero point** (uint8 tensor of shape [n][k/64], two int4 per byte, low nibble first). The Audio release is the 3-input symmetric form (zero point == 8). A runtime must implement the asymmetric form to use decoderq4.onnx; the onnx-vulkan-rs fork that produced this bundle adds packed-int4 zero-point variants of its Q4 kernels for exactly this (fork: https://github.com/matbeedotcom/onnx-vulkan-rs-edge, commit 2d660f2). onnxruntime CPU runs the 4-input form natively.

Validation

• —CPU (onnxruntime, greedy): the B4 tool-call prompt (88 ids) produces 21/21 tokens bit-identical to the bf16 PyTorch reference, including the toolcallstart / toolcallend wrapper tokens: [get_weather(city="Paris")]Checking the current weather in Paris.
• —Reproducibility: the committed export pipeline (dualrole/instructq4export.py in the project repo) re-derives decoderq4.onnx byte-identically (sha256 d839af62...).

License

LFM Open License v1.0 - inherited from the upstream Liquid releases (see LICENSE). The modifications are graph surgery + exact dtype conversion only; no weight values were retrained or requantized.