Immortalizer/moonshine-streaming-medium-onnx

Moonshine Streaming Medium — merged-decoder int8 ONNX

int8-quantized ONNX export of [Moonshine](https://github.com/usefulsensors/moonshine), the fast English speech-recognition model from [Useful Sensors](https://huggingface.co/UsefulSensors) (paper: *Moonshine: Speech Recognition for Live Transcription and Voice Commands*, arXiv:2410.15608). All model weights, architecture, and accuracy are Useful Sensors' work — please cite and credit them as the source of this model.

This repo is only a packaging change: the decoder is shipped as a single use_cache_branch-gated graph instead of the usual two-file split decoder. Nothing about the math, weights, or quantization changes.

What's different from a standard streaming ONNX export

Streaming-Moonshine ONNX exports normally ship the decoder as two files — decoder_model_quantized.onnx (step 0, no KV cache) and decoder_with_past_model_quantized.onnx (steps 1..N, with cache) — whose weights are ~90% identical, so both sit resident in RAM at once. This repo merges that pair into one decoder_model_merged_quantized.onnx switched by a use_cache_branch input — the same single-graph shape the official onnx-community/moonshine-{tiny,base} exports already use.

Result: ~209 MB smaller on disk and roughly half the resident decoder RAM, with no accuracy change — the merge operates on the already-int8 graphs (no re-quantization), and the merged decoder is bit-for-bit functionally identical to the split pair (same token sequence; logits agree to ~1e-5, pure graph-optimization reordering).

Lineage / attribution

1. 1.[Useful Sensors — Moonshine](https://huggingface.co/UsefulSensors/moonshine-streaming-medium) — the original model, weights, and architecture. The model is their work.
2. 2.[Mazino0/moonshine-streaming-medium-onnx](https://huggingface.co/Mazino0/moonshine-streaming-medium-onnx) — the int8 split-decoder ONNX export this repack is built from.
3. 3.This repo — the split decoder merged into one use_cache_branch graph (encoder and tokenizer copied verbatim).

License

MIT, inherited from the upstream Moonshine release.

Usage

Built for my-voice, an offline English push-to-talk voice-typing daemon, but it's a standard onnxruntime-loadable encoder + merged-decoder + tokenizer.json layout.