inductiveML/monarch-webgpu

MONARCH — LFM2.5 WebGPU Kernels

An open-source browser decoder for Liquid AI's LFM2.5-230M. Load the real model, generate a response, and measure your GPU. Inference happens on your device. No inference API, hosted GPU, or private model credentials are needed.

Experiment and methodology · Model files · Direct app

Run locally

Requires Node.js 20.19+ (or 22.12+) and npm.

git clone https://huggingface.co/spaces/inductiveML/monarch-webgpu
cd monarch-webgpu
npm ci
npm test
npm run dev

Open the local URL Vite prints. npm run build produces dist/ and refreshes the committed static files (index.html, assets/, and monarch/). npm run preview serves the production build with the required isolation headers. The build downloads no model weights and does not depend on any private research checkout. Model files download on demand when you click Load model.

For browser support, the runtime checks WebGPU, shader-f16, fixed 32-lane subgroups, 1,024-thread workgroups, and the storage-buffer limit before downloading weights. Current Chromium on a compatible Apple Silicon Mac is the tested path. Other devices may fail the feature or known-answer check. There is no silent server fallback.

The Space serves the committed build directly, so duplicating it requires no build credits. After changing source, run npm ci && npm test && npm run build, then commit the source and regenerated static files. source.html is the Vite input; index.html is generated. The dev server serves source.html at /.

What is included

• —Nine prompt presets, editable prompts, Markdown output, Stop and Unload controls.
• —Real greedy single-stream generation with live decode TPS and time to first token.
• —A fixed four-run benchmark at context 192 or 1,024, plus downloadable JSON results.
• —The complete source needed to build the worker: 28 pinned TypeScript/WGSL research files, tokenizer implementation, selected configuration, and correctness fixture.
• —Apache-2.0 source license, separate model license, attribution, and recorded research results.

The MIX_M40 decoder uses 72 ordinary dispatches per token. Its kernels fuse convolution work, specialize matrix geometry, use online subgroup attention, and make precision choices explicit. It does not use the experimental cross-workgroup atomic visibility protocol or a persistent token kernel.

Source map

The vendored code retains its original directory names so its relative imports and source hashes stay intact. An esbuild/Vite build packages it without changing the WGSL kernel bodies. npm run check:source verifies those hashes.

Model integrity and provenance

The model repository contains nine preconverted transport chunks totaling 168,647,680 bytes, plus tokenizer data. The runtime verifies every chunk and the reconstructed buffer:

ca1fea89fd9f3ca7e5d6d5720705c96f457a92edd0cfc811cf8d9391ac784cb0

The upstream ONNX revision and layout transformations are documented in the model card. The Space uses an immutable model repository revision. It runs a known prefill seed and 32-token continuation check before enabling the UI. Errors and device loss stop the run.

Benchmark protocol and limits

Generation is greedy, single-turn ChatML with at most 1,024 prompt tokens and 512 output tokens. Model loading and prompt prefill contribute to startup / time to first token. The live decode counter starts after the first token and includes streaming overhead. A short response may end inside the last computed 32-token group.

The fixed benchmark:

1. 1.Uses a pinned prompt and a 64-token warmup.
2. 2.Freshly primes four 256-token free-running continuations.
3. 3.Reads back every 32 tokens through a 64-token ring.
4. 4.Requires all four continuations to match exactly.
5. 5.Reports reciprocal median wall time per token and timing spread, excluding prefill.

The selected context is the center of the generated window: token positions 64–319 for 192, or 896–1151 for 1,024. Timestamp queries are optional; reported TPS is based on wall time.

The 1,574.4 tok/s figure is a recorded M4 Max / fullscreen Chromium lab observation at context 192. The quick public benchmark uses normal browser settings and does not reproduce the full controlled lab protocol. The research kernel retained NO_VERDICT_SPLIT; later B1.11a pricing remained unadjudicated because environmental/thermal conditions failed. These conditions and the distinction between recorded and live measurements are part of the result, not hidden exceptions.

Licenses and credits

Code owned by Inductive ML is released under Apache-2.0 (LICENSE). This grant does not relicense model or third-party materials. Model weights and tokenizer data retain Liquid AI's LFM Open License v1.0, including its commercial-use conditions. Hugging Face tokenizers.js 0.1.3 is Apache-2.0. React and the Markdown dependencies retain their own licenses. See NOTICE and THIRD_PARTY_NOTICES.md.

The interface was inspired by webml-community/lfm2-webgpu-kernels. The kernel/runtime implementation is MONARCH's; no source from that reference demo is included.