ProCreations/minicpm5-2b-webgpu

MiniCPM5 · The local creative engine

Intelligence, launched locally. Chat with MiniCPM5 and create websites, Word documents, Excel workbooks and PDFs entirely in your browser.

Dedicated WebGPU runtime

This Space uses a purpose-built JavaScript / WGSL runtime for MiniCPM5's 42-layer Llama architecture. All transformer layers, embeddings and output projection run on the GPU. It does not load a WASM inference binary and does not require Memory64, JSPI or SharedArrayBuffer.

• —Original Q8_0 weights: 2.68 GB, 2,516,756,480 parameters, 16 query heads and two KV heads. All 381 tensor byte hashes match the official original. The six browser GGUF shards are lossless splits, without pruning or re-quantization.
• —Direct packed-Q8 matrix kernels, fused gate/up projection and SiLU, fused residual RMS normalization, tiled prompt processing, split-K attention and resident prompt reuse.
• —GPU top-k / top-p sampling, four-token decode submission batches, and incremental UTF-8 token streaming. Top K = 0 uses full-vocabulary CPU sampling and disables speculation; model inference stays on the GPU.
• —Subgroup kernels when supported; portable workgroup reductions otherwise. Vocabulary matrices split into four bindings on devices with smaller buffer limits. Smaller shared-memory kernels cover devices with 16 KB workgroup storage.
• —FP32, FP16, 8-bit and 4-bit KV caches. A single control applies to both K and V. Quantized KV uses 32-value blocks with a 32-bit scale (36 bytes for Q8, 20 bytes for Q4).

DSpark acceleration

The official MiniCPM5-2B-DSpark draft model runs locally alongside the Q8 target. Its pinned BF16 weights add 647,558,522 bytes, making the combined download 3.33 GB.

The implementation ports the five-layer DSpark backbone and Markov head to WebGPU, projects five target hidden states, and reuses the target's embedding and output weights. It proposes up to seven tokens, then verifies them in an eight-token target pass. Greedy decoding accepts matching target tokens. Sampled decoding uses probability-ratio acceptance and residual resampling to preserve the target distribution within floating-point precision, following exact speculative sampling. Rejected suffixes are discarded from the target and draft caches. The confidence head is not used; scheduling adapts from measured verification cost instead.

Auto is the default: measure draft cost per committed token against ordinary decoding, and switch to ordinary decoding when speculation is slower. On forces drafting. Off skips downloading and allocating DSpark. You can choose on the launch screen or in model settings. Drafting benefits predictable code more than some creative or high-temperature responses; it does not guarantee a speedup on every prompt.

Measured performance

Validation on an Apple M4 Max, Chromium WebGPU, a 16K context allocation and FP16 KV:

These are greedy, short-prompt decode benchmarks; download and prompt-processing time are excluded. All 520 greedy tokens across arithmetic, code and prose matched the native llama.cpp reference, both with the subgroup path and with portable kernels plus split vocabulary bindings. The forced portable path reached 141 tok/s on the code test in Chromium; this does not emulate Safari or measure Safari's speed. The 64K/Q8 and 128K/Q4 configurations passed allocation and short-generation checks, not a full long-context quality evaluation. Throughput varies with browser, hardware, actual prompt length, temperature and draft acceptance. No universal 200 tok/s claim is made. Public benchmark summaries are in benchmarks/validation.json.

A public uncached launch with DSpark and storage off took 35.17 seconds: 33.71 seconds downloading 3.33 GB and 1.46 seconds preparing the GPU, with 400 range requests and zero retries. An opt-in cached reload of both models took 1.45 seconds with zero downloads. A live HTML generation at the default temperature of 1 measured 90.2 tok/s over 1,448 tokens; the generated code needed a revision. These measurements are from the same device and connection, not universal guarantees.

Launch and settings

The first screen follows Maple WebGPU: a full-scene 3D rocket, large typography, context slider, one KV precision control, DSpark selector, storage toggle and launch button. The rocket moves at idle, accelerates during loading and launches before chat appears. Rendering stops during chat and respects reduced-motion preferences.

• —Context: 2K, 4K, 8K, 16K, 32K, 64K, 128K. Defaults: 16K, FP16 KV, DSpark Auto, fast prompt batches.
• —Thinking and file creation can be toggled independently.
• —Temperature, top-p, top-k, repeat penalty (last 64 tokens), response budget, seed and custom instructions.
• —Temperature defaults to 1, top-p to .95, top-k to 40, and the response budget to 4,096 tokens.
• —Memory estimates include target and draft weights, both KV caches and a workspace allowance. Driver/browser overhead is additional. 128K is the model's ceiling; some devices cannot allocate it.
• —GPU preparation has live progress, cancellation and a two-minute timeout. Failed setup retains complete downloads temporarily in the tab for retry.

Four real creation tools

Ask naturally: “Build a coffee shop website,” “Create a Word project plan,” “Make an Excel budget with totals,” or “Create a stargazing PDF.”

Tools execute locally and send success or error results back to the model. Incomplete or invalid calls trigger automatic repair, with a larger bounded response budget and thinking disabled for the retry. The assistant continues after a successful tool result. A six-round limit prevents runaway loops. A real regression run forced a 128-token truncated HTML call, then automatically repaired it, created the HTML file and completed the reply.

HTML previews have no same-origin privileges or network access. Downloaded HTML preserves the generated code. Excel formulas recalculate in a spreadsheet application; previews show formula source. Word previews show document content rather than pixel-identical Word rendering. PDF fonts cover Latin/Greek/Cyrillic, with Simplified Chinese loaded on demand. Model outputs may need editing. Text attachments are supported; images and audio are not model inputs.

Downloads and privacy

Model storage is off by default, for both target and DSpark. Downloads use cache: no-store and temporary session Blobs, without writing weights to OPFS or the Cache API. Reloading requires another download. The launch screen shows the total before you start.

Enable Save model in browser to retain weights using a buffered OPFS writer. Completed files survive interrupted downloads, and later launches reuse them. Turning storage off removes previously saved model files. Downloads use validated, interleaved 8 MiB ranges with eight concurrent transfers on desktop, four on mobile, and individual retries for stalled pieces. Optional document libraries load only when needed.

Chats and creations are saved locally in IndexedDB. Prompts and generated files are never sent to an inference server. Hugging Face receives app and model download requests. There is no app telemetry. Clearing site data removes local chats and saved downloads.

Replies are checkpointed approximately once per second while streaming, including unfinished tool source. After an interrupted session, the launch page offers recovery; load the model to download the saved draft or retry the request. Completed files are preserved. Recovery restarts generation from the saved conversation, not from a saved GPU cache, and cannot restore replies lost before checkpointing was added.

The September 8 streaming update replaces nested async generators with callbacks throughout inference, chat and tool execution. A reported Safari crash was in JavaScriptCore's optimizing compiler during async-generator resumption, rather than an out-of-memory termination. The replacement passed a 196,674-chunk test in the system JavaScriptCore with simulated model computation, plus 6,066 real GPU tokens with a 64K context allocation, FP16 KV and DSpark forced on in Chromium's portable kernel path. These are separate tests; native Safari end-to-end validation is still pending. Details are in benchmarks/safari-streaming-fix.json.

A browser with WebGPU and enough GPU memory is required. Current desktop Chrome/Edge is the primary tested path. Safari and Firefox use portable kernels when needed; their performance and driver behavior vary, and this release has not been benchmarked in native Safari. If an embed restricts WebGPU or storage, open the app directly.

Development

npm ci
npm run dev
npm test
npm run build

The development-only ?test=1 harness checks actual model inference, all file formats, tool repair, optional storage and runtime coverage. ?qa-portable=1&qa-sharded=1&qa-small-workgroups=1 forces the compatibility kernel variants in the current browser. This does not emulate another browser. Results are written to ignored .qa/ through a loopback-only endpoint. QA UI is eliminated from production builds.

Hugging Face builds the static app with npm ci && npm run build, installing the exact dependency versions in package-lock.json, then serves dist/index.html from its generated build revision. The root index.html is the Vite source entry and must not be used as the deployed entry point. If the Hub reports a configuration error while the direct app still loads, validate the README metadata and inspect the generated build before changing inference code.

On macOS, the production streaming/tool loop can also be stress-tested independently of GPU computation:

mkdir -p .qa
./node_modules/.bin/esbuild scripts/jsc-stream-stress.js --bundle --format=iife --platform=browser --target=es2022 --define:import.meta.env.DEV=false --outfile=.qa/jsc-stream-stress.js
/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/Helpers/jsc .qa/jsc-stream-stress.js

Credits and licenses

• —MiniCPM5 and DSpark: OpenBMB, Apache 2.0.
• —DSpark architecture reference: DeepSpec, Apache 2.0.
• —Application and dedicated runtime: ProCreations, Apache 2.0.
• —Hugging Face Tokenizers, Three.js, Lucide, Marked, DOMPurify, docx, ExcelJS and jsPDF: respective upstream licenses.
• —Fonts: SIL Open Font License; notices in public/fonts.