shreyask/needle-playground

Needle Playground (Browser)

26M-parameter function-calling model running entirely in your browser — no server, no GPU, just onnxruntime-web + WASM. Tap a preset chip, watch JSON appear.

Credits

• —Model: Cactus-Compute/needle — the original Simple Attention Network, trained by Cactus Compute on 200B tokens of pre-training and 2B tokens of function-call post-training.
• —ONNX export: onnx-community/needle-onnx — browser-ready ONNX artifacts produced by a JAX→PyTorch port + torch.onnx.export, with byte-identical output parity against the upstream needle.generate().
• —Browser app: This Space — Vite + TypeScript, onnxruntime-web (WASM backend), sentencepiece-js for tokenization. Mirrors the layout of the official needle playground CLI.

How it works

1. 1.Page load: fetch encoder.onnx, decoder_step.onnx, needle.model from onnx-community/needle-onnx (~140 MB; cached by the browser after first visit).
2. 2.Click a preset, or type your own query + tool definitions.
3. 3.On generate: encoder runs once over [query, <tools>, JSON.stringify(tools)], then the decoder runs step-by-step with KV-cache, seeded with <eos>, until the model emits <eos> again or 256 tokens are reached.
4. 4.Output is decoded with SentencePiece, stripped of the leading <tool_call> marker, parsed as JSON, and pretty-printed.

Greedy argmax sampling, EOS-only stop — matches Cactus's native generate() configuration exactly.

Source

Full source is available in this Space (browse files above) and the same code as the upstream project repo. Build with npm install && npm run build; dev with npm run dev.

License

MIT (matching the upstream Cactus Needle license).