barha/granite-switch-4.0-350m-cti-onnx

granite-switch-4.0-350m-cti (ONNX / transformers.js)

ONNX export of a Granite Switch model — base Granite with per-token LoRA adapter switching driven by control tokens — packaged to load on transformers.js.

• —adapters: 1 (cti-technique-mapping)
• —layers: 28, kv-heads: 4, head-dim: 64
• —vocab: 100353

Available dtypes

• —fp32 — full precision (onnx/model.onnx)

Loading (native transformers.js architecture)

This repo declares its true model_type: granite_switch. The Granite Switch web runtime ships a self-registering shim that teaches transformers.js the GraniteSwitchForCausalLM architecture, so it loads via the standard AutoModelForCausalLM.from_pretrained. Granite Switch selects adapters via a causal, cumulative switch attention; the shim's custom forward threads that state (past_switch_key0 / past_switch_val0) across decode steps — the generic loop alone is not sufficient.

// import the shim once (registers granite_switch), then use the re-exported API
import { AutoModelForCausalLM, loadGraniteSwitch } from "granite-switch-web/src/granite-switch-register.js";

// loadGraniteSwitch wires the external-data sidecar (model.onnx.data) for you:
const gs = await loadGraniteSwitch("barha/granite-switch-4.0-350m-cti-onnx", { dtype: "fp32" });
const out = await gs.generate({ inputs, max_new_tokens: 16, do_sample: false });

gs_onnx.json carries the runtime metadata (layer/head counts, adapter token ids) the decode loop needs.

Provenance

Exported with granite_switch.onnx.export + granite_switch.onnx.package. See the project docs (docs/ONNX_BROWSER_PORT.md) for the export pipeline and parity gates (HF-vs-ONNX max|diff| ~1e-4, argmax 100% on the source model).