dstefl/EuroMoE-2.6B-A0.6B-Instruct-2512-q4f16_1-MLC

EuroMoE-2.6B-A0.6B-Instruct-2512 · q4f16_1 · MLC/WebGPU

## ⚠️ Known broken: the wasm fails WGSL validation (2026-08-31) The weights convert and download fine, but the WebGPU model library in this repo does not load in any browser. The MoE gating kernel emitted by the TVM generation we must pin (see below) contains a compile-time negative index: `` Error while parsing WGSL: :37:23 error: index -1 out of bounds [0..7] local_top_k_index[-1i] = 0i; GPUPipelineError: [Invalid ShaderModule] … While validating compute stage ` Constant out-of-bounds indexing is a WGSL *validation* error, so every compliant implementation rejects it — this is not a driver quirk. The pin (mlc-llm 2008fe8) is forced by web-llm **0.2.84**, whose runtime cannot instantiate libraries built on newer TVM (LinkError: TVMFFIGetCustomAllocator`), and 0.2.84 is the newest web-llm on npm as of 2026-08-31. Until web-llm ships a newer runtime (or the topk codegen fix exists below the June 2026 TVM refactor), treat this repo as weights-only: usable if you compile your own model library for a non-web MLC target, not usable in a browser. The dense EuroLLM-1.7B conversion from the identical pipeline works — the bug is MoE-specific.

[utter-project/EuroMoE-2.6B-A0.6B-Instruct-2512](https://huggingface.co/utter-project/EuroMoE-2.6B-A0.6B-Instruct-2512) converted to the MLC format, intended to run fully client-side in a browser via [web-llm](https://github.com/mlc-ai/web-llm) — see the warning above for why that does not work yet. This is a conversion, not a fine-tune — original weights, quantized to q4f16_1.

The interesting shape: 2.6 B total parameters, 0.6 B active per token (64 experts, 8 active) — closer to a small model's latency with a larger model's capacity, built by the EuroLLM/UTTER project explicitly with edge deployment in mind, covering all EU languages including Czech.

What is in this repo

• —Quantized weight shards + tokenizer + mlc-chat-config.json (context_window_size: 4096).
• —EuroMoE-2.6B-A0.6B-Instruct-2512-q4f16_1-MLC-webgpu.wasm — the model library compiled for WebGPU against the TVM generation web-llm 0.2.84 ships its runtime for (mlc-llm 2008fe8). Libraries built against newer TVM fail at load with LinkError: TVMFFIGetCustomAllocator.

How it WOULD be used (web-llm) — currently fails, see the warning

import { CreateMLCEngine } from "@mlc-ai/web-llm";

// ⚠️ As of 2026-08-31 this throws GPUPipelineError during shader
// compilation (the broken gating kernel above). Kept for the day a
// working model library replaces the wasm in this repo.
const engine = await CreateMLCEngine("EuroMoE-2.6B-A0.6B-Instruct-2512-q4f16_1-MLC", {
  appConfig: {
    model_list: [
      {
        model: "https://huggingface.co/dstefl/EuroMoE-2.6B-A0.6B-Instruct-2512-q4f16_1-MLC",
        model_id: "EuroMoE-2.6B-A0.6B-Instruct-2512-q4f16_1-MLC",
        model_lib:
          "https://huggingface.co/dstefl/EuroMoE-2.6B-A0.6B-Instruct-2512-q4f16_1-MLC/resolve/main/EuroMoE-2.6B-A0.6B-Instruct-2512-q4f16_1-MLC-webgpu.wasm",
        overrides: { context_window_size: 4096 },
      },
    ],
  },
});

How it was built

Same from-source pipeline as our EuroLLM-1.7B conversion (TVM + mlcllm built from source, emsdk web runtime, `q4f161 convertweight → genconfig → compile`). The MoE architecture converted on the first attempt — 91 minutes of CI wall-clock, 2026-08-30.

Honest quality note

The upstream card publishes no benchmark numbers and states the model "has not been aligned to human preferences". We could not measure output quality ourselves either: generation never starts, because the model library fails at shader compilation (see the warning at the top). The weights themselves are a faithful q4f16_1 quantization of the original.

License & attribution

Base model © UTTER project, released under Apache 2.0; this quantized conversion is distributed under the same license. Not affiliated with the UTTER project.