Modelpublic

cawalch/veyra-tuned-v2b-q4f16_1-MLC

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes

Model Card

veyra-tuned-v2b (q4f16_1, MLC)

QLoRA fine-tune of Qwen3-4B-Instruct-2507 for the Veyra process-authoring task: generating structured process definitions (states, transitions, guards, tasks, forms) from natural-language intent.

This repo holds the MLC/WebLLM q4f16_1 shards for in-browser inference via WebGPU. The GGUF (llama.cpp) and LoRA adapter live on the training host.

What it does

Input: a natural-language description of a business workflow.

Output: a JSON object conforming to the Veyra process-definition schema, including transition.guard_ast (structured predicate objects, not raw DSL strings).

The model is specialized to the Veyra definition shape: lowersnakecase keys, declare-before-reference, registered unit catalog keys, exactly one initial state, terminal states for outcomes, and structured guards.

Training


Base	Qwen/Qwen3-4B-Instruct-2507
Method	QLoRA (4-bit NF4, double quant)
LoRA rank	64 (alpha 128, dropout 0.05)
Target modules	q, k, v, o, gate, up, down proj
Epochs	3
LR	2e-4 cosine, 5% warmup
Effective batch	32 (per-device 1 x grad-accum 32)
Max length	4096
Loss masking	response-only
Dataset	709 examples (repair triples + generation gold + distilled), guard-AST distribution

Trained from base (not continued from v1). A continue-from-v1 attempt (v2a) failed — the model landed between representations (6.1% clean). Full analysis in the Step 8 results.

Qualification (114-intent holdout)

Metric	Target	v2b
Pass-1 parsed	>=90%	97.4%
Pass-1 clean	>=70%	75.4%
Converged valid	>=80%	93.0%
guard_invalid	<=5	5

Axis M (modify existing workflow, minimal-change fidelity): 93.3%.

Usage (WebLLM / browser)

Variant-reuses the mlc-ai/Qwen3-4B-q4f16_1-MLC WebGPU wasm (same architecture; no custom wasm compile needed).

import * as webllm from "https://esm.run/@mlc-ai/web-llm";

const engine = await webllm.CreateMLCEngine("Veyra-Qwen3-4B-v2b-q4f16_1-MLC", {
  appConfig: {
    model_list: [{
      model: "https://huggingface.co/cawalch/veyra-tuned-v2b-q4f16_1-MLC/resolve/main/",
      model_id: "Veyra-Qwen3-4B-v2b-q4f16_1-MLC",
      model_lib: "https://raw.githubusercontent.com/mlc-ai/binary-mlc-llm-libs/main/web-llm-models/v0_2_84/base/Qwen3-4B-q4f16_1_cs1k-webgpu.wasm",
      vram_required_MB: 3431.59,
      overrides: { context_window_size: 4096 },
    }],
  },
});

const response = await engine.chat.completions.create({
  messages: [{ role: "user", content: "Request paid time off; manager approves." }],
  temperature: 0,
});

Download: 2.2 GB one-time (IndexedDB cached after). Measured 14.9 tok/s decode on a discrete-GPU MacBook (Chrome, WebGPU). No response_format constraint — the GBNF grammar engine suppresses guard_ast emission; Parse+Validate+Repair on the server side is the safety net.

Usage (llama.cpp / server)

The GGUF Q4KM (2.5 GB) is served via llama-server. See the Veyra repo (llm/MODEL_HOST.md) for the serving config.

Limitations

—Domain-specific: optimized for Veyra process definitions, not general chat.
—No response_format / JSON schema constraint at inference (suppresses guard_ast; see note above). Output is validated post-hoc.
—Conditional routing (intake-bypass auto-approve guards) is emitted on ~60% of cases unconstrained; the model has a prior toward unconditional human-approval chains.
—Context window capped at 4096 for browser VRAM (native 262144).

Files

File	Purpose
`params_shard_*.bin` (74)	quantized weights (q4f16_1)
`mlc-chat-config.json`	MLC chat config (modelid, convtemplate: qwen3)
`ndarray-cache.json`	weight tensor manifest (symlink to tensor-cache.json)
`tensor-cache.json`	weight tensor manifest (mlc-llm name)

License

Apache 2.0 (inherited from Qwen3-4B-Instruct-2507 base model).