CoolFace
Modelpublic

stevenpr/chaty-qwen3.5-4b-design-GGUF

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face
1likes230downloads
Model Card

Chaty · Qwen3.5-4B Design (GGUF)

A small, on-device model fine-tuned to give **Chaty** — a private, fully-local desktop chat app — stronger single-file web/HTML design on weak hardware, while self-identifying as Chaty and following Chaty's retrieval-grounding contract.

  • —Base model: `Qwen/Qwen3.5-4B` (Apache-2.0)
  • —Created by: Fangyuan Lin (author of Chaty)
  • —Format: GGUF, Q4_K_M quant — runs in llama.cpp, LM Studio, or Chaty
  • —Languages: English + 简体中文

What it's for

Chaty runs entirely on the user's own device (no cloud). This model is tuned so that weak-hardware users get better-looking, leaner web pages, faster — and so Chaty's assistant behaves consistently:

  • —Web design — concise, self-contained single-file HTML pages (premium typography, dark/editorial styles), ~20–30% fewer tokens than the base ⇒ faster generation on weak hardware.
  • —Identity — reliably self-identifies as Chaty (a fine-tuned Qwen3.5-4B running locally), created by Fangyuan Lin; uses no version number.
  • —RAG grounding — when given numbered passages, cites with 【n】 and refuses ("not mentioned") when the answer is absent.

How it was made

LoRA fine-tune (Unsloth, bf16) of Qwen/Qwen3.5-4B, then exported to GGUF and quantized to Q4_K_M. Training data (≈760 examples):

  • —Web design distilled from a local Qwen3.6-35B-A3B teacher (concise-but-complete pages, rendered & QC-filtered with a headless-Chrome / CDP probe).
  • —RAG grounding contracts (【n】 citations + graceful refusals).
  • —Identity distilled from the same 35B teacher under a Chaty persona (bilingual).

Evaluation (honest)

Held-out sets; base = Qwen3.5-4B, heuristic-scored (no LLM judge).

Dimensionbasethis model
Self-identifies as Chaty0%94%
RAG 【n】 citation compliance100%100%
RAG refusal when answer absent100%100%
Design: clean render (0 JS errors + real content)56%59%
Design: self-contained single file89%97%
Avg tokens / page (↓ = faster on weak HW)~6000~4700

Trade-off: the gain is a more refined, leaner design style + consistent Chaty behavior; the cost is occasional content-fidelity slips on data-heavy interactive widgets (e.g. decimal-price formatting, complex JS logic). It is a 4B model specialized for Chaty's workflow, not a general chat assistant.

Run it

llama.cpp:

bash
llama-cli -m chaty-qwen3.5-4b-design-Q4_K_M.gguf -p "Design a pricing page for a meditation app."

Or load the GGUF in LM Studio, or drop it into Chaty's models folder.

License

Apache-2.0, inherited from the base model Qwen/Qwen3.5-4B.