F-Labs/minicpm5-2b-hadamard-gsq-demo

MiniCPM5-2B FQuant Adaptive — CPU Chat Demo

Gradio chat demo for `F-Labs/MiniCPM5-2B-Hadamard-GSQ`, the calibration-aware adaptive FQuant release of MiniCPM5-2B. It uses deterministic Rademacher/Hadamard rotation, sensitivity-guided mixed precision, low-rank residual compensation, and KV-BSS. The current artifact is approximately 4.02 GiB across three shards versus the 4.69 GiB BF16 snapshot. It is an experimental quality- oriented release with a custom Transformers loader, not a stock llama.cpp model.

• —Quantization framework: FQuant
• —Runs on CPU in torch.bfloat16, no tokens or secrets required (public repo).
• —max_new_tokens capped at 256; inputs truncated to 2048 tokens to keep CPU inference responsive. The 128K window is an architecture property, not a demo promise.

Deploy to Hugging Face Spaces (cpu-basic)

1. 1.Create a Space: New Space → SDK `Gradio` → Hardware `CPU basic` (free tier).
2. 2.Upload these three files to the Space root:
3. 3.app.py
4. 4.requirements.txt
5. 5.README.md (this file — frontmatter above configures the Space)
6. 6.Keep hardware on cpu-basic (2 vCPU / 16 GB RAM). No secrets needed.
7. 7.Wait for the build. First build downloads ~2 GB of weights plus the CPU torch wheel, so expect several minutes on cold start.
8. 8.Open the Space URL and chat. If the model fails to load, the UI shows the exact error instead of crashing (check Space logs / network to huggingface.co).

Run locally

pip install -r requirements.txt
python app.py
# open http://127.0.0.1:7860

Resource fit (why a 16 GB CPU Space is enough)

• —Weights: ~4.02 GiB on disk / in BF16 CPU memory.
• —Estimated peak RSS: ~6–8 GB (weights + torch CPU runtime + Gradio + activations/KV cache for ≤2048 input tokens and ≤256 new tokens).
• —Limit on cpu-basic: 16 GB — comfortable headroom, no GPU needed.
• —Cold start is slow (weight download + one-time load); CPU generation is a functional demo path rather than a throughput benchmark.

Files