Modelpublic

abenzerps/MiniCPM5-2B-GGUF

sourceHugging Faceapache-2.0updated 15d agoView on Hugging Face

11likes7.6kdownloads

Model Card

MiniCPM5-2B GGUF

GGUF quantizations of OpenBMB/MiniCPM5-2B, a 2B dense Llama-based model for local deployment, coding, reasoning, long-context work, and tool use. The source checkpoint supports a native context length of 131,072 tokens (128K).

Benchmarks

[image]

Benchmark results reported by OpenBMB for the original MiniCPM5-2B checkpoint.

[image]

Capability comparison reported by OpenBMB for the original MiniCPM5-2B checkpoint.

GGUF files

Quantization	File	Size
Q2_K	MiniCPM5-2B-Q2_K.gguf	1.04 GB
Q3KM	MiniCPM5-2B-Q3_K_M.gguf	1.29 GB
Q4_0	MiniCPM5-2B-Q4_0.gguf	1.49 GB
Q4KS	MiniCPM5-2B-Q4_K_S.gguf	1.50 GB
Q4KM	MiniCPM5-2B-Q4_K_M.gguf	1.56 GB
Q5KM	MiniCPM5-2B-Q5_K_M.gguf	1.81 GB
Q6_K	MiniCPM5-2B-Q6_K.gguf	2.07 GB
Q8_0	MiniCPM5-2B-Q8_0.gguf	2.68 GB
IQ2_M	MiniCPM5-2B-IQ2_M.gguf	0.97 GB
IQ3_M	MiniCPM5-2B-IQ3_M.gguf	1.23 GB
IQ4_XS	MiniCPM5-2B-IQ4_XS.gguf	1.42 GB

The model is text-only. No vision projector or MTP files are included. The IQ files use an importance matrix generated from WikiText-2 and are intended for recent llama.cpp builds. SHA-256 checksums are provided in `SHA256SUMS.txt`.

Chat template

The GGUF files embed the upstream chat template. `chat_template.jinja` is provided as an external copy for runtimes that require a separate template file.

Usage

Use a current llama.cpp build with MiniCPM5 support. The example below uses an 8K context; increase -c up to 131072 when sufficient memory is available.

bash

llama-cli \
  -m MiniCPM5-2B-Q4_K_M.gguf \
  -c 8192 --jinja \
  --temp 1.0 --top-p 0.95 \
  -p "Explain why reproducible builds matter."

For an OpenAI-compatible server:

bash

llama-server \
  -m MiniCPM5-2B-Q4_K_M.gguf \
  -c 8192 --jinja --host 0.0.0.0 --port 8080

Tool-call behavior depends on the serving runtime's parser and API integration; use the embedded template and verify tool calls in the target application.

Source

—Model: OpenBMB/MiniCPM5-2B
—Source revision: `3497c460c89e00520c3cfa2e73f49ab7647f1177`
—Conversion: the original Q40–Q80 files use upstream llama.cpp commit f114f91f9ed6792cf402437e3874adad98902744; the additional Q2K, Q3KM, Q4KS, IQ2M, IQ3M, and IQ4XS files use upstream commit 67672dc5b76f8bc17785a19d3dc6d1463fc2902c
—License: Apache-2.0
—Checksums: SHA256SUMS.txt