UraionLabs/MiniCPM5-2B-oQ4e

<p align="center"> <img src="https://uraionlabs.com/public/icons/icon-192.png" alt="Uraion Labs" width="56"> </p>

<p align="center"> <strong>Uraion Labs</strong><br> _{Foundational systems research.} </p>

MiniCPM5-2B — oQ4e

An MLX mixed-precision quantization of OpenBMB/MiniCPM5-2B for local inference on Apple Silicon, published by Uraion Labs.

MiniCPM5-2B-oQ4e is an MLX mixed-precision quantization of OpenBMB/MiniCPM5-2B optimized for local inference on Apple Silicon Macs. It was quantized and published by Uraion Labs using the oMLX oQe quantization workflow with importance-matrix guided sensitivity allocation. The checkpoint uses the standard MLX format and is directly compatible with both oMLX and mlx-lm.

About This Quantization

• —Quantization Level: oQ4e
• —Quantization Workflow: oMLX oQe mixed-precision quantization
• —Base Weight Precision: 4-bit
• —Mixed-Precision Profile: Mixed 4/5/6-bit precision (58 @ 5b, 9 @ 6b)
• —Layer Overrides: 67 overrides: 58 layers boosted to 5-bit, 9 layers boosted to 6-bit across attention and MLP projections
• —Output Head (`lm_head`) Precision: 4-bit
• —Group Size: 64
• —Quantization Mode: Affine (scale and bias per group)
• —Non-Quantized Tensor Precision: BF16 (LayerNorm weights, embedding scales and biases)
• —Calibration Dataset: oqe_code_multilingual (294 calibration samples)
• —Model File Size: 1.38 GB (1,410.44 MB)
• —Effective Bits Per Weight: Effective ~4.4 bits per weight balancing compression with selective precision allocation.

Model Information

Upstream MiniCPM5-2B Highlights

MiniCPM5-2B is developed by OpenBMB as the 2B-scale model in the MiniCPM5 series:

• —2B-Class Open-Source State of the Art: Reaches top-tier performance among models in its size category (averaging 53.9 across OpenBMB's evaluation set) while remaining competitive with 3B and 4B class models.
• —Native 131k Context: Supports up to 131,072 tokens context window out of the box for document synthesis, repo-wide code reasoning, and long conversation threads.
• —Standard LLaMA Architecture: Uses standard LlamaForCausalLM with Grouped-Query Attention (GQA, 16 Q heads, 2 KV heads), enabling out-of-the-box execution across standard inference runtimes.
• —High-Quality Training Datasets: Trained using OpenBMB's UltraData data curriculum, including Ultra-FineWeb, UltraX, UltraData-Code, UltraData-SFT, and UltraData-RL.
• —Agentic and Tool-Use Capabilities: Engineered for high accuracy on function calling, structured output generation, and coding assistant workflows.

Quantization Family Navigation

Quickstart

Using oMLX

oMLX provides a high-throughput runtime for MLX models on Apple Silicon:

# Install oMLX
pip install omlx

# Run inference
omlx run UraionLabs/MiniCPM5-2B-oQ4e --prompt "Explain quantum entanglement in simple terms."

Using mlx-lm

This quantization is fully compatible with Apple's standard mlx-lm library:

# Install mlx-lm
pip install mlx-lm

Python Example

from mlx_lm import load, generate

model, tokenizer = load("UraionLabs/MiniCPM5-2B-oQ4e")

prompt = "Explain quantum entanglement in simple terms."
messages = [{"role": "user", "content": prompt}]
prompt_formatted = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)

response = generate(
    model,
    tokenizer,
    prompt=prompt_formatted,
    max_tokens=512,
    temp=1.0,
    top_p=0.95,
    verbose=True,
)

CLI Example

python -m mlx_lm.generate \
  --model UraionLabs/MiniCPM5-2B-oQ4e \
  --prompt "Explain quantum entanglement in simple terms." \
  --temp 1.0 \
  --top-p 0.95 \
  --max-tokens 512

Hugging Face Download

To download the repository files locally:

hf download UraionLabs/MiniCPM5-2B-oQ4e

Recommended Generation Settings

The following sampling parameters are officially recommended by OpenBMB for MiniCPM5-2B:

When performing deterministic evaluation or greedy decoding, set temperature=0.0 or do_sample=false.

Evaluation Results and Benchmark Context

Benchmark results below are reported by OpenBMB for the original MiniCPM5-2B release. This Uraion Labs quantization has not been independently benchmarked unless explicitly stated otherwise.

Limitations and Disclaimer

This model generates text based on statistical language modeling patterns and may produce inaccurate, biased, or hallucinated statements. Responses regarding sensitive or specialized domains (such as law, medical treatment, or financial investment) must not be treated as professional advice.

This model is provided "AS IS", without warranty of any kind, express or implied. Users are responsible for verifying outputs, implementing safety guardrails, and complying with all applicable local regulations and acceptable use policies.

Original Model and Attribution

This repository contains a quantized derivative of OpenBMB/MiniCPM5-2B.

MiniCPM5-2B was developed and trained by OpenBMB. Model architecture, pre-training, instruction tuning, benchmarks, and capabilities described on this page originate from the upstream MiniCPM5-2B release.

Training-data metadata is inherited from the upstream OpenBMB/MiniCPM5-2B release. Uraion Labs performed quantization only and did not retrain this checkpoint.

Upstream Resources

• —Model Card: openbmb/MiniCPM5-2B
• —GitHub Repository: OpenBMB/MiniCPM
• —MiniCPM Technical Report: arXiv:2506.07900
• —UltraData Paper: arXiv:2602.09003
• —Online Demo: MiniCPM5-2B-Demo
• —OpenBMB MLX Deployment Guide: mlx.md

License

This repository and the underlying MiniCPM model weights are released under the Apache-2.0 License.

Citation

Please cite the original MiniCPM research when referencing this work:

@article{minicpm4,
  title={Minicpm4: Ultra-efficient llms on end devices},
  author={MiniCPM, Team},
  journal={arXiv preprint arXiv:2506.07900},
  year={2025}
}

<p align="center"> _{Quantized and published by <a href="https://uraionlabs.com">Uraion Labs</a>. Foundational systems research.} </p>