CoolFace
Modelpublic

Tdamre/MiniCPM5-1B-litert-lm

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face
3likes2.3kdownloads
Model Card

MiniCPM5-1B LiteRT-LM

This repository contains LiteRT-LM conversions of openbmb/MiniCPM5-1B for local on-device inference.

The source model is a Llama-architecture dense 1B checkpoint intended for on-device and resource-constrained use. These artifacts were exported locally from the Hugging Face safetensors checkpoint with LiteRT Torch and packaged as .litertlm files for the LiteRT-LM runtime.

Files

FileContext cacheQuantizationBackend targetStatus
MiniCPM5-1B.litertlm4096dynamic_wi8_afp32CPU/GPUHost CPU smoke-tested.
MiniCPM5-1B-c32k.litertlm32768dynamic_wi8_afp32CPU/GPULong-context export (32k cache). Host-packaged with mobile template.
MiniCPM5-1B-c64k.litertlm65536dynamic_wi8_afp32CPU/GPULong-context export (64k cache). Host-packaged with mobile template.
MiniCPM5-1B-web.litertlm2048dynamic_wi8_afp32CPU/GPUHost CPU, Android CPU/GPU, and Android emulator seven-turn chat smoke-tested.
MiniCPM5-1B-qualcomm-sm8750.litertlm2048dynamic_wi8_afp32Qualcomm SM8750 NPU AOTGenerated and loaded by the NPU runtime, but not inference-passing on the stock Redmagic shell because QNN cannot reserve the required PD memory.
MiniCPM5-1B-qualcomm-sm8750-c1024.litertlm1024dynamic_wi8_afp32Qualcomm SM8750 NPU AOTReduced-cache AOT attempt. Also loads, but still fails QNN context creation from stock shell.
chat_template.jinjan/an/an/aMobile-safe ChatML template. Replaces the source template's Python string methods that fail in Android LiteRT-LM template evaluation.
conversion_manifest.jsonn/an/an/aToolchain versions, hashes, and conversion details.

The CPU/GPU .litertlm files include a compressed Hugging Face tokenizer, LLM metadata, a quantized prefill/decode TFLite model, and a quantized external embedder. The Qualcomm files replace the prefill/decode model with an SM8750 AOT-compiled NPU model and keep the same external embedder packaging.

The May 31, 2026 refresh rebuilds all .litertlm containers with a minimal mobile-compatible ChatML template embedded in LLM metadata. This fixes Android template failures such as unknown method: string has no method named strip.

Run With LiteRT-LM

Install the LiteRT-LM CLI:

bash
uv tool install litert-lm

Run the generic artifact:

bash
litert-lm run \
  --from-huggingface-repo Tdamre/MiniCPM5-1B-litert-lm \
  MiniCPM5-1B.litertlm \
  --backend=cpu \
  --prompt="What is 2+2?"

Run the lower-cache artifact:

bash
litert-lm run \
  --from-huggingface-repo Tdamre/MiniCPM5-1B-litert-lm \
  MiniCPM5-1B-web.litertlm \
  --backend=cpu \
  --prompt="What is 2+2?"

Run the 32k-context artifact:

bash
litert-lm run \
  --from-huggingface-repo Tdamre/MiniCPM5-1B-litert-lm \
  MiniCPM5-1B-c32k.litertlm \
  --backend=cpu \
  --prompt="What is 2+2?"

Run the 64k-context artifact:

bash
litert-lm run \
  --from-huggingface-repo Tdamre/MiniCPM5-1B-litert-lm \
  MiniCPM5-1B-c64k.litertlm \
  --backend=cpu \
  --prompt="What is 2+2?"

Android Validation

Test device:

text
Nubia Redmagic 10 Pro / NX789J
Android 16, API 36
SoC: Qualcomm SM8750
GPU: Adreno 830
ABI: arm64-v8a

Android runner:

text
LiteRT-LM v0.12.0 source build
Bazel 7.6.1 via Bazelisk
Android NDK r28b
Target: //runtime/engine:litert_lm_main --config=android_arm64

Smoke prompt:

text
What is 2+2? Answer with just the number.

Results:

ArtifactBackendResultNotes
MiniCPM5-1B-web.litertlmAndroid CPU/XNNPACKReturned 4.Init 1877.13 ms, TTFT 0.38 s, prefill 61.13 tok/s, decode 50.77 tok/s.
MiniCPM5-1B-web.litertlmAndroid GPU/OpenCLReturned 4.LiteRT GPU loaded, all model subgraphs delegated to LITERT_CL. Init 8485.10 ms, TTFT 0.18 s, prefill 143.54 tok/s, decode 44.03 tok/s.
MiniCPM5-1B-web.litertlmAndroid emulator API 35 CPU/XNNPACKReturned final 6 for reggin.x86_64 LiteRT-LM Android runner, Pixel-style emulator, repaired embedded template.
MiniCPM5-1B-web.litertlmAndroid emulator API 35 CPU/XNNPACKSeven-turn chat completed.Same Conversation instance; final turn confirmed BLUE-IRIS and 17, and the reggin turn returned 6.
MiniCPM5-1B-qualcomm-sm8750.litertlmQualcomm NPU/Dispatch/QNNNot inference-passing.Dispatch and QNN load, V79 HTP skel is selected, context binaries match QNN SDK 2.44.0, then QNN fails context creation with context-size estimate 9260401152 bytes.
MiniCPM5-1B-qualcomm-sm8750-c1024.litertlmQualcomm NPU/Dispatch/QNNNot inference-passing.Reduced-cache variant lowers the context-size estimate to 5909219840 bytes, but QNN still reports no available PD for context creation.

The Qualcomm AOT artifacts are included for reproducibility and follow-up runtime work. They should not be treated as validated NPU inference artifacts for this stock Redmagic shell environment yet.

Conversion Summary

Source model revision:

text
openbmb/MiniCPM5-1B@4e9de7a0778dc1c362e983e6858f0e77542cbdca

CPU/GPU conversion command pattern:

bash
litert-torch export_hf \
  model-cache/openbmb-MiniCPM5-1B \
  <output_dir> \
  --keep_temporary_files=True \
  --prefill_lengths=128,1024 \
  --cache_length=<2048-or-4096-or-32768-or-65536> \
  --externalize_embedder=True \
  --quantization_recipe=dynamic_wi8_afp32

Qualcomm AOT conversion used ai_edge_litert.aot targeting SocManufacturer.QUALCOMM and SocModel.SM8750, then repackaged the compiled TFLite model into the .litertlm container with backend_constraint=npu.

Toolchain:

text
Python 3.12.12
litert-torch 0.9.1
litert-lm 0.12.0
ai-edge-litert 2.1.5
ai-edge-litert-sdk-qualcomm 2.1.5
ai-edge-quantizer 0.7.0
torch 2.12.0+cu130
transformers 5.9.0
QNN SDK/runtime 2.44.0 from the Qualcomm LiteRT SDK wheel

Not Included

MediaPipe .task was not uploaded. The current MediaPipe GenAI bundler documents SentencePiece tokenizer.model support, while MiniCPM5-1B ships an HF tokenizer.json and no tokenizer.model; uploading an unverified .task would be misleading.

Intel NPU AOT was not produced in this local run because this request targeted the connected Qualcomm SM8750/Adreno phone.