dystrio/MiniCPM-o-4_5-Sculpt-Throughput

MiniCPM-o 4.5 — Sculpt Throughput (keep_frac=0.82)

18% compression — moderate quality tradeoff

Structurally pruned from openbmb/MiniCPM-o-4_5 using Dystrio Sculpt. Only the Qwen3-8B LLM backbone is pruned — vision (SigLip2), audio (Whisper), and TTS (CosyVoice2) modules are untouched.

Quality (Downstream Probe — 250 questions)

Compression Details

• —keep_frac: 0.82 (18% of MLP intermediate neurons removed)
• —Method: Structural pruning with live teacher distillation (alpha=0.5)
• —Repair: Full repair pass with workload-matched training data
• —Architecture: All multimodal modules preserved; only LLM MLP layers compressed

Intended Use

Drop-in replacement for MiniCPM-o 4.5 with reduced memory footprint. Suitable for:

• —LoRA fine-tuning on memory-constrained GPUs
• —File description and indexing workloads
• —Multimodal inference with lower VRAM requirements

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "dystrio/MiniCPM-o-4_5-Sculpt-Throughput",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
    "dystrio/MiniCPM-o-4_5-Sculpt-Throughput",
    trust_remote_code=True,
)