CoolFace
Modelpublic

batiai/Mistral-Small-4-119B-GGUF

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face
0likes140downloads
Model Card

Mistral Small 4 119B-A6B GGUF — Quantized by BatiAI

Mistral's unified open-weight model — reasoning + multimodal + agentic coding in one. 119B Mixture-of-Experts with only 6B active per token, so it runs at small-model speed while keeping frontier quality. Apache 2.0 (fully commercial-friendly, no gating).

Quantized directly from [official Mistral weights](https://huggingface.co/mistralai/Mistral-Small-4-119B-2603) — not a re-quant of someone else's GGUF. Signed with BatiAI metadata for BatiFlow.

Quick Start

ollama run batiai/mistral-small-4:q4

Available Quantizations

QuantSizeRAM targetRecommended For
Q3_K_M54GB64GB MacCompact
Q4_K_M68GB96GB MacRecommended (balance)
Q5_K_M79GB128GB MacMax quality
119B total params → these are for 64GB+ Macs (M-series Max/Ultra). The 6B active means inference is fast despite the size. IQ3/IQ4 (smaller, imatrix) can be added on request.

RAM Requirements

Your Mac RAMQ3 (54GB)Q4 (68GB)Q5 (79GB)
64GB✅ tight❌❌
96GB✅✅❌ tight
128GB✅✅✅
192GB+✅✅✅ comfortable

Why Mistral Small 4?

  • —One model, three jobs — reasoning, multimodal understanding, and agentic coding unified (no model-switching).
  • —6B active / 119B total MoE — frontier-class capability at the inference speed/cost of a small model.
  • —Apache 2.0 — no license friction, no gating. Build commercial products freely.
  • —Native llama.cpp support — Mistral ships official GGUF tooling; arch (mistral3) is mainstream.

Why BatiAI Quantization?

  • —Original-source — quantized straight from Mistral's official weights, not a copy of a third-party GGUF.
  • —BatiAI-signed — general.author: BatiAI, general.url: https://flow.bati.ai.
  • —Mac-tuned selection — quant sizes chosen for real Apple Silicon RAM tiers.

Technical Details

  • —Original Model: mistralai/Mistral-Small-4-119B-2603
  • —Architecture: mistral3 MoE, 119B total / ~6B active per token
  • —License: Apache 2.0
  • —Quantized with: llama.cpp (Q8_0 intermediate → K-quants, --allow-requantize)
  • —Quantized by: BatiAI
  • —Note: YaRN config verified clean (no yarn_log_multiplier bug, unlike the earlier Medium 3.5 release).

About BatiFlow

BatiFlow — free, on-device AI automation for Mac. 5MB app, 100% local, unlimited. 60+ tools.

License

Quantized from mistralai/Mistral-Small-4-119B-2603. License: Apache 2.0.

Benchmarks

<!-- BENCH-START --> Mac 하드웨어 실측 벤치 대기 중 (bench.sh). 측정 후 자동 갱신. <!-- BENCH-END -->