batiai/Mistral-Medium-3.5-128B-GGUF

Mistral Medium 3.5 128B GGUF — Quantized by BatiAI

<p align="center"> <a href="https://flow.bati.ai"><img src="https://img.shields.io/badge/BatiFlow-macOS%20AI%20Automation-blue?style=for-the-badge&logo=apple" alt="BatiFlow"></a> <a href="https://ollama.com/batiai/mistral-medium-3.5"><img src="https://img.shields.io/badge/Ollama-batiai%2Fmistral--medium--3.5-green?style=for-the-badge" alt="Ollama"></a> </p>

Quantizations of Mistral Medium 3.5 128B (Dense, Modified MIT) — frontier coding model on a Mac. Built and verified by BatiAI for BatiFlow.

Why Mistral Medium 3.5?

• —SWE-Bench Verified: 77.6% — second only to Gemini 3.1 Pro Preview (78.8%)
• —Tau3-Telecom: 91.4% — top-tier agentic benchmark
• —128B Dense — Mistral's first flagship merged model (instruction + reasoning + coding in one weight set)
• —Modified MIT — open weights, commercial-friendly
• —256K context window + multimodal vision capabilities
• —Configurable reasoning effort per request
• —Released April 29, 2026

Quick Start

ollama pull batiai/mistral-medium-3.5:iq4

Available Quantizations

RAM Requirements

Coding Powerhouse on a Mac

A coding model that goes head-to-head with cloud frontier (Gemini 3.1 Pro), runnable entirely on-device on a 128GB MacBook Pro.

Model Comparison — Frontier Class on Mac

Why BatiAI Quantization?

Technical Details

• —Original Model: mistralai/Mistral-Medium-3.5-128B
• —Architecture: Dense decoder-only, 128B params
• —License: Modified MIT (commercial-friendly)
• —Quantized with: llama.cpp + imatrix
• —Quantized by: BatiAI

Config fix applied (YaRN, 2026-06-16)

The upstream config bug (mscale_all_dim 1→0, fixed here) propagated to the GGUF metadata as mistral3.rope.scaling.yarn_log_multiplier = 1.0 (correct value 0.0). Since YaRN scaling is read from metadata at load time — not baked into weights — we patched the value in place; no re-quantization needed. All quants (IQ3XXS / IQ4XS / Q5KM) are patched and live (verified 0.0). Filenames unchanged — just re-download to pick up the fix. For long-context workloads, vLLM is the recommended runtime.

About BatiFlow

BatiFlow — free, on-device AI automation for Mac. 5MB app, 100% local, unlimited. 60+ tools — KakaoTalk, iMessage, Slack, Calendar, browser, file system.

License

Quantized from mistralai/Mistral-Medium-3.5-128B. License: Modified MIT.

Benchmarks

<!-- BENCH-START -->

<!-- BENCH-END -->