Modelpublic

rapid-mlx/Ling-3.0-tiny-MLX-4bit

sourceHugging Facemitupdated 1mo agoView on Hugging Face

6likes3.7kdownloads

Model Card

Ling-3.0-tiny — MLX 4-bit

The first MLX conversion of inclusionAI/Ling-3.0-tiny: a 7.9B-total / 1.3B-active sparse-MoE reasoner (128 experts, top-8 + 1 shared) with a KDA + MLA hybrid attention stack and 131K context, MIT licensed.

4.2 GB at 4.507 bits/weight — it fits and runs on an 8 GB Apple Silicon Mac.


Quantization	4-bit, group size 64 (router kept 8-bit, short-conv weights fp)
Size on disk	4.2 GB
Context	131,072 tokens
Active parameters	1.3B per token
License	MIT (inherited from the base model)

Serve it

The bailing_hybrid architecture is not in upstream mlx-lm yet — this checkpoint is served by rapid-mlx, which ships a verified native implementation (reference parity 1.5e-6 against the official modeling code):

bash

pip install -U rapid-mlx   # 0.12.10 or newer
rapid-mlx serve ling-3.0-tiny-4bit

You get an OpenAI-compatible server on localhost:8000 with reasoning (reasoning_content) and tool calling parsed natively — thinking is controlled with chat_template_kwargs: {"enable_thinking": true} or the model's detailed thinking on/off system-prompt switch.

Once mlx-lm gains native bailing_hybrid support, this checkpoint will load there unchanged.

Conversion provenance

Converted with mlx_lm.convert (quantize=True, qbits=4, qgroupsize=64) running rapid-mlx's vendored `bailinghybrid implementation ([PR #1817](https://github.com/raullenchai/Rapid-MLX/pull/1817)), which was verified against the official modelingbailingmoe_v3.py` on identical random weights to a max logits deviation of 1.5e-6 (full prefill) / 1.9e-6 (token-by-token incremental) before conversion. End-to-end chat / reasoning / tool-call behaviour validated on an M2 Pro Mac mini.