penta2himajin/MiLMMT-46-1B-v0.1-6bit

MiLMMT-46-1B-v0.1 — MLX 6-bit

Apple MLX weights for `xiaomi-research/MiLMMT-46-1B-v0.1`, Xiaomi's 46-language translation model (a translation fine-tune of Gemma 3 1B).

This is the MLX build to use. The 4-bit sibling is 269 MB smaller but visibly degraded — see below.

Why 6-bit and not 4-bit

Naive 4-bit post-training quantization measurably damages this model. Every regression it introduces disappears at 6 bits, with no calibration step of any kind. Both builds were run on the same 16-sentence JA↔EN set, on the same machine, with a byte-identical prompt and greedy decoding, so the difference is attributable to the quantization rather than to the harness:

Small models are disproportionately sensitive to naive PTQ. Calibrated quantization (mlx_lm.dwq, mlx_lm.awq) might reach this quality at 4-bit size, but was not needed to fix the regressions and is untried here.

Versus the GGUF

At equal size, this build is competitive with `penta2himajin/MiLMMT-46-1B-v0.1-Q4_K_M-GGUF` on quality and throughput, and loses on cold load — MLX has no mmap equivalent, so it pays for the whole file on every load. Timings are from a battery-powered run; read them as ratios, not absolutes.

On quality the two trade places sentence by sentence. One example where this build is the better of the two: "The cabinet approved the basic policy…" renders as 「内閣は…承認した」here, against the GGUF's ungrammatical 「閣議決定は…承認した」.

Pick MLX if you are already on an MLX stack; pick the GGUF if cold load matters (an app that evicts weights between uses, for instance).

Prompt format

MiLMMT ships no chat template. Do not apply one — prompt it raw, exactly as the upstream model card specifies:

Translate this from <source language name> to <target language name>:
<source language name>: <source sentence>
<target language name>:

Use the language names from the upstream card's supported-language list (e.g. Japanese, English). Recommended decoding: top_k=1, temperature=0.

pip install mlx-lm
PROMPT=$'Translate this from Japanese to English:\nJapanese: こんにちは、今日も一日頑張りましょう。\nEnglish:'
mlx_lm.generate --model penta2himajin/MiLMMT-46-1B-v0.1-6bit \
  --prompt "$PROMPT" --max-tokens 128 --temp 0.0

Because there is no turn marker, generation can run on into a second translation block. Stop on the string Translate this from, or trim at it.

Conversion

mlx_lm.convert --hf-path xiaomi-research/MiLMMT-46-1B-v0.1 \
  -q --q-bits 6 --mlx-path MiLMMT-46-1B-v0.1-6bit

Converted with mlx-lm 0.31.3.

License

Gemma Terms of Use, inherited from the base model — not Apache 2.0. MiLMMT-46-1B is derived from Gemma 3, so redistribution and use are governed by the Gemma Terms and the Gemma Prohibited Use Policy, which pass through to anyone who obtains these weights.

Citation

@misc{shang2026scalingmodeldatamultilingual,
      title={Scaling Model and Data for Multilingual Machine Translation with Open Large Language Models},
      author={Yuzhe Shang and Pengzhi Gao and Wei Liu and Jian Luan and Jinsong Su},
      year={2026},
      eprint={2602.11961},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2602.11961},
}