Modelpublic

penta2himajin/MiLMMT-46-1B-v0.1-4bit

sourceHugging Facegemmaupdated 2mo agoView on Hugging Face

0likes37downloads

Model Card

MiLMMT-46-1B-v0.1 — MLX 4-bit

Apple MLX weights for `xiaomi-research/MiLMMT-46-1B-v0.1`, Xiaomi's 46-language translation model (a translation fine-tune of Gemma 3 1B).


Quantization	4-bit, `--q-bits 4` (4.501 bits per weight as reported by the converter)
Size on disk	731 MB
Runtime	`mlx-lm` / mlx-swift-lm
`model_type`	`gemma3_text`

Read this before using it

Use [`penta2himajin/MiLMMT-46-1B-v0.1-6bit`](https://huggingface.co/penta2himajin/MiLMMT-46-1B-v0.1-6bit) instead.

Naive 4-bit measurably damages this model, and every regression it introduces disappears at 6 bits — no calibration, just --q-bits 6. The 6-bit build is 269 MB larger and is the one you want. This repository stays up so the comparison remains reproducible, not because it is a good artifact.

The Q4KM GGUF, `penta2himajin/MiLMMT-46-1B-v0.1-Q4_K_M-GGUF`, is also a better choice than this build, and loads ~4× faster than either MLX build because llama.cpp mmaps its weights.

Both were run on the same 16-sentence JA↔EN set, on the same machine, with a byte-identical prompt (the format below) and greedy decoding, so the difference is attributable to the quantization rather than to the harness. Timings are from a battery-powered run and are useful as a ratio, not as absolute figures.

	GGUF Q4_K_M	MLX 4-bit
Disk	1.01 GB	731 MB
Cold load	366 ms	2034 ms
p50 latency	248 ms	269 ms
Throughput	215 chars/s	153 chars/s

Regressions this conversion introduces, none of which the GGUF produces:

Source	GGUF Q4_K_M	MLX 4-bit
来年度の予算編成	"next fiscal year's budget"	"fiscal 2022" — hallucinated year
のぞみ1号 (JA→EN)	"Nozomi 1 train"	"the Shinkansen" — train name lost
"enqueues an async task"	「非同期タスクを並列に並べ」	「無動作タスクをエスケープして」
"I hope this message finds you well"	「皆様に喜ばれることを願っています」	「今週のご挨拶」

Small models are disproportionately sensitive to naive post-training quantization. Raising the bit width to 6 closes the gap completely — every row in the table above is fixed in the 6-bit build, which lands at 1.0 GB and 204 chars/s. Calibrated quantization (mlx_lm.dwq, mlx_lm.awq) might reach that quality at this size, but was not needed and is untried.

Prompt format

MiLMMT ships no chat template. Do not apply one — prompt it raw, exactly as the upstream model card specifies:

text

Translate this from <source language name> to <target language name>:
<source language name>: <source sentence>
<target language name>:

Use the language names from the upstream card's supported-language list (e.g. Japanese, English). Recommended decoding: top_k=1, temperature=0.

bash

pip install mlx-lm
PROMPT=$'Translate this from Japanese to English:\nJapanese: こんにちは、今日も一日頑張りましょう。\nEnglish:'
mlx_lm.generate --model penta2himajin/MiLMMT-46-1B-v0.1-4bit \
  --prompt "$PROMPT" --max-tokens 128 --temp 0.0

Because there is no turn marker, generation can run on into a second translation block. Stop on the string Translate this from, or trim at it.

Conversion

bash

mlx_lm.convert --hf-path xiaomi-research/MiLMMT-46-1B-v0.1 \
  -q --q-bits 4 --mlx-path MiLMMT-46-1B-v0.1-4bit

Converted with mlx-lm 0.31.3.

License

Gemma Terms of Use, inherited from the base model — not Apache 2.0. MiLMMT-46-1B is derived from Gemma 3, so redistribution and use are governed by the Gemma Terms and the Gemma Prohibited Use Policy, which pass through to anyone who obtains these weights.

Citation

bibtex

@misc{shang2026scalingmodeldatamultilingual,
      title={Scaling Model and Data for Multilingual Machine Translation with Open Large Language Models},
      author={Yuzhe Shang and Pengzhi Gao and Wei Liu and Jian Luan and Jinsong Su},
      year={2026},
      eprint={2602.11961},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2602.11961},
}