Modelpublic

sahilchachra/hy-mt2-1.8b-mxfp4-mlx

sourceHugging Faceotherupdated 4mo agoView on Hugging Face

0likes37downloads

Model Card

hy-mt2-1.8b-mxfp4-mlx

Quantized version of tencent/Hy-MT2-1.8B for Apple Silicon using MLX.

Hy-MT2-1.8B is Tencent's multilingual translation model covering 40+ languages.

Quantization: Block floating-point MX FP4 (microscaling) Precision: ~4 bits/weight Group size: 32 Disk size: 917 MB Quantized by: sahilchachra

About this variant

Microscaling (MX) block floating-point quantization at FP4 precision. Uses a shared floating-point exponent per block of 32 weights instead of integer affine scaling — different numerical properties vs affine int4.

Benchmark results

Evaluated on Apple M5 Pro with MLX. Model loaded once; performance and quality measured in a single pass.

Performance

	This model	FP16 baseline
Prefill (tok/s)	1475.39	1269.81
Decode (tok/s)	227.95	77.12
Peak memory (GB)	1.224	3.72
Disk size (MB)	917	3897

Translation quality (FLORES-200 devtest)

Reported as chrF++ (higher is better). Sample-size noted per pair.

Direction	This model	FP16 baseline	n
engLatn→fraLatn	64.09	63.81	20
engLatn→deuLatn	57.0	57.66	20
engLatn→zhoHans	26.22	29.09	20
engLatn→jpnJpan	31.15	34.19	20
engLatn→spaLatn	56.15	56.5	20
fraLatn→engLatn	63.38	64.58	20
zhoHans→engLatn	52.78	55.17	20
jpnJpan→engLatn	51.53	55.29	20

Avg chrF++: 55.46 vs FP16 56.95 Avg BLEU: 29.62 vs FP16 30.71

Context scaling (decode tok/s)

Context length	Decode tok/s
~128 tokens	223.0
~256 tokens	220.3
~512 tokens	220.8
~1024 tokens	217.3

Usage

Install

bash

pip install mlx-lm

Translate

python

from mlx_lm import load, generate

model, tokenizer = load("sahilchachra/hy-mt2-1.8b-mxfp4-mlx")

prompt = (
    "Translate the following text from English to French.\n"
    "English: The early bird catches the worm.\n"
    "French:"
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=128, verbose=True))

Stream

python

from mlx_lm import load, stream_generate

model, tokenizer = load("sahilchachra/hy-mt2-1.8b-mxfp4-mlx")
for chunk in stream_generate(model, tokenizer, prompt="Translate \"Hello world\" to Japanese:", max_tokens=64):
    print(chunk.text, end="", flush=True)

All variants in this collection

Model	Method
sahilchachra/hy-mt2-1.8b-4bit-mlx	Affine int4 (group 64)
sahilchachra/hy-mt2-1.8b-8bit-mlx	Affine int8 (group 64)
sahilchachra/hy-mt2-1.8b-mxfp4-mlx	Block float MX FP4 ← this model
sahilchachra/hy-mt2-1.8b-mxfp8-mlx	Block float MX FP8

Notes

—Requires Apple Silicon (M1 or later) with MLX
—Benchmarks run on Apple M5 Pro, 24 GB unified memory
—FLORES-200 sample sizes are small — treat chrF/BLEU figures as indicative, not definitive
—License: see tencent/Hy-MT2-1.8B for the original model's license terms

Original model

See tencent/Hy-MT2-1.8B for full model details, supported languages, and intended use.