Modelpublic

mlx-community/Inkling-Small-mxfp4

sourceHugging Faceupdated 2mo agoView on Hugging Face

4likes308downloads

Model Card

Inkling Small MXFP4

MXFP4 version of thinkingmachines/Inkling-Small.

This checkpoint uses less memory than thinkingmachines/Inkling-Small-NVFP4, but quantizes more tensors to MXFP4. The official NVFP4 checkpoint, on the other hand, only applies 4-bit quantization to routed experts, so quality should be better.

For the best quality, please run the official NVFP4 checkpoint directly with MLX, there's no need to convert:

bash

mlx_vlm.generate --prompt "who are you?" --model thinkingmachines/Inkling-Small-NVFP4

Or, if you want to try the version in this repo (lower memory consumption, faster):

bash

mlx_vlm.generate --prompt "who are you?" --model mlx-community/Inkling-Small-mxfp4

Note: please make sure you use the Inkling Small mlx-vlm PR.