Modelpublic

bsisduck/North-Mini-Code-1.0-MLX-MXFP8

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

1likes47downloads

Model Card

North-Mini-Code-1.0 MLX MXFP8

MXFP8 MLX conversion of `CohereLabs/North-Mini-Code-1.0`.

—Source revision: effaeda477c041c107d5a3d8c599cb5d6c5878ef
—Architecture: Cohere2MoeForCausalLM / cohere2_moe
—Quantization: MLX mxfp8, group size 32, 8 bits
—Artifact size: 32.36 GB, 7 safetensor shards
—Verification: conversion completed and all safetensor headers were readable
—Local runtime on M2 Max 32 GB: failed with Metal out-of-memory

Requires pinned experimental MLX-LM cohere2_moe support until it lands in a release:

bash

pip install "mlx-lm @ git+https://github.com/Terrencezzj/mlx-lm.git@f43507c5c30bdebdb92d308ac11aa8f96b418c2e"