CoolFace
Datasetpublic

prykin/flash-moe-weights

Flash-MoE Weights: Qwen3.5-397B-A17B (4-bit, Metal) Pre-packed weights for flash-moe — a pure C/Metal inference engine that runs the 397B-parameter Qwen3.5 MoE model on a single MacBook. Source Model mlx-community/Qwen3.5-397B-A17B-4bit File Structure File Size Description model_weights.bin 5.5 GB Non-expert weights (embeddings, attention, norms, shared expert, routing gates) model_weights.json 371 KB Tensor manifest (offsets, shapes… See the full description on the dataset page: https://huggingface.co/datasets/prykin/flash-moe-weights.

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face
0likes42downloads
Dataset Card

Flash-MoE Weights: Qwen3.5-397B-A17B (4-bit, Metal)

Pre-packed weights for flash-moe — a pure C/Metal inference engine that runs the 397B-parameter Qwen3.5 MoE model on a single MacBook.

Source Model

mlx-community/Qwen3.5-397B-A17B-4bit

File Structure

FileSizeDescription
model_weights.bin5.5 GBNon-expert weights (embeddings, attention, norms, shared expert, routing gates)
model_weights.json371 KBTensor manifest (offsets, shapes, dtypes)
packed_experts/layer_XX.bin3.4 GB x 60Per-layer expert weights (512 experts x 7,077,888 bytes each)
packed_experts/layout.json--Expert binary layout descriptor
vocab.bin2.2 MBVocabulary for token decoding (GPT-2 byte-decoded)
tokenizer.bin7.8 MBBPE tokenizer data
shaders.metal55 KBMetal compute shaders

Total: ~208 GB

Usage

bash
git clone https://github.com/danveloper/flash-moe
cd flash-moe/metal_infer
make

# Download weights (requires ~210 GB free space)
# Place this dataset's contents into a directory, e.g. ~/models/flash_mlx_4bit/

cd ~/models/flash_mlx_4bit
/path/to/flash-moe/metal_infer/infer --prompt "Explain quantum computing" --tokens 100

Hardware Requirements

  • —Apple Silicon Mac (M1/M2/M3/M4)
  • —24 GB+ unified memory (48 GB recommended for better page cache hit rate)
  • —~210 GB SSD space
  • —macOS 24+

Notes

  • —All weights are 4-bit quantized (affine, group_size=64)
  • —60 MoE layers, 512 experts per layer, K=4 active per token
  • —Expert weights stream from SSD on demand via pread()
  • —vocab.bin includes the GPT-2 byte-to-unicode reverse mapping fix