Modelpublic

mlx-community/Z1T-0-mlx

sourceHugging Faceapache-2.0updated 21d agoView on Hugging Face

0likes206downloads

Model Card

Z1T-0 (MLX)

MLX conversion of **Extropic-AI/Z1T-0** — the first open-weight Z1T, an attention-free, DyT-normed, fixed-sparsity decoder LM designed for Extropic's Z1 probabilistic hardware (research post, code).

The original is a JAX/Equinox checkpoint (.eqx) with a bespoke architecture (no softmax attention: causal depthwise conv + causal cumulative pool; tanh-linear fixed fan-in-4 sparse projections; Dynamic-Tanh as the only norm). This repo ports it to MLX and ships it as safetensors + a GPT-2 tokenizer.

Faithfulness: greedy decoding is token-for-token identical to the JAX reference; logits match to max|Δ|≈1.4e-2 (float32 accumulation-order noise), with 100% per-position argmax agreement.

Files

—model.safetensors — weights (float32, ~1.24B params; dominated by the 50257×12288 embed/unembed).
—config.json — model_type: z1t + architecture fields.
—tokenizer.json / vocab.json / merges.txt / tokenizer_config.json — standard GPT-2 BPE.
—mlx_z1t.py, run.py — self-contained MLX inference (no JAX, no mlx-vlm needed).

Usage — standalone (no dependencies beyond MLX)

bash

pip install mlx numpy tiktoken safetensors
python run.py "The meaning of life is" 60

Usage — via mlx-vlm

Requires the z1t model in mlx-vlm (mlx_vlm/models/z1t/):

python

from mlx_vlm import load, generate
model, processor = load("AlazarM/Z1T-0-mlx")
print(generate(model, processor, "The meaning of life is", max_tokens=60, temperature=0.0).text)

Notes

—This is a portability port (runs on Apple Silicon), not a throughput build. Prefill is ~150 µs/token on an M5 Max; single-token decode is unoptimized.
—The checkpoint ships a numerically-zero positional table (AFTConv already encodes position).
—The model is a small 4-layer research artifact; generations are short and repetitive.