Modelpublic

Brooooooklyn/Agents-A1-mxfp4-mlx

sourceHugging Faceapache-2.0updated 24d agoView on Hugging Face

0likes191downloads

Model Card

Agents-A1 — Unsloth MXFP4 + MXFP8 (MLX)

`Brooooooklyn/Agents-A1-mxfp4-mlx` is an Apple Silicon MLX quantization of `InternScience/Agents-A1`. Agents-A1 is a 35B-A3B Qwen3.5-family mixture-of-experts agent model with 40 language-model layers, 256 experts with top-8 routing, hybrid linear/full attention, and an MTP-capable config. The published checkpoint does not include an mtp.* tensor subtree.

This model is part of the Qwen Unsloth tensor-class recipe for MLX on macOS and DGX collection.

The source was the BF16 checkpoint at revision `55100f11160f545dc45545c677699ace74f6bd10`.

Update — re-converted 2026-09-01

The MXFP4 and MXFP8 weights are re-encoded. mlx-node now picks each MX block's shared E8M0 exponent itself instead of rounding log2(amax) to nearest: MXFP4 tries both candidate exponents and keeps the lower squared error, and MXFP8 takes the ceiling so a block can never saturate. Measured MXFP8 weight error across a checkpoint drops from 6.91% to 2.66%.

Weight bytes changed; the tensor-class map, shard layout and tensor counts are unchanged. Earlier revisions remain in this repository's history.

Quantization recipe

This is a data-free, weight-only Apple translation of the Unsloth Qwen3.6 NVFP4 recipe:

—tensors assigned NVFP4 by the recipe are stored as MXFP4, 4-bit with group size 32;
—tensors assigned FP8 by the recipe are stored as MXFP8, 8-bit with group size 32;
—every excluded tensor stays BF16.

This ports the recipe's tensor-class selection. It does not claim numerical parity with NVIDIA NVFP4/FP8 activation quantization. Both MXFP classes use MLX weight-only quantized matmul with BF16/A16 activations. The artifact contains no activation-calibration or FP8 KV-cache-calibration data, global scales, or W4A4/W8A8 execution metadata. No imatrix, calibration dataset, or AWQ-style pre-scaling was used, and no upstream numerical or performance parity is claimed.

Tensor class	Stored format
Routed expert `switch_mlp.{gate,up,down}_proj`, layers 0–31	MXFP4 4/32
Shared expert `{gate,up,down}_proj`, layers 0–31	MXFP4 4/32
The same routed/shared FFN projections, layers 32–39	MXFP8 8/32
Full-attention `{q,k,v,o}_proj`	MXFP8 8/32
Linear-attention `in_proj_qkv`, `in_proj_z`, `out_proj`	MXFP8 8/32
`lm_head`	MXFP8 8/32
Embeddings; router `mlp.gate` and `shared_expert_gate`; `in_proj_a/b`; GDN state, convolution, and norm tensors; all other norms	BF16
MTP tensors	Not present in the published source checkpoint
Vision tower and merger tensors	BF16

The resulting config contains 192 MXFP4 module entries and 179 MXFP8 module entries. The final eight FFN layers are intentionally MXFP8; this is the regular, accuracy-oriented 35B recipe rather than the all-FFN-FP4 "Fast" variant.

Compatibility and usage

This checkpoint requires @mlx-node/lm and @mlx-node/core 0.0.8 or newer. The published prebuilt macOS binary requires macOS 26 or newer; macOS 14 or newer can use a source build.

bash

npm install @mlx-node/lm@^0.0.8 @mlx-node/core@^0.0.8

typescript

import { loadSession } from '@mlx-node/lm';

const session = await loadSession('./Agents-A1-mxfp4-mlx');
const result = await session.send('Reply briefly: what can you help with?');
console.log(result.text);

Reproduction

Converter: mlx-node at or after PR #131, which made the tuned MX weight encoders and the NVFP4 power-of-two lift unconditional. v0.0.8 reproduces the earlier revision of this repository, not the current weights. The normalized reproducible invocation from the mlx-node repository root was:

bash

mlx convert \
  --input .cache/models/agents-a1 \
  --output .cache/models/agents-a1-unsloth-mxfp4-mlx \
  --model-type qwen3_5_moe \
  --dtype bfloat16 \
  --quantize \
  --q-recipe unsloth \
  --q-bits 4 \
  --q-mxfp

Validation

A local mlx-node load-and-generate smoke test completed successfully and produced the one-token output OK. This functional smoke test is separate from the performance benchmark below and does not validate model quality.

Benchmark

Measured on an Apple M5 Max with 128 GiB of unified memory, Darwin 25.5.0/arm64, Node.js 24.13.1, and @mlx-node/lm, @mlx-node/core, and @mlx-node/core-darwin-arm64 0.0.8.

The table reports the median of three measured runs. Each run used a fresh child process and fresh model load, with zero warmups and a 60-second cooldown. Generation was deterministic: temperature: 0, reasoning effort none, 106 prompt tokens, and a 512-token generation limit. All three runs generated 512 tokens and ended with finishReason: "length".

Metric	Median
Model load	37.100 s
Time to first token	293.644 ms
Prefill throughput	360.982 tok/s
Decode throughput	81.422 tok/s
Generation wall time	6.711 s
End-to-end wall time	43.656 s

Raw per-run samples and the machine-readable summary are available in benchmark.json.

The model files were read from external storage. Load timing is sensitive to external-storage and OS file-cache state; the three measured loads ranged from 2.375 to 239.231 seconds. Treat load time as host- and cache-specific. These measurements describe only this prompt and configuration and must not be used to infer model quality.

License and attribution

The source model card declares the Apache-2.0 license. Model capability and training credit belong to InternScience. The tensor-class recipe is credited to Unsloth, and this repository only converts the BF16 source weights to an MLX-native mixed MXFP4/MXFP8 representation.