Modelpublic

edbordin-linktree/decider-0.8b-executorch-mlx

sourceHugging Faceapache-2.0updated 3d agoView on Hugging Face

0likes7downloads

Model Card

decider-0.8b: compact ExecuTorch MLX export

Converted from Mapika/decider-0.8b, checkpoint revision 1ea54127d3bd52f6d753d9257b32a6380b873907. Original model, training, typed-decision interface and calibration: Mark Marosi / Mapika, based on Qwen3.5. No additional training or fine-tuning was performed for this export.

Execution changes and credits

FP16 weights/activations, FP32 recurrent state, fused projections/normalization, padding removal, and bounded parallel suffix recurrence. Prefixes may be shared within a request; no answers or state are cached across requests.

Export source: Decider MLX fork. PyTorch/ExecuTorch, Apple MLX, Qwen and Hugging Face Transformers supply the underlying implementation and runtime components. No upstream endorsement is implied. The Apache-2.0 license is included. These are export/runtime changes, not a newly trained model.

Download and call

Designed for local inference on Apple Silicon using the ExecuTorch MLX backend. Install the MLX runtime and fork, then:

python

from decider.mlx_backend import MLXDecider

model = MLXDecider.from_pretrained("edbordin-linktree/decider-0.8b-executorch-mlx")
result = model(
    {"ticket": "I was charged twice. Please refund the duplicate."},
    {"team": {"type": "choice", "instructions": "Which team should handle this?",
              "criteria": {"billing": "Charges, invoices and refunds",
                           "technical": "Bugs and technical problems"}}},
)
print(result["answers"])

Or start the compatible local HTTP server:

python -m decider.serve --backend mlx --model edbordin-linktree/decider-0.8b-executorch-mlx

The loader downloads and caches the complete bundle automatically, including calibration and tokenizer assets. Set revision="COMMIT_HASH" in Python or --revision COMMIT_HASH on the server to pin a release. After the first download, Python's local_files_only=True loads the cached bundle offline. This is an ExecuTorch artifact, not an mlx-lm or Transformers checkpoint.

Only load trusted artifacts. Verify files against SHA256SUMS. Provenance and the binary checksum are in provenance.json. Tested: Python 3.12, Torch 2.14.0, ExecuTorch 1.5.0, Transformers 5.17.0, M4 Pro Mac with 48 GB memory. Other runtime versions/hardware and operation without full Xcode are not yet validated.

Limits

—Maximum 575 rendered tokens per row, subject to separate prefix and suffix bounds of 512 and 63 tokens.
—Up to 6 rows per forward; the adapter splits larger groups.
—No larger reference fallback: unsupported shapes fail. No vision or 32k context support in this compact artifact.
—Inherits the base model's limitations, including imperfect conditional-rule following. Small numerical parity checks are not a general accuracy benchmark.
—Run one model at a time. Export/parity tests use much more memory than serving. A 48 GB test machine does not establish compatibility with every smaller Mac.

See the upstream model card for training details and evaluations; those results belong to the model authors, not this conversion.