mlboydaisuke/FastContext-1.0-4B-CoreAI

Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's coreai-torch (LLMs: coreai.llm.export) into .aimodel bundles that run on the GPU or the Neural Engine, e.g. Qwen3-8B 4-bit decodes at 94 tok/s on an M4 Max GPU, MLX 90 under the same protocol (apple-silicon-llm-bench, macOS 27 beta 26A5353q, 2026-06-11).

<!-- gen-cards:devicemark begin (managed by scripts/gen-cards + tools/devicemark_row.py — edit cards.json, not this block) --> This model has no row on DeviceMark, the on-device LLM leaderboard. <!-- gen-cards:devicemark end -->

[!WARNING] ## Deprecated (2026-07-21) This port is retired. Two independent reasons: 1. Microsoft removed the upstream model (microsoft/FastContext-1.0-4B-SFT) from Hugging Face and GitHub on 2026-06-30 without explanation, so the original weights are no longer available. 2. The shipped .aimodelc here was compiled from a coreai-torch 0.4.0 conversion and fails to load on iOS/macOS 27 beta 2+. Rebuilding it requires the original weights (see 1). The repo is kept for reference. Every other model in the zoo has been recovered.

FastContext-1.0-4B-SFT — Core AI (on-device, iPhone)

microsoft/FastContext-1.0-4B-SFT converted to Apple Core AI for on-device inference, served by the CoreAIChat app.

FastContext is a long-context repository-exploration agent (Qwen3-4B-Instruct backbone, supervised-fine-tuned on exploration traces: broad first-turn search, multi-turn evidence gathering, and precise file:line citation generation).

Contents

• —gpu/ — AOT-compiled GPU (MPSGraph) bundle, 4-bit linear-INT4, h18p (iPhone 17 / iPhone 18 class). ~2.1 GB. Drop-in for CoreAIChat's Documents/models/fastcontext_4b_gpu.

Device class: this is an AOT h18p bundle (iPhone 17 / 18 class). On-device specialization of a 4B graph is not viable, so the bundle is precompiled for the h18p GPU — the same approach Core AI uses for the Gemma-4B bundle.

Verified on iPhone 17 Pro (iOS 27)

• —Parity vs HF fp16 (teacher-forced): 23/24 argmax, perplexity 1.41 — the single flip is a near-tie quant-noise position.
• —On-device GPU: ~334 tok/s prefill, ~16.5 tok/s decode (512-prompt / 1024-gen), load ~8 s cold, ~0.3 GB resident (weights memory-mapped).

Use

Install CoreAIChat, open the model picker, and select FastContext 4B.

License

MIT, inherited from the base model microsoft/FastContext-1.0-4B-SFT. See LICENSE.

<!-- funnel:v1 -->

More models in this format: Core AI Model Zoo — 75 models, each with the recipe that produced it.

Want a different model on-device? Open a request — free, open weights only; the export and its measured numbers get published publicly.

<!-- /funnel:v1 -->