Modelpublic

mlboydaisuke/qwen3-vl-4b-stateful-coreml

sourceHugging Faceapache-2.0updated 19d agoView on Hugging Face

0likes78downloads

Model Card

Use it from Swift

Add the package

Package.swift:

swift

.package(url: "https://github.com/john-rocky/CoreML-LLM", branch: "main"),

// In your target:
.product(name: "CoreMLLLM", package: "CoreML-LLM"),

Platforms: iOS 18+ / macOS 15+.

Download + chat (one call)

swift

import CoreMLLLM

let llm = try await CoreMLLLM.load(repo: "mlboydaisuke/qwen3-vl-4b-stateful-coreml")

let stream = try await llm.generate(
    [CoreMLLLM.Message(role: .user, content: "Hello!")],
    maxTokens: 256
)
for await chunk in stream { print(chunk, terminator: "") }

With an image

swift

import CoreGraphics

let cgImage: CGImage = ...   // your CGImage

let stream = try await llm.generate(
    [CoreMLLLM.Message(role: .user,
                       content: "What's in this image?")],
    image: cgImage,
    maxTokens: 256
)
for await chunk in stream { print(chunk, terminator: "") }

Qwen3-VL 4B — Core ML stateful

Core ML port of `Qwen/Qwen3-VL-4B-Instruct` for iPhone / iPad / Mac Apple Neural Engine. Text + vision, 3.51 GB on disk.

The KV cache lives inside the ANE via MLState, so it does not spill to GPU memory as the context grows. Same layout as `qwen3-vl-2b-stateful-coreml`, scaled to 4B: 6 body chunks instead of 4.

Files

qwen3_vl_4b_stateful_chunks/
├── chunk_0.mlpackage … chunk_5.mlpackage   ← body, multifunction: infer (T=1) + prefill_b8 (T=8)
├── chunk_0_vision.mlpackage                ← chunk_0 + DeepStack injection
├── chunk_head.mlpackage                    ← final_norm + lm_head + in-graph argmax
└── embed_weight.bin                        ← raw fp16 embed table, Swift mmaps it

qwen3_vl_4b_vision/
└── vision.mlpackage                        ← image encoder + DeepStack taps

Component	Size
6 body chunks + vision chunk	2.12 GB
head	195 MB
embed table	778 MB
vision	415 MB
total	3.51 GB

Each body chunk carries two Core ML functions sharing one state. Swift creates the MLState once from the prefill instance and reuses it across both — Core ML binds state by name and shape, not by MLModel instance.

What this repo does NOT ship

—No `model_config.json` — Core ML packs shapes into each .mlpackage, and coremltools reads them directly.
—No tokenizer / processor — pull them from the upstream model:

python

from transformers import AutoTokenizer, AutoProcessor
tok  = AutoTokenizer.from_pretrained("Qwen/Qwen3-VL-4B-Instruct")
proc = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-4B-Instruct")

Standalone usage (Python / Mac)

python

import coremltools as ct, numpy as np
from huggingface_hub import snapshot_download

local = snapshot_download("mlboydaisuke/qwen3-vl-4b-stateful-coreml")
root  = f"{local}/qwen3_vl_4b_stateful_chunks"

prefill_chunks = [ct.models.MLModel(
    f"{root}/chunk_{i}.mlpackage", function_name="prefill_b8"
) for i in range(6)]
decode_chunks  = [ct.models.MLModel(
    f"{root}/chunk_{i}.mlpackage", function_name="infer"
) for i in range(6)]
head = ct.models.MLModel(f"{root}/chunk_head.mlpackage")

# State is created once and shared across infer + prefill_b8.
state = prefill_chunks[0].make_state()

Conversion

github.com/john-rocky/CoreML-LLM

License

Apache 2.0 (inherits from the base model).

More models in this format: Core ML Model Zoo — 46 models, each with the recipe that produced it.

Want a different model on-device? Open a request — free, open weights only; the export and its measured numbers get published publicly.