fcmeyer/F2LLM-v2-0.6B-bf16-mlx

F2LLM-v2-0.6B-bf16-mlx

MLX-native port of codefuse-ai/F2LLM-v2-0.6B (a 0.6B Qwen3-based multilingual embedding model, 1024-dim, last-token pooling, L2-normalized, MRL-trained) for Apple Silicon via mlx-embeddings.

• —Conversion: python -m mlx_embeddings.convert --hf-path codefuse-ai/F2LLM-v2-0.6B --mlx-path ./F2LLM-v2-0.6B-bf16-mlx --dtype bfloat16 (mlx-embeddings 0.1.0).
• —Weights: bit-exact bf16 copy of the source safetensors (max abs diff 0.0 across all 310 tensors; key prefix model. added per mlx-embeddings convention; no quantization). 1.1 GB.
• —Files: model.safetensors (+ index), config.json, tokenizer files, modules.json, config_sentence_transformers.json, 1_Pooling/config.json (last-token pooling, include_prompt=true).

Usage

from mlx_embeddings.utils import load

model, tokenizer = load("fcmeyer/F2LLM-v2-0.6B-bf16-mlx")

query_prompt = "Instruct: Given a question, retrieve passages that can help answer the question.\nQuery: "
texts = [
    query_prompt + "What is F2LLM used for?",
    "We present F2LLM, a family of fully open embedding LLMs.",
    "F2LLM 是 CodeFuse 开源的系列嵌入模型。",
]
inputs = tokenizer.batch_encode_plus(
    texts, return_tensors="mlx", padding=True, truncation=True, max_length=4096,
)
outputs = model(inputs["input_ids"], attention_mask=inputs["attention_mask"])
embeddings = outputs.text_embeds  # pooled + normalized, (3, 1024)
similarity = embeddings[0:1] @ embeddings[1:].T

Notes:

• —Use the query prompt for queries, not for documents/passages (same convention as the source model card). No prompt is needed for symmetric tasks (STS, clustering).
• —MRL: truncate then re-normalize, e.g. e = e[..., :128]; e = e / max(norm(e), 1e-9).
• —truncation=True keeps the head + appends EOS (verified: first 511 tokens + EOS at max_length=512).
• —Batching: rows that need no padding are exactly order- and batch-invariant (max abs diff 0.0). When a batch mixes very different lengths (e.g. 28 vs 2001 tokens), the short row's output differs slightly from a length-matched run (~0.005 max abs on that probe; torch reference diff is 0.0). This is an upstream mlx-embeddings Qwen3 fast-SDPA masking behavior (reproduces on the float32 conversion too, smaller at ~0.14 hidden-state diff vs ~2.4 in bf16), not a weight error. For max fidelity, pad to similar lengths or encode length-mismatched inputs separately.

Verification (torch bf16 reference vs this MLX bf16, Apple M5 Max)

10-text suite (query + EN/ZH/RU docs + unrelated + short/long/code/mixed-language):

• —Per-row cosine similarity (after re-normalizing): worst 0.9985 (~3.2°), typical ≥ 0.9992 (~2.1°). Same for the fp32 conversion (worst 0.9995) modulo backend matmul/softmax numerics — i.e. the gap is the MLX attention backend, not the bf16 weights.
• —Query-vs-all cosine deltas ≤ 0.004; ranking identical (10/10), top-1 identical, unrelated doc ranked last (10/10) on both.
• —MRL truncation (renorm after truncation) at 1024/128/8 dims: worst cosine 0.9985/0.9985/0.9969, ranking preserved at all dims.
• —Tokenizer parity: identical token streams to the source tokenizer on the probe (28 tokens, same head/tail incl. EOS 151645).
• —Latency: ~8 ms/batch (bs=1), ~500 ms/batch (bs=8, incl. 2001-token doc).

Citation

@misc{f2llm-v2,
  title={F2LLM-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual World},
  author={Ziyin Zhang and Zihan Liao and Hang Yu and Peng Di and Rui Wang},
  year={2026},
  eprint={2603.19223},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2603.19223},
}

Source model card and license (Apache-2.0): https://huggingface.co/codefuse-ai/F2LLM-v2-0.6B