opticsWolf/jina-embeddings-v5-text-small-retrieval-onnx-fp16

jina-embeddings-v5-text-small-retrieval (ONNX FP16)

FP16 export of the official `jina-embeddings-v5-text-small-retrieval` ONNX weights (onnx/model.onnx, base revision 6856e76), for CUDA inference via ONNX Runtime. Same graph, same contract (last-token pooling, Matryoshka 32–1024); weights cast FP32→FP16 with IO kept in FP32.

• —onnx/model.onnx + onnx/model.onnx_data (1.19 GB), tokenizer.json unchanged.
• —License inherits the base model: CC-BY-NC-4.0 (non-commercial, Jina AI).

Recipe

import onnx
from onnxruntime.transformers.float16 import convert_float_to_float16
model = onnx.load("onnx/model.onnx", load_external_data=True)
fp16 = convert_float_to_float16(model, keep_io_types=True)
onnx.save_model(fp16, "model.onnx", save_as_external_data=True,
                location="model.onnx_data", all_tensors_to_one_file=True)

Measured drift vs FP32 (17 texts × 5 queries, dim 512)

Drift is length-correlated (worst case = a 77K-char document; short texts ≥ 0.999). FP16 on CPU is not viable (emulated kernels, >40x slower than FP32-CPU) — this artifact is CUDA-only by design. Full spike log with the INT8/q4f16 arms that were measured and rejected: see the embroider / okfgraph repos.