CoolFace
Modelpublic

tamikisg/multilingual-e5-small-coreml

sourceHugging Facemitupdated 8mo agoView on Hugging Face
1likes109downloads
Model Card

Multilingual E5 Small - CoreML

This is a CoreML conversion of intfloat/multilingual-e5-small for iOS/macOS deployment.

Model Description

Multilingual E5 Small is a multilingual sentence embedding model optimized for semantic search and retrieval tasks. This CoreML version enables on-device inference on Apple platforms.

Key Features

  • —Multilingual: Supports 100+ languages including English, Japanese, Chinese, Korean, German, French, Spanish, and more
  • —Search-optimized: Designed specifically for retrieval tasks
  • —Cross-lingual: Can match queries in one language to documents in another
  • —On-device: Runs locally on iPhone/iPad/Mac without internet

Model Details

PropertyValue
Base Modelintfloat/multilingual-e5-small
Embedding Dimensions384
Max Sequence Length256 (configurable up to 512)
Model Size~224 MB
PrecisionFloat16
Minimum iOS17.0
Minimum macOS14.0

Usage

Input Format

E5 models use prefixes to distinguish between queries and documents:

  • —Query: "query: your search query here"
  • —Document: "passage: your document text here"

Swift Example

swift
import CoreML

// Load model
let config = MLModelConfiguration()
config.computeUnits = .cpuAndNeuralEngine
let model = try MLModel(contentsOf: modelURL, configuration: config)

// Prepare inputs (after tokenization)
let inputIds: MLMultiArray = // tokenized input
let attentionMask: MLMultiArray = // attention mask

// Run inference
let input = try MLDictionaryFeatureProvider(dictionary: [
    "input_ids": MLFeatureValue(multiArray: inputIds),
    "attention_mask": MLFeatureValue(multiArray: attentionMask)
])
let output = try model.prediction(from: input)
let embeddings = output.featureValue(for: "embeddings")?.multiArrayValue

Tokenizer

Use the included tokenizer.json with swift-transformers:

swift
import Tokenizers

let tokenizer = try await AutoTokenizer.from(modelFolder: tokenizerURL)
let encoded = tokenizer.encode(text: "query: your text")

Performance

Tested on iOS with Neural Engine:

DeviceInference Time
iPhone 15 Pro~15ms
iPhone 13~25ms
M1 Mac~10ms

Accuracy Comparison

Tested with 10 mixed Japanese/English technical queries:

ModelAccuracyAvg Score
Apple NLEmbedding20%0.558
This Model (E5)100%0.860

Files

  • —MultilingualE5Small.mlpackage/ - CoreML model package
  • —tokenizer.json - Tokenizer vocabulary and configuration
  • —tokenizer_config.json - Tokenizer settings

Conversion

Converted using coremltools with FP16 precision:

python
import coremltools as ct

mlmodel = ct.convert(
    traced_model,
    inputs=[
        ct.TensorType(name="input_ids", shape=(1, 256), dtype=np.int32),
        ct.TensorType(name="attention_mask", shape=(1, 256), dtype=np.int32),
    ],
    outputs=[
        ct.TensorType(name="embeddings", dtype=np.float16),
    ],
    convert_to="mlprogram",
    minimum_deployment_target=ct.target.iOS17,
    compute_precision=ct.precision.FLOAT16,
)

License

MIT License (same as the base model)

Citation

bibtex
@article{wang2024multilingual,
  title={Multilingual E5 Text Embeddings: A Technical Report},
  author={Wang, Liang and Yang, Nan and Huang, Xiaolong and Yang, Linjun and Majumder, Rangan and Wei, Furu},
  journal={arXiv preprint arXiv:2402.05672},
  year={2024}
}

Acknowledgments