zeroentropy/zembed-1-embedding

<img src="https://i.imgur.com/oxvhvQu.png"/>

Releasing zeroentropy/zembed-1

In retrieval systems, embedding models determine the quality of your search.

However, SOTA embedding models are closed-source and proprietary. At ZeroEntropy, we've trained a SOTA 4B open-weight multilingual embedding model that outperforms every competitor we benchmarked, and we're launching it here on HuggingFace.

This model outperforms OpenAI text-embedding-large, Cohere Embed v4, gemini-embedding-001, and voyage-4-nano across finance, healthcare, legal, conversational, manufacturing, code, and STEM.

zembed-1 is distilled directly from our SOTA reranker zerank-2 using our zELO methodology, which models relevance scores as adjusted Elo ratings. Standard contrastive training on binary labels can't match this signal. See our blog post for details.

The model supports flexible dimension projections (2560, 1280, 640, 320, 160, 80, 40) and quantization down to binary, compressing a full 8 KB vector to under 128 bytes with a controlled accuracy trade-off. See our Technical Report (Coming soon!) for details on the projection method. zembed-1 is multilingual from the ground up, with over half the training data in non-English languages.

This model is released under the Apache License 2.0.

Model Details

How to Use

from sentence_transformers import SentenceTransformer

# Initialize model
model = SentenceTransformer(
    "zeroentropy/zembed-1",
    trust_remote_code=True,
    model_kwargs={"torch_dtype": "bfloat16"},
)

# Define query and documents
query = "What is backpropagation?"
documents = [
    "Backpropagation is a fundamental algorithm for training neural networks by computing gradients.",
    "Gradient descent is used to optimize model parameters during the training process.",
    "Neural network training relies on efficient computation of derivatives through backpropagation.",
]

# Encode query and documents (uses task-specific prompts automatically)
query_embeddings = model.encode_query(query)
document_embeddings = model.encode_document(documents)
# (2560,) (3, 2560)

# Compute cosine similarities
similarities = model.similarity(query_embeddings, document_embeddings)
# tensor([[0.7525, 0.5670, 0.6835]])

The model can also be used through ZeroEntropy's /models/embed endpoint.

Evaluations

NDCG@10 scores between zembed-1 and competing embedding models, averaged across public and private benchmarks per domain. Full per-benchmark breakdown here.

<img src="assets/zembedevalchart.png" alt="Bar chart comparing zembed-1 NDCG@10 scores against competing embedding models across domains" width="1000"/>

License

This model is licensed under the Apache License 2.0.