NAMAA-Space/zarra
Zarra: Arabic Static Embedding Model
Zarra is a static embedding model built using the Model2Vec distillation framework. It is a distilled version of a Sentence Transformer, specifically optimized for the Arabic language. Unlike traditional transformer-based models, Zarra produces static embeddings, enabling ultra-fast inference on both CPU and GPU—making it ideal for resource-constrained environments or real-time applications.
Why Zarra?
⚡ Exceptional Speed: Delivers embeddings up to 500x faster than sentence transformers.
🧠 Compact & Efficient: Up to 50x smaller in size, allowing easy deployment on edge devices.
🧰 Versatile: Well-suited for search, clustering, classification, deduplication, and more.
🌍 Arabic-First: Specifically trained on high-quality Arabic data, ensuring relevance and performance across a range of Arabic NLP tasks.
<p align="center"> <img src="https://cdn-uploads.huggingface.co/production/uploads/628f7a71dd993507cfcbe587/3JEnPfgF2BfbN5H81K0XD.png" alt="Speed vs Performance Chart" width="700"/> </p>
About Model2Vec
The Model2Vec distillation technique transfers knowledge from large transformer models into lightweight static embedding spaces, preserving semantic quality while dramatically improving speed and efficiency. Zarra represents the best of both worlds: the semantic power of transformers and the speed and simplicity of static vectors.
Installation
Install model2vec using pip:
pip install model2vecUsage
Using Model2Vec
The Model2Vec library is the fastest and most lightweight way to run Model2Vec models.
Load this model using the from_pretrained method:
from model2vec import StaticModel
# Load a pretrained Model2Vec model
model = StaticModel.from_pretrained("NAMAA-Space/zarra")
# Compute text embeddings
embeddings = model.encode(["Example sentence"])Using Sentence Transformers
You can also use the Sentence Transformers library to load and use the model:
from sentence_transformers import SentenceTransformer
# Load a pretrained Sentence Transformer model
model = SentenceTransformer("NAMAA-Space/zarra")
# Compute text embeddings
embeddings = model.encode(["Example sentence"])How it Works
Model2vec creates a small, fast, and powerful model that outperforms other static embedding models by a large margin on all tasks we could find, while being much faster to create than traditional static embedding models such as GloVe. Best of all, you don't need any data to distill a model using Model2Vec.
It works by passing a vocabulary through a sentence transformer model, then reducing the dimensionality of the resulting embeddings using PCA, and finally weighting the embeddings using SIF weighting. During inference, we simply take the mean of all token embeddings occurring in a sentence.
Benchmark on Arabic
Speed
- Zarra and Bojji excel in speed, achieving 26893.63 and 27478.15 sentences per second on CPU, respectively, far surpassing CUDA-based models like arabictripletmatryoshka_v2 (610.64).
- Top Performer: Bojji is the fastest model, slightly ahead of Zarra and potion-multilingual-128M (27145.31), highlighting the efficiency of Model2Vec-based models on CPU.
- Key Observation: The high speed of Zarra and Bojji on CPU makes them ideal for resource-constrained environments, offering significant advantages over CUDA-dependent models.
Size of the Model
- Zarra is the smallest model, with only 64 million parameters and 244.14 MB in size, making it 2.39 times smaller than the largest model (getmultilingualbase).
- Bojji is slightly larger at 124.88 million parameters and 476.40 MB, but still significantly smaller than most other models.
- Top Performer: Zarra leads in compactness, offering the smallest footprint, which is critical for deployment on resource-limited devices.
- Key Observation: The compact size of Zarra and Bojji aligns with their design goal of efficiency, making them highly suitable for edge computing and real-time applications.