ukung/semantic-lite
2221
semantic-lite
Lightweight multilingual embedding model for retrieval and semantic similarity, built from Qwen3-0.6B via layer pruning (28 → 6 layers). 2.4× smaller than the base model while matching or beating dedicated embedding models on retrieval.
- Dimensions: 1024
- Parameters: 250M (pruned from 596M)
- Languages: inherits Qwen3's 100+ language coverage
- Max sequence: 8192 tokens
Installation
pip install sentence-transformersQuick Start
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("ukung/semantic-lite")
embeddings = model.encode(["Hello world", "Halo dunia"])
# shape: (2, 1024)Use Cases
1. Semantic Similarity
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("ukung/semantic-lite")
sentences = [
"A man is playing a guitar.",
"A person is playing a musical instrument.",
"The stock market crashed today.",
]
embeddings = model.encode(sentences, normalize_embeddings=True)
similarity = embeddings @ embeddings.T
print(similarity[0, 1]) # 0.96 (similar)
print(similarity[0, 2]) # 0.79 (different)2. Semantic Search
from sentence_transformers import SentenceTransformer
import numpy as np
model = SentenceTransformer("ukung/semantic-lite")
corpus = [
"How to cook fried rice",
"Guide to stock market investing",
"Tips for caring for your cat",
"Python programming tutorial for beginners",
"How to grow chili peppers in pots",
]
corpus_embeddings = model.encode(corpus, normalize_embeddings=True)
query = "Easy home cooking recipes"
query_embedding = model.encode(query, normalize_embeddings=True)
scores = query_embedding @ corpus_embeddings.T
best = int(np.argmax(scores))
print(corpus[best]) # "How to cook fried rice"Tip: for short, abstract queries (e.g. "I want to learn coding"), use an asymmetric prompt for better precision:
query_emb = model.encode(query, prompt="query: ", normalize_embeddings=True)
doc_emb = model.encode(corpus, prompt="passage: ", normalize_embeddings=True)3. Cross-Lingual Retrieval
model = SentenceTransformer("ukung/semantic-lite")
documents = [
"How to bake a chocolate cake",
"Cara membuat kue cokelat",
"Comment faire un gâteau au chocolat",
"Cómo hacer un pastel de chocolate",
"The weather forecast for tomorrow",
]
doc_embeddings = model.encode(documents, normalize_embeddings=True)
query = "Resep kue cokelat" # Indonesian query
query_embedding = model.encode(query, normalize_embeddings=True)
scores = query_embedding @ doc_embeddings.T
print(documents[int(np.argmax(scores))]) # "Cara membuat kue cokelat"4. Clustering
from sentence_transformers import SentenceTransformer
from sklearn.cluster import KMeans
model = SentenceTransformer("ukung/semantic-lite")
texts = [
"I love pizza and pasta",
"Italian food is delicious",
"Python is a great programming language",
"I code in Python daily",
"The cat is sleeping on the sofa",
"My dog loves to play fetch",
]
embeddings = model.encode(texts, normalize_embeddings=True)
labels = KMeans(n_clusters=3, n_init=10, random_state=42).fit(embeddings).labels_
print(labels) # [2 2 0 0 1 1]5. Duplicate Detection
model = SentenceTransformer("ukung/semantic-lite")
sentences = [
"How do I reset my password?",
"What is the process to reset a password?",
"Where can I buy a new laptop?",
]
embeddings = model.encode(sentences, normalize_embeddings=True)
similarity = embeddings @ embeddings.T
print(similarity[0, 1]) # 0.91 (duplicate)
print(similarity[0, 2]) # 0.85 (not duplicate)6. Paraphrase Mining
from sentence_transformers import SentenceTransformer, util
model = SentenceTransformer("ukung/semantic-lite")
corpus = [
"What is the capital of France?",
"Paris is the capital of France.",
"How do I learn Python?",
"Python is a programming language.",
"The capital city of France is Paris.",
]
pairs = util.paraphrase_mining(model, corpus, top_k=3)
for score, i, j in pairs:
print(f"{score:.3f}: {corpus[i]} <-> {corpus[j]}")7. Retrieval-Augmented Generation (RAG)
model = SentenceTransformer("ukung/semantic-lite")
knowledge_base = [
"Qwen3 is a family of large language models by Alibaba.",
"Sentence-transformers is a Python library for embeddings.",
"The Eiffel Tower is located in Paris, France.",
]
kb_embeddings = model.encode(knowledge_base, normalize_embeddings=True)
query = "What is Qwen3?"
query_embedding = model.encode(query, normalize_embeddings=True)
scores = query_embedding @ kb_embeddings.T
context = knowledge_base[int(np.argmax(scores))]
print(context) # "Qwen3 is a family of large language models by Alibaba."8. Recommendation
model = SentenceTransformer("ukung/semantic-lite")
items = [
"Action movie with explosions",
"Romantic comedy film",
"Sci-fi space adventure",
"Horror ghost story",
]
item_embeddings = model.encode(items, normalize_embeddings=True)
user_preference = "I enjoy science fiction and space"
user_embedding = model.encode(user_preference, normalize_embeddings=True)
scores = user_embedding @ item_embeddings.T
print(items[int(np.argmax(scores))]) # "Sci-fi space adventure"9. Batch Encoding
model = SentenceTransformer("ukung/semantic-lite")
documents = [f"Document {i} about topic {i % 5}" for i in range(1000)]
embeddings = model.encode(
documents,
batch_size=32,
normalize_embeddings=True,
show_progress_bar=True,
)
print(embeddings.shape) # (1000, 1024)Multilingual Support
Inherits Qwen3's 100+ language coverage. Mean cosine similarity to English across 22 tested languages: 0.786 (21/22 above 0.6).
Specifications
Evaluation
Retrieval benchmark on an internal Indonesian corpus (20 documents, 8 queries):
Limitations
- Zero-shot classification is not supported. The model's embeddings are not calibrated for label-matching tasks. Use a dedicated zero-shot classifier instead.
- Short, abstract queries may require an asymmetric prompt (
query:/passage:) for optimal retrieval precision. - Evaluation is on a small internal corpus. Results on large public benchmarks (MTEB, BEIR) are not yet validated.
License
Apache 2.0 (inherited from Qwen3).
