Modelpublic

rekabytes/Aranda-v1

sourceHugging Facemitupdated 2mo agoView on Hugging Face

1likes7downloads

Model Card

Aranda-v1

Aranda-v1 is a sentence embedding model specialized for Malaysian text retrieval, including Bahasa Malaysia, Manglish (Malaysian English code-switching), and cross-lingual BM↔English matching. It outperforms all tested baselines on overall retrieval Recall@1, Recall@5, Recall@10, and MRR, and is the best model on every individual language category (BM, Manglish, English, Cross-lingual).

Training

Two-phase contrastive curriculum:

Phase 1 — Breadth: MultipleNegativesRankingLoss on 1M Malaysian positive pairs (paraphrases, social media, news, QA). LR=1e-5, 1000 steps.

Phase 2 — Discrimination: Fine-tuned on 58K diverse hard-negative triplets (Lowyat, Twitter, Facebook, formal BM QA, English anchors, cross-lingual pairs) with explicit mined hard negatives. LR=2e-6, 2000 steps.

Base model: paraphrase-multilingual-mpnet-base-v2

Evaluation

Tested on 4,149 retrieval queries (BM, Manglish, English, Cross-lingual) with ~25 candidates per query, plus 4 additional eval sets (heuristic similarity, mined holdout, Mesolitica reranker, English STS).

Overall Retrieval (4,149 queries)

Model	Recall@1	Recall@5	Recall@10	MRR	Precision@10
mpnet-base	0.8631	0.9937	0.9973	0.9200	0.1785
multilingual-e5-base	0.8470	0.9940	0.9988	0.9091	0.1792
labse	0.8009	0.9882	0.9986	0.8821	0.1780
distilbert-multilingual-quora	0.7973	0.9636	0.9858	0.8719	0.1728
Aranda-v1	0.8891	0.9961	0.9998	0.9364	0.1788

Per-Language Recall@1

Model	BM	Manglish	English	Cross-lingual
mpnet-base	0.8200	0.8988	0.8577	0.8267
multilingual-e5-base	0.7938	0.9104	0.8658	0.6167
labse	0.7354	0.8993	0.6872	0.7300
distilbert-multilingual-quora	0.7584	0.8320	0.7866	0.7633
Aranda-v1	0.8431	0.9290	0.8792	0.8500

Heuristic Similarity 2K (per-language Spearman)

Model	Overall	BM	Manglish
mpnet-base	0.4799	0.5535	0.3937
multilingual-e5-base	0.3582	0.2714	0.4680
labse	0.3249	0.2866	0.4397
distilbert-multilingual-quora	0.4585	0.4857	0.4682
Aranda-v1	0.4532	0.4915	0.4536

Mined Holdout 2K (positive vs negative margins)

Model	pos>neg %	mean margin	mean pos sim	mean neg sim
mpnet-base	98.1%	0.4880	0.6576	0.1696
multilingual-e5-base	98.15%	0.1129	0.8805	0.7676
labse	98.3%	0.4193	0.5960	0.1768
distilbert-multilingual-quora	94.6%	0.1041	0.9336	0.8295
Aranda-v1	98.2%	0.4318	0.7334	0.3016

Mesolitica Reranker Test (NDCG@10)

Model	Overall NDCG@10
mpnet-base	0.4699
multilingual-e5-base	0.4705
labse	0.4695
distilbert-multilingual-quora	0.4690
Aranda-v1	0.4706

English STS (mteb/stsbenchmark-sts)

Model	Spearman
mpnet-base	0.8682
multilingual-e5-base	0.8420
labse	0.7225
distilbert-multilingual-quora	0.7866
Aranda-v1	0.8369

Usage

python

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("rekabytes/Aranda-v1")

# Encode texts
embeddings = model.encode([
    "macam mana nak renew lesen memandu",
    "how to renew driving license",
    "saya nak makan nasi lemak"
], normalize_embeddings=True)

# Cosine similarity
similarities = embeddings @ embeddings.T
print(similarities)

With RAG / vector search

python

# Encode your document corpus (do this once)
doc_embeddings = model.encode(documents, normalize_embeddings=True)

# At query time
query_embedding = model.encode([query], normalize_embeddings=True)
scores = query_embedding @ doc_embeddings.T
top_k = scores.argsort()[0][-5:][::-1]

Intended Use

—RAG context retrieval for Malaysian applications
—Semantic search over BM/Manglish/English document corpora
—Cross-lingual matching (BM ↔ English)
—Dense retrieval in hybrid search pipelines (paired with BM25)

Limitations

—English STS performance is below the base mpnet model (0.8369 vs 0.8682) — the model specialized for Malaysian text
—Not a reranker — use a cross-encoder for second-stage reranking
—Tested on Malaysian web data; performance may vary on other Southeast Asian languages

Model Details

—Architecture: XLM-RoBERTa (base)
—Embedding dimension: 768
—Max sequence length: 128
—Pooling: Mean
—Normalization: L2