Waqf-AI/arabic-splade-efficient

1---2language: ar3license: apache-2.04library_name: sentence-transformers5tags:6- sentence-transformers7- sparse-encoder8- splade9- arabic10- retrieval11datasets:12- oddadmix/arabic-triplets-large13base_model: distilbert-base-multilingual-cased14metrics:15- ndcg@1016- mrr@1017---18 19# Arabic SPLADE — Phase 320 21Efficient symmetric SPLADE using DistilBERT multilingual for faster inference.22 23## Architecture24 25Symmetric shared (MLMTransformer+SpladePooling, sequential)26 27**Base model:** distilbert-base-multilingual-cased28 29## Training30 31- **Dataset:** `oddadmix/arabic-triplets-large` (104K triplets, 92K unique passages)32- **Loss:** `SpladeLoss(SparseMultipleNegativesRankingLoss, q_reg=5e-5, d_reg=3e-5)`33- **Batch:** 16 per GPU, grad accum 434- **Learning rate:** 2e-535- **Epochs:** 136- **AMP:** fp1637- **Sampler:** NO_DUPLICATES38 39## Evaluation on Arabic NanoBEIR (13 datasets)40 41| Metric | Score |42|--------|-------|43| NDCG@10 | 0.2528 |44| MRR@10 | 0.3052 |45 46For reference: BM25 scores 0.3824 NDCG@10, 0.4483 MRR@10 on the same benchmark.47 48## Training Details49 50DistilBERT multilingual (6-layer, 119K vocab), ~2x faster than AraBERT51 52### Hardware53- 2× NVIDIA TITAN RTX (23.5 GB each)54- DDP via `torchrun`55 56## Usage57 58```python59from sentence_transformers.sparse_encoder import SparseEncoder60 61model = SparseEncoder("Abdelkareem/arabic-splade-efficient")62embeddings = model.encode([63    "ما هي عاصمة مصر؟",64    "القاهرة هي عاصمة مصر وأكبر مدنها.",65])66print(embeddings.shape)67# Decode top tokens68decoded = model.decode(embeddings, top_k=10)69for d in decoded:70    print(d)71```72