abdulmunimjemal/xlm-r-retrieval-am-v1

SentenceTransformer Fine-Tuned for Amharic Retrieval

This model is a sentence-transformers model finetuned on Amharic QA triplets. It maps sentences and paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

• —Model Type: Sentence Transformer
• —Base Model: sentence-transformers/paraphrase-xlm-r-multilingual-v1
• —Training Task: Triplet loss with Matryoshka loss
• —Language: Amharic
• —Maximum Sequence Length: 128 tokens
• —Output Dimensionality: 768 dimensions
• —Similarity Function: Cosine Similarity

Training Overview

• —Training Data: Custom Amharic QA triplets (with positive and negative examples)
• —Training Strategy: The model was finetuned using a combination of triplet loss and a Matryoshka loss, with evaluation performed using a TripletEvaluator.
• —Hyperparameters:
• —Epochs: 3
• —Batch Size: 16
• —Learning Rate: 1e-6
• —Warmup Ratio: 0.08
• —Weight Decay: 0.05

Evaluation

The model was evaluated on a held-out test set using cosine similarity as the metric:

Usage

To use the model in your own project:

1. 1.Install Sentence Transformers:

   pip install -U sentence-transformers

1. 1.Load the Model:

   from sentence_transformers import SentenceTransformer

   model = SentenceTransformer("abdulmunimjemal/xlm-r-retrieval-am-v5")
   sentences = [
       "ሰማይ ምን አይነት ቀለም ነው?",
       "ሰማይ ሰማያዊ ቀለም አለው።" ,
       "እኔ ምሳ እንጀራ በላሁ።" ,
       "ባሕር ምን አይነት ቀለም ነው?",
       "አየር በምድር ዙሪያ ያለ ነው።"
   ]
   embeddings = model.encode(sentences)
   print(embeddings.shape)  # Expected output: (5, 768)

1. 1.Compute Similarity:

   from sklearn.metrics.pairwise import cosine_similarity
   similarities = cosine_similarity(embeddings, embeddings)
   print(similarities.shape)  # Expected output: (5, 5)

Model Architecture

Below is an outline of the model architecture:

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_mean_tokens': True, ...})
)

Training Environment

• —Python: 3.11.11
• —Sentence Transformers: 3.3.1
• —Transformers: 4.47.1
• —PyTorch: 2.5.1+cu124
• —Accelerate: 1.2.1
• —Datasets: 3.2.0
• —Tokenizers: 0.21.0

Citation

If you use this model in your research, please cite it appropriately.

@misc{your_model,
  title = {SentenceTransformer Fine-Tuned for Amharic Retrieval},
  author = {Abdulmunim J. Jemal},
  year = {2025},
  howpublished = {Hugging Face Model Hub, \url{https://huggingface.co/abdulmunimjemal/xlm-r-retrieval-am-v1}}
}