thivy/embeddinggemma-300m-norwegian-health-cachedmnrl-v21

EmbeddingGemma 300M Norwegian Health (CachedMNRL Fine-tuned)

A Norwegian health-domain embedding model fine-tuned with CachedMultipleNegativesRankingLoss combining explicit hard negatives with in-batch negatives for optimal ranking performance.

Quick Start

from sentence_transformers import SentenceTransformer

model = SentenceTransformer('thivy/embeddinggemma-300m-norwegian-health-cachedmnrl-v1')

# Encode texts
query = "Hva er symptomene på diabetes?"
documents = [
    "Diabetes gir tørste, hyppig vannlating og tretthet.",
    "Høyt blodtrykk kan gi hodepine."
]

embeddings = model.encode([query] + documents)
similarity = model.similarity(embeddings[0], embeddings[1:])
print(similarity)

Model Details

• —Base Model: google/embeddinggemma-300m
• —Embedding Size: 768
• —Max Tokens: 2048
• —Language: Norwegian
• —Domain: Health & Medical

Training

Dataset: 330,120 triplets (anchor, positive, negative) from Norwegian health documents

Configuration:

• —Loss: CachedMultipleNegativesRankingLoss (InfoNCE + GradCache)
• —Learning Rate: 1.5e-5
• —Logical Batch Size: 192 (96 per GPU × 2 GPUs)
• —Mini-batch Size: 24 (GradCache)
• —Epochs: 1
• —Hardware: 2× A100 80GB with DDP

Key Innovation: CachedMNRL uses GradCache to enable large logical batch sizes (more in-batch negatives) while keeping GPU memory manageable. Each anchor sees its explicit hard negative plus all in-batch samples as candidates, combining the best of both approaches.

Training Lineage

Use Cases

• —Medical question answering
• —Health information retrieval
• —RAG systems for healthcare
• —Semantic search in Norwegian medical texts

Limitations

• —Optimized for Norwegian health/medical content
• —May not generalize well to other domains or languages
• —Best with sufficient context (full sentences/paragraphs)

Citation

@misc{embeddinggemma-300m-norwegian-health-cachedmnrl,
    title={EmbeddingGemma 300M Norwegian Health CachedMNRL Fine-tuned},
    author={Thivy},
    year={2026},
    url={https://huggingface.co/thivy/embeddinggemma-300m-norwegian-health-cachedmnrl-v1}
}

License

Inherits license from base EmbeddingGemma model (Gemma license).