CoolFace
Modelpublic

agraharr/finetune-matryoshka-telecom-embeddings

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
0likes4downloads
Model Card

agraharr/finetune-matryoshka-telecom-embeddings

Task: Domain-adapted Sentence Embeddings — Telecom NLI, Retrieval, Similarity

Model Overview

This model is a domain-specialized telecom sentence embedding model. It was finetuned from tomaarsen/mpnet-base-nli-matryoshka using Multiple Negatives Ranking Loss (contrastive SFT) over thousands of telecom QA pairs.

  • —Base: MPNet (Matryoshka sequence pooling)
  • —Finetune: Hard positive (label=1) sentence pairs from telecom Q&A, standards, research
  • —Downstream: Semantic search, similarity, clustering, answer retrieval in telecom tasks

How to Use (Embedding any sentence)

python
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("agraharr/finetune-matryoshka-telecom-embeddings")
emb = model.encode(["What is a gNodeB in 5G?"])
# emb is a (1, embedding_dim) numpy array

Batch encoding:

python
sentences = [
    "What is Open RAN?",
    "Define DSS in telecom",
    "Explain SDN virtualization"
]
vecs = model.encode(sentences)

Training details

  • —Library: sentence-transformers
  • —Model: tomaarsen/mpnet-base-nli-matryoshka
  • —Loss: MultipleNegativesRankingLoss
  • —Positives: Label=1 sentence pairs (QA) from telecom-wide dataset (teleqna_sentence_pairs.tsv)
  • —Epochs: 4 Batch: 32
  • —Optimizer: AdamW (default sbert)
  • —Hardware: a10g-large GPU on Hugging Face Spaces
  • —Logging: Trackio (space)

Data

  • —~46,322 telecom sentence pairs (questions, correct answers)
  • —Constructed & curated from standards (3GPP, IEEE, industry lexicons, publications)
  • —All pairs: teleqna_sentence_pairs.tsv

Citation

  • —Base Model: Tijs Tomaarsen. "mpnet-base-nli-matryoshka." Hugging Face. https://huggingface.co/tomaarsen/mpnet-base-nli-matryoshka
  • —Matryoshka Pooling: Liu, X. et al. "Matryoshka Representations for Efficient and Robust Sentence Embeddings." NeurIPS 2023. arXiv:2310.06665
  • —Data Curation: agraharr, Telecom QA mining 2024 (see repo for script/data generation)

Author & Contact

  • —Trained/curated by agraharr
  • —Feedback/issues: https://huggingface.co/agraharr