CoolFace
Modelpublic

Dl26/Veyra-Embed-125M

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
3likes28downloads
Model Card

Veyra-Embed-125M: Sentence Similarity Embedding Model

Veyra-Embed-125M is a from-scratch sentence embedding model developed by Dl26. It maps text into normalized dense vectors for sentence similarity, semantic search, clustering, and retrieval-style experiments.

The model is trained with a contrastive objective over real sentence pairs using in-batch negatives. It is designed as a compact encoder-style embedding model rather than a generative language model.

Model Details

PropertyValue
DeveloperDl26
Model typeSentence embedding encoder
Parameters125,560,320
Hidden size768
Layers4
Tokenizer vocab size32,430
Model vocab size135,000
Embedding size768
Max length64
ObjectiveSymmetric contrastive InfoNCE
Training datasentence-transformers/all-nli, sentence-transformers/stsb, sentence-transformers/quora-duplicates, embedding-data/QQP_triplets

Usage

This checkpoint contains raw PyTorch/safetensors weights plus tokenizer files. A compatible implementation should create the same encoder architecture from config.json, load model.safetensors, then mean-pool and normalize the output embedding.

python
from safetensors.torch import load_file

state_dict = load_file("model.safetensors")

Intended Use

  • —sentence similarity
  • —semantic search
  • —duplicate question retrieval
  • —clustering
  • —lightweight embedding research
  • —ranking experiments

Limitations

  • —It is not a generative language model.
  • —It should be evaluated on target retrieval and similarity datasets before use.
  • —It will not match large production embedding models trained on much larger curated mixtures.