kaixkhazaki/e5-base-german-sentence-similarity

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

e5-base-german-sentence-similarity

This model is a fine-tuned version of intfloat/e5-base on german subset of the stsbmultimt dataset. It achieves the following results on the evaluation set:

Validation:

• —Loss: 0.9118
• —Pearson: 0.7952 Test:
• —Loss: 1.2162
• —Pearson: 0.7252

Usage

#install sentence-transformers
!pip install -U sentence-transformers

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("kaixkhazaki/e5-base-german-sentence-similarity")

sentences = [
    'Ein älterer Herr genießt die Natur auf einer Parkbank.',
    'Ein alter Mann vertieft sich in eine Zeitung im Stadtpark.',
    'Ein Teenager hört Musik auf einer Bank.'
]
embeddings = model.encode(sentences)

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)

>>
tensor([[1.0000, 0.9044, 0.7919],
        [0.9044, 1.0000, 0.7757],
        [0.7919, 0.7757, 1.0000]])

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

• —Dataset: STS Benchmark
• —

Training procedure

additional dropout implemented on model to avoid overfitting(hiddendropoutprob=0.25 ,attentionprobsdropout_prob = 0.25)

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 1e-05
• —trainbatchsize: 16
• —evalbatchsize: 32
• —seed: 42
• —optimizer: Use adamwtorch with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
• —lrschedulertype: cosine
• —lrschedulerwarmup_steps: 500
• —num_epochs: 12

Training results

Framework versions

• —Transformers 4.48.0
• —Pytorch 2.2.2
• —Datasets 3.2.0
• —Tokenizers 0.21.0