Modelpublic

RomainDarous/multiSts_meanPooling_mistranslationModel

sourceHugging Faceupdated 2y agoView on Hugging Face

0likes12downloads

Model Card

SentenceTransformer based on RomainDarous/pretrainingoriginal_model

This is a sentence-transformers model finetuned from RomainDarous/pre_training_original_model on the multi_stsb_de, multi_stsb_es, multi_stsb_fr, multi_stsb_it, multi_stsb_nl, multi_stsb_pl, multi_stsb_pt, multi_stsb_ru and multi_stsb_zh datasets. It maps sentences & paragraphs to a 512-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

—Model Type: Sentence Transformer
—Base model: RomainDarous/pre_training_original_model
—Maximum Sequence Length: 128 tokens
—Output Dimensionality: 512 dimensions
—Similarity Function: Cosine Similarity
—Training Datasets:
—multi_stsb_de
—multi_stsb_es
—multi_stsb_fr
—multi_stsb_it
—multi_stsb_nl
—multi_stsb_pl
—multi_stsb_pt
—multi_stsb_ru
—multi_stsb_zh
—Languages: de, en, es, fr, it, nl, pl, pt, ru, zh

Model Sources

—Documentation: Sentence Transformers Documentation
—Repository: Sentence Transformers on GitHub
—Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: DistilBertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 512, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh'})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

bash

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("RomainDarous/multists_finetuned_original_model")
# Run inference
sentences = [
    'Dois cães a lutar na neve.',
    'Dois cães brincam na neve.',
    'Pode sempre perguntar, então é a escolha do autor a aceitar ou não.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 512]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

Evaluation

Metrics

Semantic Similarity

—Datasets: sts-eval, sts-test, sts-test, sts-test, sts-test, sts-test, sts-test, sts-test, sts-test and sts-test
—Evaluated with <code>EmbeddingSimilarityEvaluator</code>

Metric	sts-eval	sts-test
pearson_cosine	0.6494	0.6608
spearman_cosine	0.6712	0.6617

Semantic Similarity

—Dataset: sts-eval
—Evaluated with <code>EmbeddingSimilarityEvaluator</code>

Metric	Value
pearson_cosine	0.6649
spearman_cosine	0.686

Semantic Similarity

—Dataset: sts-eval
—Evaluated with <code>EmbeddingSimilarityEvaluator</code>

Metric	Value
pearson_cosine	0.6575
spearman_cosine	0.6819

Semantic Similarity

—Dataset: sts-eval
—Evaluated with <code>EmbeddingSimilarityEvaluator</code>

Metric	Value
pearson_cosine	0.6483
spearman_cosine	0.6739

Semantic Similarity

—Dataset: sts-eval
—Evaluated with <code>EmbeddingSimilarityEvaluator</code>

Metric	Value
pearson_cosine	0.6577
spearman_cosine	0.6798

Semantic Similarity

—Dataset: sts-eval
—Evaluated with <code>EmbeddingSimilarityEvaluator</code>

Metric	Value
pearson_cosine	0.5801
spearman_cosine	0.6026

Semantic Similarity

—Dataset: sts-eval
—Evaluated with <code>EmbeddingSimilarityEvaluator</code>

Metric	Value
pearson_cosine	0.6446
spearman_cosine	0.6738

Semantic Similarity

—Dataset: sts-eval
—Evaluated with <code>EmbeddingSimilarityEvaluator</code>

Metric	Value
pearson_cosine	0.6061
spearman_cosine	0.6211

Semantic Similarity

—Dataset: sts-eval
—Evaluated with <code>EmbeddingSimilarityEvaluator</code>

Metric	Value
pearson_cosine	0.6672
spearman_cosine	0.6865

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Datasets

multistsbde

—Dataset: multi_stsb_de at 3acaa3d
—Size: 5,749 training samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 5 tokens</li><li>mean: 12.05 tokens</li><li>max: 37 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 12.01 tokens</li><li>max: 37 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.45</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:---------------------------------------------------------------|:--------------------------------------------------------------------------|:--------------------------------| | <code>Ein Flugzeug hebt gerade ab.</code> | <code>Ein Flugzeug hebt gerade ab.</code> | <code>1.0</code> | | <code>Ein Mann spielt eine große Flöte.</code> | <code>Ein Mann spielt eine Flöte.</code> | <code>0.7599999904632568</code> | | <code>Ein Mann streicht geriebenen Käse auf eine Pizza.</code> | <code>Ein Mann streicht geriebenen Käse auf eine ungekochte Pizza.</code> | <code>0.7599999904632568</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

multistsbes

—Dataset: multi_stsb_es at 3acaa3d
—Size: 5,749 training samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 7 tokens</li><li>mean: 12.28 tokens</li><li>max: 36 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 12.14 tokens</li><li>max: 31 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.45</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:----------------------------------------------------------------|:----------------------------------------------------------------------|:--------------------------------| | <code>Un avión está despegando.</code> | <code>Un avión está despegando.</code> | <code>1.0</code> | | <code>Un hombre está tocando una gran flauta.</code> | <code>Un hombre está tocando una flauta.</code> | <code>0.7599999904632568</code> | | <code>Un hombre está untando queso rallado en una pizza.</code> | <code>Un hombre está untando queso rallado en una pizza cruda.</code> | <code>0.7599999904632568</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

multistsbfr

—Dataset: multi_stsb_fr at 3acaa3d
—Size: 5,749 training samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 6 tokens</li><li>mean: 12.47 tokens</li><li>max: 38 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 12.37 tokens</li><li>max: 31 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.45</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:-----------------------------------------------------------|:---------------------------------------------------------------------|:--------------------------------| | <code>Un avion est en train de décoller.</code> | <code>Un avion est en train de décoller.</code> | <code>1.0</code> | | <code>Un homme joue d'une grande flûte.</code> | <code>Un homme joue de la flûte.</code> | <code>0.7599999904632568</code> | | <code>Un homme étale du fromage râpé sur une pizza.</code> | <code>Un homme étale du fromage râpé sur une pizza non cuite.</code> | <code>0.7599999904632568</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

multistsbit

—Dataset: multi_stsb_it at 3acaa3d
—Size: 5,749 training samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 7 tokens</li><li>mean: 12.92 tokens</li><li>max: 36 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 12.81 tokens</li><li>max: 30 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.45</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:--------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:--------------------------------| | <code>Un aereo sta decollando.</code> | <code>Un aereo sta decollando.</code> | <code>1.0</code> | | <code>Un uomo sta suonando un grande flauto.</code> | <code>Un uomo sta suonando un flauto.</code> | <code>0.7599999904632568</code> | | <code>Un uomo sta spalmando del formaggio a pezzetti su una pizza.</code> | <code>Un uomo sta spalmando del formaggio a pezzetti su una pizza non cotta.</code> | <code>0.7599999904632568</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

multistsbnl

—Dataset: multi_stsb_nl at 3acaa3d
—Size: 5,749 training samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 6 tokens</li><li>mean: 12.12 tokens</li><li>max: 33 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 12.04 tokens</li><li>max: 30 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.45</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:--------------------------------------------------------|:--------------------------------------------------------------------|:--------------------------------| | <code>Er gaat een vliegtuig opstijgen.</code> | <code>Er gaat een vliegtuig opstijgen.</code> | <code>1.0</code> | | <code>Een man speelt een grote fluit.</code> | <code>Een man speelt fluit.</code> | <code>0.7599999904632568</code> | | <code>Een man smeert geraspte kaas op een pizza.</code> | <code>Een man strooit geraspte kaas op een ongekookte pizza.</code> | <code>0.7599999904632568</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

multistsbpl

—Dataset: multi_stsb_pl at 3acaa3d
—Size: 5,749 training samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 6 tokens</li><li>mean: 13.24 tokens</li><li>max: 46 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 13.08 tokens</li><li>max: 34 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.45</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:-----------------------------------------------------------|:------------------------------------------------------------------------|:--------------------------------| | <code>Samolot wystartował.</code> | <code>Samolot wystartował.</code> | <code>1.0</code> | | <code>Człowiek gra na dużym flecie.</code> | <code>Człowiek gra na flecie.</code> | <code>0.7599999904632568</code> | | <code>Mężczyzna rozsiewa na pizzy rozdrobniony ser.</code> | <code>Mężczyzna rozsiewa rozdrobniony ser na niegotowanej pizzy.</code> | <code>0.7599999904632568</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

multistsbpt

—Dataset: multi_stsb_pt at 3acaa3d
—Size: 5,749 training samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:---------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 7 tokens</li><li>mean: 13.0 tokens</li><li>max: 37 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 12.99 tokens</li><li>max: 34 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.45</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:------------------------------------------------------------------|:----------------------------------------------------------------------------------|:--------------------------------| | <code>Um avião está a descolar.</code> | <code>Um avião aéreo está a descolar.</code> | <code>1.0</code> | | <code>Um homem está a tocar uma grande flauta.</code> | <code>Um homem está a tocar uma flauta.</code> | <code>0.7599999904632568</code> | | <code>Um homem está a espalhar queijo desfiado numa pizza.</code> | <code>Um homem está a espalhar queijo desfiado sobre uma pizza não cozida.</code> | <code>0.7599999904632568</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

multistsbru

—Dataset: multi_stsb_ru at 3acaa3d
—Size: 5,749 training samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 5 tokens</li><li>mean: 12.66 tokens</li><li>max: 47 tokens</li></ul> | <ul><li>min: 5 tokens</li><li>mean: 12.67 tokens</li><li>max: 36 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.45</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:------------------------------------------------|:---------------------------------------------------------------------|:--------------------------------| | <code>Самолет взлетает.</code> | <code>Взлетает самолет.</code> | <code>1.0</code> | | <code>Человек играет на большой флейте.</code> | <code>Человек играет на флейте.</code> | <code>0.7599999904632568</code> | | <code>Мужчина разбрасывает сыр на пиццу.</code> | <code>Мужчина разбрасывает измельченный сыр на вареную пиццу.</code> | <code>0.7599999904632568</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

multistsbzh

—Dataset: multi_stsb_zh at 3acaa3d
—Size: 5,749 training samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 7 tokens</li><li>mean: 12.55 tokens</li><li>max: 37 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 12.73 tokens</li><li>max: 28 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.45</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:------------------------------|:----------------------------------|:--------------------------------| | <code>一架飞机正在起飞。</code> | <code>一架飞机正在起飞。</code> | <code>1.0</code> | | <code>一个男人正在吹一支大笛子。</code> | <code>一个人在吹笛子。</code> | <code>0.7599999904632568</code> | | <code>一名男子正在比萨饼上涂抹奶酪丝。</code> | <code>一名男子正在将奶酪丝涂抹在未熟的披萨上。</code> | <code>0.7599999904632568</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

Evaluation Datasets

multistsbde

—Dataset: multi_stsb_de at 3acaa3d
—Size: 1,500 evaluation samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 6 tokens</li><li>mean: 18.96 tokens</li><li>max: 51 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 19.01 tokens</li><li>max: 55 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.42</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:-------------------------------------------------------------|:-----------------------------------------------------------|:-------------------------------| | <code>Ein Mann mit einem Schutzhelm tanzt.</code> | <code>Ein Mann mit einem Schutzhelm tanzt.</code> | <code>1.0</code> | | <code>Ein kleines Kind reitet auf einem Pferd.</code> | <code>Ein Kind reitet auf einem Pferd.</code> | <code>0.949999988079071</code> | | <code>Ein Mann verfüttert eine Maus an eine Schlange.</code> | <code>Der Mann füttert die Schlange mit einer Maus.</code> | <code>1.0</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

multistsbes

—Dataset: multi_stsb_es at 3acaa3d
—Size: 1,500 evaluation samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 7 tokens</li><li>mean: 18.41 tokens</li><li>max: 45 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 18.24 tokens</li><li>max: 51 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.42</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:----------------------------------------------------------------------|:---------------------------------------------------------------------|:-------------------------------| | <code>Un hombre con un casco está bailando.</code> | <code>Un hombre con un casco está bailando.</code> | <code>1.0</code> | | <code>Un niño pequeño está montando a caballo.</code> | <code>Un niño está montando a caballo.</code> | <code>0.949999988079071</code> | | <code>Un hombre está alimentando a una serpiente con un ratón.</code> | <code>El hombre está alimentando a la serpiente con un ratón.</code> | <code>1.0</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

multistsbfr

—Dataset: multi_stsb_fr at 3acaa3d
—Size: 1,500 evaluation samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 6 tokens</li><li>mean: 19.77 tokens</li><li>max: 50 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 19.62 tokens</li><li>max: 56 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.42</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:-------------------------------------------------------------------------|:----------------------------------------------------------------------------|:-------------------------------| | <code>Un homme avec un casque de sécurité est en train de danser.</code> | <code>Un homme portant un casque de sécurité est en train de danser.</code> | <code>1.0</code> | | <code>Un jeune enfant monte à cheval.</code> | <code>Un enfant monte à cheval.</code> | <code>0.949999988079071</code> | | <code>Un homme donne une souris à un serpent.</code> | <code>L'homme donne une souris au serpent.</code> | <code>1.0</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

multistsbit

—Dataset: multi_stsb_it at 3acaa3d
—Size: 1,500 evaluation samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 6 tokens</li><li>mean: 19.05 tokens</li><li>max: 48 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 19.03 tokens</li><li>max: 56 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.42</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:------------------------------------------------------------------|:---------------------------------------------------------------|:-------------------------------| | <code>Un uomo con l'elmetto sta ballando.</code> | <code>Un uomo che indossa un elmetto sta ballando.</code> | <code>1.0</code> | | <code>Un bambino piccolo sta cavalcando un cavallo.</code> | <code>Un bambino sta cavalcando un cavallo.</code> | <code>0.949999988079071</code> | | <code>Un uomo sta dando da mangiare un topo a un serpente.</code> | <code>L'uomo sta dando da mangiare un topo al serpente.</code> | <code>1.0</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

multistsbnl

—Dataset: multi_stsb_nl at 3acaa3d
—Size: 1,500 evaluation samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 6 tokens</li><li>mean: 19.12 tokens</li><li>max: 49 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 18.95 tokens</li><li>max: 50 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.42</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:-----------------------------------------------------|:-----------------------------------------------------|:-------------------------------| | <code>Een man met een helm is aan het dansen.</code> | <code>Een man met een helm is aan het dansen.</code> | <code>1.0</code> | | <code>Een jong kind rijdt op een paard.</code> | <code>Een kind rijdt op een paard.</code> | <code>0.949999988079071</code> | | <code>Een man voedt een muis aan een slang.</code> | <code>De man voert een muis aan de slang.</code> | <code>1.0</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

multistsbpl

—Dataset: multi_stsb_pl at 3acaa3d
—Size: 1,500 evaluation samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:---------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 7 tokens</li><li>mean: 21.6 tokens</li><li>max: 58 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 21.47 tokens</li><li>max: 56 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.42</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:---------------------------------------------------|:---------------------------------------------------|:-------------------------------| | <code>Tańczy mężczyzna w twardym kapeluszu.</code> | <code>Tańczy mężczyzna w twardym kapeluszu.</code> | <code>1.0</code> | | <code>Małe dziecko jedzie na koniu.</code> | <code>Dziecko jedzie na koniu.</code> | <code>0.949999988079071</code> | | <code>Człowiek karmi węża myszką.</code> | <code>Ten człowiek karmi węża myszką.</code> | <code>1.0</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

multistsbpt

—Dataset: multi_stsb_pt at 3acaa3d
—Size: 1,500 evaluation samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 7 tokens</li><li>mean: 19.26 tokens</li><li>max: 48 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 19.08 tokens</li><li>max: 50 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.42</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:------------------------------------------------------------|:-----------------------------------------------------------|:-------------------------------| | <code>Um homem de chapéu duro está a dançar.</code> | <code>Um homem com um capacete está a dançar.</code> | <code>1.0</code> | | <code>Uma criança pequena está a montar a cavalo.</code> | <code>Uma criança está a montar a cavalo.</code> | <code>0.949999988079071</code> | | <code>Um homem está a alimentar um rato a uma cobra.</code> | <code>O homem está a alimentar a cobra com um rato.</code> | <code>1.0</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

multistsbru

—Dataset: multi_stsb_ru at 3acaa3d
—Size: 1,500 evaluation samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 6 tokens</li><li>mean: 20.91 tokens</li><li>max: 55 tokens</li></ul> | <ul><li>min: 5 tokens</li><li>mean: 20.95 tokens</li><li>max: 65 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.42</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:------------------------------------------------------|:----------------------------------------------|:-------------------------------| | <code>Человек в твердой шляпе танцует.</code> | <code>Мужчина в твердой шляпе танцует.</code> | <code>1.0</code> | | <code>Маленький ребенок едет верхом на лошади.</code> | <code>Ребенок едет на лошади.</code> | <code>0.949999988079071</code> | | <code>Мужчина кормит мышь змее.</code> | <code>Человек кормит змею мышью.</code> | <code>1.0</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

multistsbzh

—Dataset: multi_stsb_zh at 3acaa3d
—Size: 1,500 evaluation samples
—Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
—Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 5 tokens</li><li>mean: 19.81 tokens</li><li>max: 53 tokens</li></ul> | <ul><li>min: 5 tokens</li><li>mean: 19.67 tokens</li><li>max: 56 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.42</li><li>max: 1.0</li></ul> |
—Samples: | sentence1 | sentence2 | score | |:---------------------------|:--------------------------|:-------------------------------| | <code>一个戴着硬帽子的人在跳舞。</code> | <code>一个戴着硬帽的人在跳舞。</code> | <code>1.0</code> | | <code>一个小孩子在骑马。</code> | <code>一个孩子在骑马。</code> | <code>0.949999988079071</code> | | <code>一个人正在用老鼠喂蛇。</code> | <code>那人正在给蛇喂老鼠。</code> | <code>1.0</code> |
—Loss: <code>CoSENTLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

Training Hyperparameters

Non-Default Hyperparameters

—eval_strategy: steps
—per_device_train_batch_size: 16
—per_device_eval_batch_size: 16
—num_train_epochs: 4
—warmup_ratio: 0.1

All Hyperparameters

<details><summary>Click to expand</summary>

—overwrite_output_dir: False
—do_predict: False
—eval_strategy: steps
—prediction_loss_only: True
—per_device_train_batch_size: 16
—per_device_eval_batch_size: 16
—per_gpu_train_batch_size: None
—per_gpu_eval_batch_size: None
—gradient_accumulation_steps: 1
—eval_accumulation_steps: None
—torch_empty_cache_steps: None
—learning_rate: 5e-05
—weight_decay: 0.0
—adam_beta1: 0.9
—adam_beta2: 0.999
—adam_epsilon: 1e-08
—max_grad_norm: 1.0
—num_train_epochs: 4
—max_steps: -1
—lr_scheduler_type: linear
—lr_scheduler_kwargs: {}
—warmup_ratio: 0.1
—warmup_steps: 0
—log_level: passive
—log_level_replica: warning
—log_on_each_node: True
—logging_nan_inf_filter: True
—save_safetensors: True
—save_on_each_node: False
—save_only_model: False
—restore_callback_states_from_checkpoint: False
—no_cuda: False
—use_cpu: False
—use_mps_device: False
—seed: 42
—data_seed: None
—jit_mode_eval: False
—use_ipex: False
—bf16: False
—fp16: False
—fp16_opt_level: O1
—half_precision_backend: auto
—bf16_full_eval: False
—fp16_full_eval: False
—tf32: None
—local_rank: 0
—ddp_backend: None
—tpu_num_cores: None
—tpu_metrics_debug: False
—debug: []
—dataloader_drop_last: False
—dataloader_num_workers: 0
—dataloader_prefetch_factor: None
—past_index: -1
—disable_tqdm: False
—remove_unused_columns: True
—label_names: None
—load_best_model_at_end: False
—ignore_data_skip: False
—fsdp: []
—fsdp_min_num_params: 0
—fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
—fsdp_transformer_layer_cls_to_wrap: None
—accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
—deepspeed: None
—label_smoothing_factor: 0.0
—optim: adamw_torch
—optim_args: None
—adafactor: False
—group_by_length: False
—length_column_name: length
—ddp_find_unused_parameters: None
—ddp_bucket_cap_mb: None
—ddp_broadcast_buffers: False
—dataloader_pin_memory: True
—dataloader_persistent_workers: False
—skip_memory_metrics: True
—use_legacy_prediction_loop: False
—push_to_hub: False
—resume_from_checkpoint: None
—hub_model_id: None
—hub_strategy: every_save
—hub_private_repo: None
—hub_always_push: False
—gradient_checkpointing: False
—gradient_checkpointing_kwargs: None
—include_inputs_for_metrics: False
—include_for_metrics: []
—eval_do_concat_batches: True
—fp16_backend: auto
—push_to_hub_model_id: None
—push_to_hub_organization: None
—mp_parameters:
—auto_find_batch_size: False
—full_determinism: False
—torchdynamo: None
—ray_scope: last
—ddp_timeout: 1800
—torch_compile: False
—torch_compile_backend: None
—torch_compile_mode: None
—dispatch_batches: None
—split_batches: None
—include_tokens_per_second: False
—include_num_input_tokens_seen: False
—neftune_noise_alpha: None
—optim_target_modules: None
—batch_eval_metrics: False
—eval_on_start: False
—use_liger_kernel: False
—eval_use_gather_object: False
—average_tokens_across_devices: False
—prompts: None
—batch_sampler: batch_sampler
—multi_dataset_batch_sampler: proportional

</details>

Training Logs

Epoch	Step	Training Loss	multi stsb de loss	multi stsb es loss	multi stsb fr loss	multi stsb it loss	multi stsb nl loss	multi stsb pl loss	multi stsb pt loss	multi stsb ru loss	multi stsb zh loss	sts-eval_spearman_cosine	sts-test_spearman_cosine
1.0	3240	4.6594	4.6488	4.6520	4.6401	4.6637	4.6435	4.6943	4.6786	4.6902	4.6578	0.5620	-
2.0	6480	4.4285	4.6860	4.6755	4.6796	4.6655	4.6472	4.7655	4.6910	4.7783	4.6939	0.6592	-
3.0	9720	4.1541	4.9416	5.0391	4.9025	4.9229	4.9449	5.0618	5.0057	5.0001	4.9986	0.6764	-
4.0	12960	3.8671	5.3776	5.5136	5.3842	5.3216	5.3303	5.4847	5.4591	5.3623	5.4139	0.6865	0.6617

Framework Versions

—Python: 3.11.10
—Sentence Transformers: 3.3.1
—Transformers: 4.47.1
—PyTorch: 2.3.1+cu121
—Accelerate: 1.2.1
—Datasets: 3.2.0
—Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CoSENTLoss

bibtex

@online{kexuefm-8847,
    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
    author={Su Jianlin},
    year={2022},
    month={Jan},
    url={https://kexue.fm/archives/8847},
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->