ritulk/MPNET_finetuned_on_stsb_multi_mt_dataset
050
1---2tags:3- sentence-transformers4- sentence-similarity5- feature-extraction6- generated_from_trainer7- dataset_size:57498- loss:MultipleNegativesRankingLoss9- loss:CosineSimilarityLoss10base_model: ritulk/MPNET-fine-tuned-political-clustering11widget:12- source_sentence: Der Mann hat über die Internetkamera mit einem Mädchen gesprochen.13 sentences:14 - Eine Gruppe älterer Menschen posiert um einen Esstisch.15 - Ein Teenager spricht über eine Webcam mit einem Mädchen.16 - >-17 Mindestlohngesetze schaden den am wenigsten Qualifizierten, den am wenigsten18 Produktiven am meisten.19- source_sentence: Eine Frau schreibt etwas.20 sentences:21 - Es gibt kein "Standbild", das nicht relativ zu einem anderen Objekt ist.22 - >-23 Ein blondhaariges Kind, das vor einem Haus auf der Trompete spielt, während24 sein jüngerer Bruder zusieht.25 - Eine Frau schneidet grüne Zwiebeln.26- source_sentence: >-27 Sterne entstehen in Sternentstehungsgebieten, die ihrerseits aus28 Molekülwolken entstehen.29 sentences:30 - >-31 Sie bezieht sich auf die maximale Blendenzahl (definiert als das Verhältnis32 von Brennweite zu effektivem Blendendurchmesser).33 - >-34 Es ist möglich, dass ein Sonnensystem wie unseres außerhalb einer Galaxie35 existiert.36 - >-37 Es gibt einen sehr guten Grund, die Gattin der Königin nicht als "König" zu38 bezeichnen - denn sie sind nicht der König.39- source_sentence: Der Spieler schießt die Siegpunkte.40 sentences:41 - Die Dame frittierte das panierte Fleisch in heißem Öl.42 - Der Basketballspieler ist dabei, Punkte für sein Team zu sammeln.43 - >-44 Obwohl ich glaube, dass Searle sich irrt, glaube ich nicht, dass Sie das45 Problem gefunden haben.46- source_sentence: Zwei Weißkopfseeadler auf einem Ast.47 sentences:48 - Die Frau schneidet Kartoffeln.49 - Ein Mann, der in einem Raum auf dem Boden sitzt, klimpert auf einer Gitarre.50 - Zwei Adler sitzen auf einem Ast.51pipeline_tag: sentence-similarity52library_name: sentence-transformers53metrics:54- pearson_cosine55- spearman_cosine56model-index:57- name: SentenceTransformer based on ritulk/MPNET-fine-tuned-political-clustering58 results:59 - task:60 type: semantic-similarity61 name: Semantic Similarity62 dataset:63 name: Unknown64 type: unknown65 metrics:66 - type: pearson_cosine67 value: 0.656810847517478468 name: Pearson Cosine69 - type: spearman_cosine70 value: 0.65762142513048971 name: Spearman Cosine72 - type: pearson_cosine73 value: 0.675955748015631574 name: Pearson Cosine75 - type: spearman_cosine76 value: 0.675038332565139677 name: Spearman Cosine78 - type: pearson_cosine79 value: 0.764099679245965180 name: Pearson Cosine81 - type: spearman_cosine82 value: 0.761924873027734483 name: Spearman Cosine84datasets:85- PhilipMay/stsb_multi_mt86---87 88# SentenceTransformer based on ritulk/MPNET-fine-tuned-political-clustering89 90This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [ritulk/MPNET-fine-tuned-political-clustering](https://huggingface.co/ritulk/MPNET-fine-tuned-political-clustering). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.91 92## Model Details93 94### Model Description95- **Model Type:** Sentence Transformer96- **Base model:** [ritulk/MPNET-fine-tuned-political-clustering](https://huggingface.co/ritulk/MPNET-fine-tuned-political-clustering) <!-- at revision ae9c82780eb3f2f97dd6943140a34e78030ce7bd -->97- **Maximum Sequence Length:** 512 tokens98- **Output Dimensionality:** 768 dimensions99- **Similarity Function:** Cosine Similarity100<!-- - **Training Dataset:** Unknown -->101<!-- - **Language:** Unknown -->102<!-- - **License:** Unknown -->103 104### Model Sources105 106- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)107- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)108- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)109 110### Full Model Architecture111 112```113SentenceTransformer(114 (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: MPNetModel 115 (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})116)117```118 119## Usage120 121### Direct Usage (Sentence Transformers)122 123First install the Sentence Transformers library:124 125```bash126pip install -U sentence-transformers127```128 129Then you can load this model and run inference.130```python131from sentence_transformers import SentenceTransformer132 133# Download from the 🤗 Hub134model = SentenceTransformer("ritulk/MPNET_finetuned_on_stsb_multi_mt_dataset")135# Run inference136sentences = [137 'Zwei Weißkopfseeadler auf einem Ast.',138 'Zwei Adler sitzen auf einem Ast.',139 'Ein Mann, der in einem Raum auf dem Boden sitzt, klimpert auf einer Gitarre.',140]141embeddings = model.encode(sentences)142print(embeddings.shape)143# [3, 768]144 145# Get the similarity scores for the embeddings146similarities = model.similarity(embeddings, embeddings)147print(similarities.shape)148# [3, 3]149```150 151<!--152### Direct Usage (Transformers)153 154<details><summary>Click to see the direct usage in Transformers</summary>155 156</details>157-->158 159<!--160### Downstream Usage (Sentence Transformers)161 162You can finetune this model on your own dataset.163 164<details><summary>Click to expand</summary>165 166</details>167-->168 169<!--170### Out-of-Scope Use171 172*List how the model may foreseeably be misused and address what users ought not to do with the model.*173-->174 175## Evaluation176 177### Metrics178 179#### Semantic Similarity180 181* Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)182 183| Metric | Value |184|:--------------------|:-----------|185| pearson_cosine | 0.6568 |186| **spearman_cosine** | **0.6576** |187 188#### Semantic Similarity189 190* Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)191 192| Metric | Value |193|:--------------------|:----------|194| pearson_cosine | 0.676 |195| **spearman_cosine** | **0.675** |196 197#### Semantic Similarity198 199* Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)200 201| Metric | Value |202|:--------------------|:-----------|203| pearson_cosine | 0.7641 |204| **spearman_cosine** | **0.7619** |205 206<!--207## Bias, Risks and Limitations208 209*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*210-->211 212<!--213### Recommendations214 215*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*216-->217 218## Training Details219 220### Training Dataset221 222#### PhilipMay/stsb_multi_mt223 224* Size: 5,749 training samples225* Columns: <code>text</code>, <code>text_pair</code>, and <code>score</code>226* Approximate statistics based on the first 1000 samples:227 | | text | text_pair | score |228 |:--------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------|229 | type | string | string | float |230 | details | <ul><li>min: 7 tokens</li><li>mean: 14.58 tokens</li><li>max: 49 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 14.6 tokens</li><li>max: 47 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.45</li><li>max: 1.0</li></ul> |231* Samples:232 | text | text_pair | score |233 |:---------------------------------------------------------------|:--------------------------------------------------------------------------|:--------------------------------|234 | <code>Ein Flugzeug hebt gerade ab.</code> | <code>Ein Flugzeug hebt gerade ab.</code> | <code>1.0</code> |235 | <code>Ein Mann spielt eine große Flöte.</code> | <code>Ein Mann spielt eine Flöte.</code> | <code>0.7599999904632568</code> |236 | <code>Ein Mann streicht geriebenen Käse auf eine Pizza.</code> | <code>Ein Mann streicht geriebenen Käse auf eine ungekochte Pizza.</code> | <code>0.7599999904632568</code> |237* Loss: [<code>CosineSimilarityLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters:238 ```json239 {240 "loss_fct": "torch.nn.modules.loss.MSELoss"241 }242 ```243 244### Evaluation Dataset245 246#### Unnamed Dataset247 248* Size: 1,500 evaluation samples249* Columns: <code>text</code>, <code>text_pair</code>, and <code>score</code>250* Approximate statistics based on the first 1000 samples:251 | | text | text_pair | score |252 |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------|253 | type | string | string | float |254 | details | <ul><li>min: 6 tokens</li><li>mean: 25.19 tokens</li><li>max: 65 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 25.21 tokens</li><li>max: 70 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.42</li><li>max: 1.0</li></ul> |255* Samples:256 | text | text_pair | score |257 |:-------------------------------------------------------------|:-----------------------------------------------------------|:------------------|258 | <code>Ein Mann mit einem Schutzhelm tanzt.</code> | <code>Ein Mann mit einem Schutzhelm tanzt.</code> | <code>1.0</code> |259 | <code>Ein kleines Kind reitet auf einem Pferd.</code> | <code>Ein Kind reitet auf einem Pferd.</code> | <code>0.95</code> |260 | <code>Ein Mann verfüttert eine Maus an eine Schlange.</code> | <code>Der Mann füttert die Schlange mit einer Maus.</code> | <code>1.0</code> |261* Loss: [<code>CosineSimilarityLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters:262 ```json263 {264 "loss_fct": "torch.nn.modules.loss.MSELoss"265 }266 ```267 268### Training Hyperparameters269#### Non-Default Hyperparameters270 271- `eval_strategy`: steps272- `per_device_train_batch_size`: 16273- `per_device_eval_batch_size`: 16274- `num_train_epochs`: 5275- `warmup_ratio`: 0.1276- `fp16`: True277- `batch_sampler`: no_duplicates278 279#### All Hyperparameters280<details><summary>Click to expand</summary>281 282- `overwrite_output_dir`: False283- `do_predict`: False284- `eval_strategy`: steps285- `prediction_loss_only`: True286- `per_device_train_batch_size`: 16287- `per_device_eval_batch_size`: 16288- `per_gpu_train_batch_size`: None289- `per_gpu_eval_batch_size`: None290- `gradient_accumulation_steps`: 1291- `eval_accumulation_steps`: None292- `torch_empty_cache_steps`: None293- `learning_rate`: 5e-05294- `weight_decay`: 0.0295- `adam_beta1`: 0.9296- `adam_beta2`: 0.999297- `adam_epsilon`: 1e-08298- `max_grad_norm`: 1.0299- `num_train_epochs`: 5300- `max_steps`: -1301- `lr_scheduler_type`: linear302- `lr_scheduler_kwargs`: {}303- `warmup_ratio`: 0.1304- `warmup_steps`: 0305- `log_level`: passive306- `log_level_replica`: warning307- `log_on_each_node`: True308- `logging_nan_inf_filter`: True309- `save_safetensors`: True310- `save_on_each_node`: False311- `save_only_model`: False312- `restore_callback_states_from_checkpoint`: False313- `no_cuda`: False314- `use_cpu`: False315- `use_mps_device`: False316- `seed`: 42317- `data_seed`: None318- `jit_mode_eval`: False319- `use_ipex`: False320- `bf16`: False321- `fp16`: True322- `fp16_opt_level`: O1323- `half_precision_backend`: auto324- `bf16_full_eval`: False325- `fp16_full_eval`: False326- `tf32`: None327- `local_rank`: 0328- `ddp_backend`: None329- `tpu_num_cores`: None330- `tpu_metrics_debug`: False331- `debug`: []332- `dataloader_drop_last`: False333- `dataloader_num_workers`: 0334- `dataloader_prefetch_factor`: None335- `past_index`: -1336- `disable_tqdm`: False337- `remove_unused_columns`: True338- `label_names`: None339- `load_best_model_at_end`: False340- `ignore_data_skip`: False341- `fsdp`: []342- `fsdp_min_num_params`: 0343- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}344- `tp_size`: 0345- `fsdp_transformer_layer_cls_to_wrap`: None346- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}347- `deepspeed`: None348- `label_smoothing_factor`: 0.0349- `optim`: adamw_torch350- `optim_args`: None351- `adafactor`: False352- `group_by_length`: False353- `length_column_name`: length354- `ddp_find_unused_parameters`: None355- `ddp_bucket_cap_mb`: None356- `ddp_broadcast_buffers`: False357- `dataloader_pin_memory`: True358- `dataloader_persistent_workers`: False359- `skip_memory_metrics`: True360- `use_legacy_prediction_loop`: False361- `push_to_hub`: False362- `resume_from_checkpoint`: None363- `hub_model_id`: None364- `hub_strategy`: every_save365- `hub_private_repo`: None366- `hub_always_push`: False367- `gradient_checkpointing`: False368- `gradient_checkpointing_kwargs`: None369- `include_inputs_for_metrics`: False370- `include_for_metrics`: []371- `eval_do_concat_batches`: True372- `fp16_backend`: auto373- `push_to_hub_model_id`: None374- `push_to_hub_organization`: None375- `mp_parameters`: 376- `auto_find_batch_size`: False377- `full_determinism`: False378- `torchdynamo`: None379- `ray_scope`: last380- `ddp_timeout`: 1800381- `torch_compile`: False382- `torch_compile_backend`: None383- `torch_compile_mode`: None384- `dispatch_batches`: None385- `split_batches`: None386- `include_tokens_per_second`: False387- `include_num_input_tokens_seen`: False388- `neftune_noise_alpha`: None389- `optim_target_modules`: None390- `batch_eval_metrics`: False391- `eval_on_start`: False392- `use_liger_kernel`: False393- `eval_use_gather_object`: False394- `average_tokens_across_devices`: False395- `prompts`: None396- `batch_sampler`: no_duplicates397- `multi_dataset_batch_sampler`: proportional398 399</details>400 401### Training Logs402| Epoch | Step | Training Loss | Validation Loss | spearman_cosine |403|:------:|:----:|:-------------:|:---------------:|:---------------:|404| 0.2778 | 100 | 0.6009 | 0.9181 | - |405| 0.5556 | 200 | 0.4724 | 0.8744 | - |406| 0.8333 | 300 | 0.449 | 0.8405 | - |407| -1 | -1 | - | - | 0.6576 |408| 0.2778 | 100 | 0.0781 | 0.9378 | - |409| 0.5556 | 200 | 0.0772 | 0.9290 | - |410| 0.8333 | 300 | 0.2281 | 0.8876 | - |411| 1.1111 | 400 | 0.3267 | 0.9336 | - |412| 1.3889 | 500 | 0.2936 | 0.8612 | - |413| 1.6667 | 600 | 0.2283 | 0.8569 | - |414| 1.9444 | 700 | 0.2448 | 0.8589 | - |415| 2.2222 | 800 | 0.1877 | 0.8418 | - |416| 2.5 | 900 | 0.1693 | 0.8351 | - |417| 2.7778 | 1000 | 0.1635 | 0.8588 | - |418| 3.0556 | 1100 | 0.1642 | 0.8260 | - |419| 3.3333 | 1200 | 0.1027 | 0.8380 | - |420| 3.6111 | 1300 | 0.0983 | 0.8407 | - |421| 3.8889 | 1400 | 0.0978 | 0.8317 | - |422| 4.1667 | 1500 | 0.1187 | 0.8376 | - |423| 4.4444 | 1600 | 0.0977 | 0.8465 | - |424| 4.7222 | 1700 | 0.0686 | 0.8492 | - |425| 5.0 | 1800 | 0.0587 | 0.8485 | - |426| -1 | -1 | - | - | 0.6750 |427| 0.2778 | 100 | 0.0656 | 0.0464 | - |428| 0.5556 | 200 | 0.0564 | 0.0454 | - |429| 0.8333 | 300 | 0.0498 | 0.0496 | - |430| 1.1111 | 400 | 0.042 | 0.0408 | - |431| 1.3889 | 500 | 0.0384 | 0.0416 | - |432| 1.6667 | 600 | 0.0319 | 0.0427 | - |433| 1.9444 | 700 | 0.0332 | 0.0427 | - |434| 2.2222 | 800 | 0.0249 | 0.0416 | - |435| 2.5 | 900 | 0.0232 | 0.0408 | - |436| 2.7778 | 1000 | 0.0219 | 0.0415 | - |437| 3.0556 | 1100 | 0.0215 | 0.0409 | - |438| 3.3333 | 1200 | 0.0158 | 0.0402 | - |439| 3.6111 | 1300 | 0.0171 | 0.0387 | - |440| 3.8889 | 1400 | 0.0152 | 0.0393 | - |441| 4.1667 | 1500 | 0.0126 | 0.0389 | - |442| 4.4444 | 1600 | 0.0124 | 0.0389 | - |443| 4.7222 | 1700 | 0.0118 | 0.0393 | - |444| 5.0 | 1800 | 0.0127 | 0.0391 | - |445| -1 | -1 | - | - | 0.7619 |446 447 448### Framework Versions449- Python: 3.11.12450- Sentence Transformers: 4.0.2451- Transformers: 4.50.3452- PyTorch: 2.6.0+cu124453- Accelerate: 1.5.2454- Datasets: 3.5.0455- Tokenizers: 0.21.1456 457## Citation458 459### BibTeX460 461#### Sentence Transformers462```bibtex463@inproceedings{reimers-2019-sentence-bert,464 title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",465 author = "Reimers, Nils and Gurevych, Iryna",466 booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",467 month = "11",468 year = "2019",469 publisher = "Association for Computational Linguistics",470 url = "https://arxiv.org/abs/1908.10084",471}472```473 474<!--475## Glossary476 477*Clearly define terms in order to be accessible across audiences.*478-->479 480<!--481## Model Card Authors482 483*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*484-->485 486<!--487## Model Card Contact488 489*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*490-->