KhaledReda/all-MiniLM-L6-v57-pair_score

all-MiniLM-L6-v57-pair_score

This is a sentence-transformers model finetuned from KhaledReda/all-MiniLM-L6-v55-pair_score on the pairs_with_scores_v46 dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• —Model Type: Sentence Transformer
• —Base model: KhaledReda/all-MiniLM-L6-v55-pair_score <!-- at revision 0d6070f921bab24f725f26605a711ff96a295e6a -->
• —Maximum Sequence Length: 256 tokens
• —Output Dimensionality: 384 dimensions
• —Similarity Function: Cosine Similarity
• —Training Dataset:
• —pairs_with_scores_v46
• —Language: en
• —License: apache-2.0

Model Sources

• —Documentation: Sentence Transformers Documentation
• —Repository: Sentence Transformers on GitHub
• —Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'peeled shrimps',
    'bakeware utensil rose gold cake stand brass cake stand mazari cake stand 45 inches round brass rose gold brass cake stand.',
    'rice strainer stainer plastic strainer transparent bowl strainer white strainer lid strainer homepro strainer 1 bowl rice plastic transparent and white material made from durable high-quality plastic. design transparent bowl with a white slotted lid for easy straining. handle equipped with a sturdy ergonomic handle for comfortable grip and easy handling. functionality efficiently drains water from rice pasta or other grains. versatility can be used for washing and straining various ingredients. convenience the lid features integrated slots to ensure smooth water flow while preventing food from escaping. easy to clean smooth surfaces and detachable parts for hassle-free cleaning.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, -0.0809,  0.0101],
#         [-0.0809,  1.0000,  0.1054],
#         [ 0.0101,  0.1054,  1.0000]])

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

pairswithscores_v46

• —Dataset: pairs_with_scores_v46 at 81054f8
• —Size: 77,996,746 training samples
• —Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
• —Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:---------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 3 tokens</li><li>mean: 6.55 tokens</li><li>max: 19 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 46.46 tokens</li><li>max: 246 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.24</li><li>max: 1.0</li></ul> |
• —Samples: | sentence1 | sentence2 | score | |:---------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------| | <code>mandala</code> | <code>dog grooming omni guard grooming wipes 40 extra-thick soft and moist pet wipes. convenient travel pack you can toss in your bag purse or car away dirt and odor. brighten and use for touch up or bath altemative. nice fragrance and contains aloe vera alcohol and lanolin-free pet grooming products for cleaning pet paws and undercoats. protects against environmental damage with antioxidant properties of ingredients. will not remove spot-on flea control treatments.</code> | <code>0.0</code> | | <code>telescopic scrapper</code> | <code>azelast nasal spray</code> | <code>0.0</code> | | <code>niacinamide</code> | <code>reby - curly patterns towel mint bath towel soft melange effect towel colorful curly towel easy care curly towel soft and colorful towel stylish curly towel low maintenance curly towel reby towel ricrac towel reby modern rectangular mint curly patterns this cool curly towel offers you a range of 7 different colors giving you a soft mélange effect of combining the softness of the material and the accentuated colors. wash on 40 c tumble dry do not bleach low heat ironing.</code> | <code>0.0</code> |
• —Loss: <code>CoSENTLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

Evaluation Dataset

pairswithscores_v46

• —Dataset: pairs_with_scores_v46 at 81054f8
• —Size: 391,944 evaluation samples
• —Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
• —Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:---------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 3 tokens</li><li>mean: 6.43 tokens</li><li>max: 29 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 45.85 tokens</li><li>max: 256 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.23</li><li>max: 1.0</li></ul> |
• —Samples: | sentence1 | sentence2 | score | |:-----------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------| | <code>long sleeves cardigan</code> | <code>gel pack plaster</code> | <code>0.0</code> | | <code>office statue</code> | <code>shirt boucle dress w618 women just4women s dress polyester boucle fabric black product details w618 fabric 100 polyestersize of the item on the image size s height 100cm chest 36-35delivery this item will be delivered from 3-5 dayswashing instruction wash inside out at or below 30 cdelicate and gentle machine wash cyclesdo not bleachdon t tumble drydrape flat drying in the shadesteam iron only</code> | <code>0.0</code> | | <code>anti hair loss spray</code> | <code>wood pouf</code> | <code>0.0</code> |
• —Loss: <code>CoSENTLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

Training Hyperparameters

Non-Default Hyperparameters

• —eval_strategy: steps
• —per_device_train_batch_size: 64
• —per_device_eval_batch_size: 64
• —learning_rate: 2e-05
• —num_train_epochs: 1
• —warmup_ratio: 0.1
• —fp16: True

All Hyperparameters

<details><summary>Click to expand</summary>

• —overwrite_output_dir: False
• —do_predict: False
• —eval_strategy: steps
• —prediction_loss_only: True
• —per_device_train_batch_size: 64
• —per_device_eval_batch_size: 64
• —per_gpu_train_batch_size: None
• —per_gpu_eval_batch_size: None
• —gradient_accumulation_steps: 1
• —eval_accumulation_steps: None
• —torch_empty_cache_steps: None
• —learning_rate: 2e-05
• —weight_decay: 0.0
• —adam_beta1: 0.9
• —adam_beta2: 0.999
• —adam_epsilon: 1e-08
• —max_grad_norm: 1.0
• —num_train_epochs: 1
• —max_steps: -1
• —lr_scheduler_type: linear
• —lr_scheduler_kwargs: {}
• —warmup_ratio: 0.1
• —warmup_steps: 0
• —log_level: passive
• —log_level_replica: warning
• —log_on_each_node: True
• —logging_nan_inf_filter: True
• —save_safetensors: True
• —save_on_each_node: False
• —save_only_model: False
• —restore_callback_states_from_checkpoint: False
• —no_cuda: False
• —use_cpu: False
• —use_mps_device: False
• —seed: 42
• —data_seed: None
• —jit_mode_eval: False
• —use_ipex: False
• —bf16: False
• —fp16: True
• —fp16_opt_level: O1
• —half_precision_backend: auto
• —bf16_full_eval: False
• —fp16_full_eval: False
• —tf32: None
• —local_rank: 0
• —ddp_backend: None
• —tpu_num_cores: None
• —tpu_metrics_debug: False
• —debug: []
• —dataloader_drop_last: False
• —dataloader_num_workers: 0
• —dataloader_prefetch_factor: None
• —past_index: -1
• —disable_tqdm: False
• —remove_unused_columns: True
• —label_names: None
• —load_best_model_at_end: False
• —ignore_data_skip: False
• —fsdp: []
• —fsdp_min_num_params: 0
• —fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
• —fsdp_transformer_layer_cls_to_wrap: None
• —accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
• —deepspeed: None
• —label_smoothing_factor: 0.0
• —optim: adamw_torch
• —optim_args: None
• —adafactor: False
• —group_by_length: False
• —length_column_name: length
• —ddp_find_unused_parameters: None
• —ddp_bucket_cap_mb: None
• —ddp_broadcast_buffers: False
• —dataloader_pin_memory: True
• —dataloader_persistent_workers: False
• —skip_memory_metrics: True
• —use_legacy_prediction_loop: False
• —push_to_hub: False
• —resume_from_checkpoint: None
• —hub_model_id: None
• —hub_strategy: every_save
• —hub_private_repo: None
• —hub_always_push: False
• —hub_revision: None
• —gradient_checkpointing: False
• —gradient_checkpointing_kwargs: None
• —include_inputs_for_metrics: False
• —include_for_metrics: []
• —eval_do_concat_batches: True
• —fp16_backend: auto
• —push_to_hub_model_id: None
• —push_to_hub_organization: None
• —mp_parameters:
• —auto_find_batch_size: False
• —full_determinism: False
• —torchdynamo: None
• —ray_scope: last
• —ddp_timeout: 1800
• —torch_compile: False
• —torch_compile_backend: None
• —torch_compile_mode: None
• —include_tokens_per_second: False
• —include_num_input_tokens_seen: False
• —neftune_noise_alpha: None
• —optim_target_modules: None
• —batch_eval_metrics: False
• —eval_on_start: False
• —use_liger_kernel: False
• —liger_kernel_config: None
• —eval_use_gather_object: False
• —average_tokens_across_devices: False
• —prompts: None
• —batch_sampler: batch_sampler
• —multi_dataset_batch_sampler: proportional
• —router_mapping: {}
• —learning_rate_mapping: {}

</details>

Training Logs

<details><summary>Click to expand</summary>