KhaledReda/all-MiniLM-L6-v67-pair_score

all-MiniLM-L6-v67-pair_score

This is a sentence-transformers model finetuned from KhaledReda/all-MiniLM-L6-v66-pair_score on the pairs_with_scores_v54 dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• —Model Type: Sentence Transformer
• —Base model: KhaledReda/all-MiniLM-L6-v66-pair_score <!-- at revision 4fdea10e45e818ae541c474e59c3a5ed62897a9d -->
• —Maximum Sequence Length: 256 tokens
• —Output Dimensionality: 384 dimensions
• —Similarity Function: Cosine Similarity
• —Training Dataset:
• —pairs_with_scores_v54
• —Language: en
• —License: apache-2.0

Model Sources

• —Documentation: Sentence Transformers Documentation
• —Repository: Sentence Transformers on GitHub
• —Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'sigvaris code 10221 arm without a palm',
    'wooden raised bed square raised bed growing raised bed waterresistant raised bed ready systems units grow more in less space with the square foot gardening inspired wooden raised bed perfect for small and big spaces and gardens if you want to start growing your own organic food. one 3ftx3ft bed produces a salad for one child every day of the growing season. one 4ftx4ft bed produces a salad for one adult one 4ftx8ft bed produces a salad for two adults all beds are made of high-grade water-resistant african teak wood and come with mel s mix an organic custom soil mix that lasts from 7 to 10 years the soil mix consists of peat moss perlite coarse vermiculite vermicompost and is 100 organic. price includes deliver',
    'novell nail fashion sticker 16pcs/42 nail sticker novell nail sticker',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, -0.0532, -0.0843],
#         [-0.0532,  1.0000, -0.1830],
#         [-0.0843, -0.1830,  1.0000]])

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

pairswithscores_v54

• —Dataset: pairs_with_scores_v54 at cd47d78
• —Size: 12,834,902 training samples
• —Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
• —Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:---------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 3 tokens</li><li>mean: 7.46 tokens</li><li>max: 23 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 56.15 tokens</li><li>max: 234 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.26</li><li>max: 1.0</li></ul> |
• —Samples: | sentence1 | sentence2 | score | |:--------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------| | <code>nivea deep bcarbedeo ron fm 15</code> | <code>canon maxify gx7040 printer canon maxify gx 7040 printer maxify printer</code> | <code>0.0</code> | | <code>paris velvet the pencil</code> | <code>tender beef piccata white rice tender beef strips slow cooked tender beef brown sauce tender beef tender piccata white rice piccata tender beef strips slow cooked in brown sauce and served with a side of white rice.</code> | <code>0.0</code> | | <code>golden hour slipper</code> | <code>iris in metallic fuschia raffia metallic fuchsia slippers metallic slippers fuchsia slippers pool side slippers women slippers summer slippers iris slippers raffia slippers step out in our iris sandals. detailed design and slender sole these simple sandals pair perfectly with any outfit. it looks great from poolside to night walks. wear with cutoff jeans or a simple flowing sun dress. size fits true to size if you wear a half size take next size up. ex if you wear 38.5 order 39</code> | <code>1.0</code> |
• —Loss: <code>CoSENTLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

Evaluation Dataset

pairswithscores_v54

• —Dataset: pairs_with_scores_v54 at cd47d78
• —Size: 64,497 evaluation samples
• —Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
• —Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | score | |:--------|:---------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 4 tokens</li><li>mean: 7.37 tokens</li><li>max: 28 tokens</li></ul> | <ul><li>min: 4 tokens</li><li>mean: 58.16 tokens</li><li>max: 231 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.26</li><li>max: 1.0</li></ul> |
• —Samples: | sentence1 | sentence2 | score | |:----------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------| | <code>tunisie artemis-ivory coffee pot - rim</code> | <code>al alwani ajwa al madinah dates ajwa dates al alwani dates premium soft saudi dates with a fruity flavour and fine texture. dates are rich in fibre vitamins and minerals including soluble fibre potassium and iron.</code> | <code>0.0</code> | | <code>eye ring plated with</code> | <code>crown silver ring handcrafted ring 925 silver ring crown ring from our unique designs we created these beautiful pieces.-925 silver.</code> | <code>1.0</code> | | <code>macarona fusilli</code> | <code>tritace 1.25mg 14/tab 2ex.new tritace tablets</code> | <code>0.0</code> |
• —Loss: <code>CoSENTLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

Training Hyperparameters

Non-Default Hyperparameters

• —eval_strategy: steps
• —per_device_train_batch_size: 64
• —per_device_eval_batch_size: 64
• —learning_rate: 2e-05
• —num_train_epochs: 2
• —warmup_ratio: 0.1
• —fp16: True

All Hyperparameters

<details><summary>Click to expand</summary>

• —overwrite_output_dir: False
• —do_predict: False
• —eval_strategy: steps
• —prediction_loss_only: True
• —per_device_train_batch_size: 64
• —per_device_eval_batch_size: 64
• —per_gpu_train_batch_size: None
• —per_gpu_eval_batch_size: None
• —gradient_accumulation_steps: 1
• —eval_accumulation_steps: None
• —torch_empty_cache_steps: None
• —learning_rate: 2e-05
• —weight_decay: 0.0
• —adam_beta1: 0.9
• —adam_beta2: 0.999
• —adam_epsilon: 1e-08
• —max_grad_norm: 1.0
• —num_train_epochs: 2
• —max_steps: -1
• —lr_scheduler_type: linear
• —lr_scheduler_kwargs: {}
• —warmup_ratio: 0.1
• —warmup_steps: 0
• —log_level: passive
• —log_level_replica: warning
• —log_on_each_node: True
• —logging_nan_inf_filter: True
• —save_safetensors: True
• —save_on_each_node: False
• —save_only_model: False
• —restore_callback_states_from_checkpoint: False
• —no_cuda: False
• —use_cpu: False
• —use_mps_device: False
• —seed: 42
• —data_seed: None
• —jit_mode_eval: False
• —use_ipex: False
• —bf16: False
• —fp16: True
• —fp16_opt_level: O1
• —half_precision_backend: auto
• —bf16_full_eval: False
• —fp16_full_eval: False
• —tf32: None
• —local_rank: 0
• —ddp_backend: None
• —tpu_num_cores: None
• —tpu_metrics_debug: False
• —debug: []
• —dataloader_drop_last: False
• —dataloader_num_workers: 0
• —dataloader_prefetch_factor: None
• —past_index: -1
• —disable_tqdm: False
• —remove_unused_columns: True
• —label_names: None
• —load_best_model_at_end: False
• —ignore_data_skip: False
• —fsdp: []
• —fsdp_min_num_params: 0
• —fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
• —fsdp_transformer_layer_cls_to_wrap: None
• —accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
• —deepspeed: None
• —label_smoothing_factor: 0.0
• —optim: adamw_torch
• —optim_args: None
• —adafactor: False
• —group_by_length: False
• —length_column_name: length
• —ddp_find_unused_parameters: None
• —ddp_bucket_cap_mb: None
• —ddp_broadcast_buffers: False
• —dataloader_pin_memory: True
• —dataloader_persistent_workers: False
• —skip_memory_metrics: True
• —use_legacy_prediction_loop: False
• —push_to_hub: False
• —resume_from_checkpoint: None
• —hub_model_id: None
• —hub_strategy: every_save
• —hub_private_repo: None
• —hub_always_push: False
• —hub_revision: None
• —gradient_checkpointing: False
• —gradient_checkpointing_kwargs: None
• —include_inputs_for_metrics: False
• —include_for_metrics: []
• —eval_do_concat_batches: True
• —fp16_backend: auto
• —push_to_hub_model_id: None
• —push_to_hub_organization: None
• —mp_parameters:
• —auto_find_batch_size: False
• —full_determinism: False
• —torchdynamo: None
• —ray_scope: last
• —ddp_timeout: 1800
• —torch_compile: False
• —torch_compile_backend: None
• —torch_compile_mode: None
• —include_tokens_per_second: False
• —include_num_input_tokens_seen: False
• —neftune_noise_alpha: None
• —optim_target_modules: None
• —batch_eval_metrics: False
• —eval_on_start: False
• —use_liger_kernel: False
• —liger_kernel_config: None
• —eval_use_gather_object: False
• —average_tokens_across_devices: False
• —prompts: None
• —batch_sampler: batch_sampler
• —multi_dataset_batch_sampler: proportional
• —router_mapping: {}
• —learning_rate_mapping: {}

</details>

Training Logs

<details><summary>Click to expand</summary>