CoolFace
Modelpublic

spl4shedEdu/mpnet_IAB

sourceHugging Faceupdated 1y agoView on Hugging Face
0likes103downloads
Model Card

SentenceTransformer based on sentence-transformers/all-mpnet-base-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-mpnet-base-v2. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • —Model Type: Sentence Transformer
  • —Base model: sentence-transformers/all-mpnet-base-v2 <!-- at revision 84f2bcc00d77236f9e89c8a360a00fb1139bf47d -->
  • —Maximum Sequence Length: 384 tokens
  • —Output Dimensionality: 768 tokens
  • —Similarity Function: Cosine Similarity <!-- - Training Dataset: Unknown --> <!-- - Language: Unknown --> <!-- - License: Unknown -->

Model Sources

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 384, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

bash
pip install -U sentence-transformers

Then you can load this model and run inference.

python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'paint sealant sonax profiline polymer net shield 75 ml aerosol can 1994 bmw 318is base coupe miscellaneous page 24 note innovative surface protection based on hybrid polymers protects the paintwork by means of a resistant network made from organic and inorganic components can be applied quickly easily intensively freshens up paint color produces silky smooth with an outstanding drip off effect one 75 ml should complete average size car produced by sonax identifiers is 223000m941 category of automotive',
    'paint sealant sonax profiline polymer net shield 75 ml aerosol can 1991 bmw 325i base convertible miscellaneous page 23 note innovative surface protection based on hybrid polymers protects the paintwork by means of a resistant network made from organic and inorganic components can be applied quickly easily intensively freshens up paint color produces silky smooth with an outstanding drip off effect one 75 ml should complete average size car produced by sonax identifiers is 223000m941 category of automotive',
    'honeywell accessories for terminal cod99exmb12 honeywell cod871238012 honeywell dolphin 99ex mobile base vehicle kit charging cradle rs232 universal mounting bracket and 12v cigarette lighter power adapter produced by honeywell metrologic identifiers is 99exmb12 category of computersandaccessories',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

Unnamed Dataset
  • —Size: 281,342 training samples
  • —Columns: <code>anchor</code> and <code>positive</code>
  • —Approximate statistics based on the first 1000 samples: | | anchor | positive | |:--------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | | details | <ul><li>min: 25 tokens</li><li>mean: 74.69 tokens</li><li>max: 384 tokens</li></ul> | <ul><li>min: 25 tokens</li><li>mean: 75.63 tokens</li><li>max: 384 tokens</li></ul> |
  • —Samples: | anchor | positive | |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>honeywell hand held products dolphin 99509951 series mobile computer usb cable 6 ft 18m 80000355e usb cable 6 ft 18m identifiers is 80000355e category of computersandaccessories</code> | <code>hand held usb cable 6 ft hand ft 80000355e scanner accessories cdwcom hand held products is the leading provider of imagebased data collection solutions for mobile wireless and transaction processing applications to end users throughout world by investing in hhp products its customers are able reduce costs improve service position their companies future growth identifiers is 26121604 category of computersandaccessories</code> | | <code>intake boot air mass sensor to throttle housing 1995 bmw 318i base convertible intake system page 2 note from 0994 produced by oem identifiers is 13711247829m58 category of automotive</code> | <code>intake boot air mass sensor to throttle housing 1995 bmw 318i base convertible intake system page 2 produced by crp identifiers is 13711247829int category of automotive</code> | | <code>blue sky panorama with transparent clouds vector image sky images over 150 000 vector blue sky panorama with transparent clouds vector background image identifiers is 15266707 category of officeproducts</code> | <code>blue sky panorama with transparent clouds vector image images within landscapes nature over 55 000 vector blue sky panorama with transparent clouds vector background image identifiers is 15266707 category of officeproducts</code> |
  • —Loss: <code>CachedMultipleNegativesRankingLoss</code> with these parameters:
json
  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }

Evaluation Dataset

Unnamed Dataset
  • —Size: 70,336 evaluation samples
  • —Columns: <code>anchor</code> and <code>positive</code>
  • —Approximate statistics based on the first 1000 samples: | | anchor | positive | |:--------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | | details | <ul><li>min: 25 tokens</li><li>mean: 78.58 tokens</li><li>max: 384 tokens</li></ul> | <ul><li>min: 22 tokens</li><li>mean: 81.44 tokens</li><li>max: 384 tokens</li></ul> |
  • —Samples: | anchor | positive | |:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>heater hose inlet from cylinder head to water valve 1997 bmw 318i base sedan heater system page 3 produced by genuine bmw identifiers is 64211394295boe category of automotive</code> | <code>heater hose inlet from cylinder head to water valve 1996 bmw 318i base convertible heater system page 3 produced by genuine bmw identifiers is 64211394295boe category of automotive</code> | | <code>harris harris group inc group 1 full quote netdaniacom produced by source nasdaq identifiers is isinus4138331040 category of toolsandhomeimprovement</code> | <code>harris harris group inc 1 statistics netdaniacom group produced by source nasdaq identifiers is isinus4138331040 category of toolsandhomeimprovement</code> | | <code>swiffer dusters with extendable handledusters plastic handle extends to 3 ft 1 per kit handledusters ft kitpag82074 buy online at janeice products identifiers is pag82074 category of toolsandhomeimprovement key specifications are weight per case std pkg quantity package one handle and three dusters description includes item cube 008276 upc code 037000447504 pack 00037000820741 length 092 width 022 height 042 0476</code> | <code>6 pack value bundle pag82074 dusters plastic handle extends to 3 ft 1 dusters per kitus feather page 5 the janitorial marketus now its easier than ever to get those hardtoreach places pivoting head can be adjusted and locked into place for cleaning angled surfaces such as ceiling fans cabinet corners baseboards refill dusters sold separately one handle three per box bristle material fiber color white plastic greenus produced by pag82074us identifiers is pag82074 category of toolsandhomeimprovement</code> |
  • —Loss: <code>CachedMultipleNegativesRankingLoss</code> with these parameters:
json
  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }

Training Hyperparameters

Non-Default Hyperparameters
  • —eval_strategy: steps
  • —learning_rate: 1e-05
  • —num_train_epochs: 2
  • —warmup_ratio: 0.1
  • —fp16: True
  • —auto_find_batch_size: True
  • —batch_sampler: no_duplicates
All Hyperparameters

<details><summary>Click to expand</summary>

  • —overwrite_output_dir: False
  • —do_predict: False
  • —eval_strategy: steps
  • —prediction_loss_only: True
  • —per_device_train_batch_size: 8
  • —per_device_eval_batch_size: 8
  • —per_gpu_train_batch_size: None
  • —per_gpu_eval_batch_size: None
  • —gradient_accumulation_steps: 1
  • —eval_accumulation_steps: None
  • —torch_empty_cache_steps: None
  • —learning_rate: 1e-05
  • —weight_decay: 0.0
  • —adam_beta1: 0.9
  • —adam_beta2: 0.999
  • —adam_epsilon: 1e-08
  • —max_grad_norm: 1.0
  • —num_train_epochs: 2
  • —max_steps: -1
  • —lr_scheduler_type: linear
  • —lr_scheduler_kwargs: {}
  • —warmup_ratio: 0.1
  • —warmup_steps: 0
  • —log_level: passive
  • —log_level_replica: warning
  • —log_on_each_node: True
  • —logging_nan_inf_filter: True
  • —save_safetensors: True
  • —save_on_each_node: False
  • —save_only_model: False
  • —restore_callback_states_from_checkpoint: False
  • —no_cuda: False
  • —use_cpu: False
  • —use_mps_device: False
  • —seed: 42
  • —data_seed: None
  • —jit_mode_eval: False
  • —use_ipex: False
  • —bf16: False
  • —fp16: True
  • —fp16_opt_level: O1
  • —half_precision_backend: auto
  • —bf16_full_eval: False
  • —fp16_full_eval: False
  • —tf32: None
  • —local_rank: 0
  • —ddp_backend: None
  • —tpu_num_cores: None
  • —tpu_metrics_debug: False
  • —debug: []
  • —dataloader_drop_last: False
  • —dataloader_num_workers: 0
  • —dataloader_prefetch_factor: None
  • —past_index: -1
  • —disable_tqdm: False
  • —remove_unused_columns: True
  • —label_names: None
  • —load_best_model_at_end: False
  • —ignore_data_skip: False
  • —fsdp: []
  • —fsdp_min_num_params: 0
  • —fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
  • —fsdp_transformer_layer_cls_to_wrap: None
  • —accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
  • —deepspeed: None
  • —label_smoothing_factor: 0.0
  • —optim: adamw_torch
  • —optim_args: None
  • —adafactor: False
  • —group_by_length: False
  • —length_column_name: length
  • —ddp_find_unused_parameters: None
  • —ddp_bucket_cap_mb: None
  • —ddp_broadcast_buffers: False
  • —dataloader_pin_memory: True
  • —dataloader_persistent_workers: False
  • —skip_memory_metrics: True
  • —use_legacy_prediction_loop: False
  • —push_to_hub: False
  • —resume_from_checkpoint: None
  • —hub_model_id: None
  • —hub_strategy: every_save
  • —hub_private_repo: False
  • —hub_always_push: False
  • —gradient_checkpointing: False
  • —gradient_checkpointing_kwargs: None
  • —include_inputs_for_metrics: False
  • —eval_do_concat_batches: True
  • —fp16_backend: auto
  • —push_to_hub_model_id: None
  • —push_to_hub_organization: None
  • —mp_parameters:
  • —auto_find_batch_size: True
  • —full_determinism: False
  • —torchdynamo: None
  • —ray_scope: last
  • —ddp_timeout: 1800
  • —torch_compile: False
  • —torch_compile_backend: None
  • —torch_compile_mode: None
  • —dispatch_batches: None
  • —split_batches: None
  • —include_tokens_per_second: False
  • —include_num_input_tokens_seen: False
  • —neftune_noise_alpha: None
  • —optim_target_modules: None
  • —batch_eval_metrics: False
  • —eval_on_start: False
  • —eval_use_gather_object: False
  • —batch_sampler: no_duplicates
  • —multi_dataset_batch_sampler: proportional

</details>

Training Logs

EpochStepTraining Lossloss
0.199070000.00830.0029
0.3981140000.00260.0019
0.5971210000.00150.0014
0.7962280000.00130.0011
0.9952350000.00130.0010
1.1943420000.00080.0010
1.3933490000.00050.0009
1.5924560000.00030.0009

Framework Versions

  • —Python: 3.10.13
  • —Sentence Transformers: 3.0.1
  • —Transformers: 4.44.0
  • —PyTorch: 2.2.1
  • —Accelerate: 0.33.0
  • —Datasets: 2.21.0
  • —Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers
bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
CachedMultipleNegativesRankingLoss
bibtex
@misc{gao2021scaling,
    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup}, 
    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
    year={2021},
    eprint={2101.06983},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->