LamaDiab/MiniLM-V18Data-128ConstantBATCH-SemanticEngine
089
1---2tags:3- sentence-transformers4- sentence-similarity5- feature-extraction6- dense7- generated_from_trainer8- dataset_size:7059059- loss:MultipleNegativesSymmetricRankingLoss10base_model: sentence-transformers/all-MiniLM-L6-v211widget:12- source_sentence: gerber baby food fruits apples bananas & cereal13 sentences:14 - world of sweets puzzle15 - baby food16 - baby food17- source_sentence: granville original one bite original rice crispy squares18 sentences:19 - ' one bite rice crispy '20 - sweet21 - bounty wafer rolls22- source_sentence: rosa / porcelain us andalusia mug23 sentences:24 - mug25 - ' rosa mug'26 - melamine small plate - teal27- source_sentence: cetaphil sunscreen spf 50+ cream 89 ml28 sentences:29 - sunscreen30 - ' cetaphil sunscreen cream'31 - garnier intensity (6.60) intense ruby32- source_sentence: italian dolce provolone33 sentences:34 - trident - gum strawberry flavor - 5 per pack35 - experience the authentic taste of italy with our italian dolce provolone. indulge36 in its creamy texture, delicate flavors, and versatility in both simple and sophisticated37 culinary creations.38 - dairy39pipeline_tag: sentence-similarity40library_name: sentence-transformers41metrics:42- cosine_accuracy43model-index:44- name: SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v245 results:46 - task:47 type: triplet48 name: Triplet49 dataset:50 name: Unknown51 type: unknown52 metrics:53 - type: cosine_accuracy54 value: 0.965401172637939555 name: Cosine Accuracy56---57 58# SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v259 60This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.61 62## Model Details63 64### Model Description65- **Model Type:** Sentence Transformer66- **Base model:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) <!-- at revision c9745ed1d9f207416be6d2e6f8de32d1f16199bf -->67- **Maximum Sequence Length:** 256 tokens68- **Output Dimensionality:** 384 dimensions69- **Similarity Function:** Cosine Similarity70<!-- - **Training Dataset:** Unknown -->71<!-- - **Language:** Unknown -->72<!-- - **License:** Unknown -->73 74### Model Sources75 76- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)77- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)78- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)79 80### Full Model Architecture81 82```83SentenceTransformer(84 (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})85 (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})86 (2): Normalize()87)88```89 90## Usage91 92### Direct Usage (Sentence Transformers)93 94First install the Sentence Transformers library:95 96```bash97pip install -U sentence-transformers98```99 100Then you can load this model and run inference.101```python102from sentence_transformers import SentenceTransformer103 104# Download from the 🤗 Hub105model = SentenceTransformer("LamaDiab/MiniLM-V18Data-128ConstantBATCH-SemanticEngine")106# Run inference107sentences = [108 'italian dolce provolone',109 'experience the authentic taste of italy with our italian dolce provolone. indulge in its creamy texture, delicate flavors, and versatility in both simple and sophisticated culinary creations.',110 'trident - gum strawberry flavor - 5 per pack',111]112embeddings = model.encode(sentences)113print(embeddings.shape)114# [3, 384]115 116# Get the similarity scores for the embeddings117similarities = model.similarity(embeddings, embeddings)118print(similarities)119# tensor([[1.0000, 0.8659, 0.1693],120# [0.8659, 1.0000, 0.1826],121# [0.1693, 0.1826, 1.0000]])122```123 124<!--125### Direct Usage (Transformers)126 127<details><summary>Click to see the direct usage in Transformers</summary>128 129</details>130-->131 132<!--133### Downstream Usage (Sentence Transformers)134 135You can finetune this model on your own dataset.136 137<details><summary>Click to expand</summary>138 139</details>140-->141 142<!--143### Out-of-Scope Use144 145*List how the model may foreseeably be misused and address what users ought not to do with the model.*146-->147 148## Evaluation149 150### Metrics151 152#### Triplet153 154* Evaluated with [<code>TripletEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator)155 156| Metric | Value |157|:--------------------|:-----------|158| **cosine_accuracy** | **0.9654** |159 160<!--161## Bias, Risks and Limitations162 163*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*164-->165 166<!--167### Recommendations168 169*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*170-->171 172## Training Details173 174### Training Dataset175 176#### Unnamed Dataset177 178* Size: 705,905 training samples179* Columns: <code>anchor</code>, <code>positive</code>, and <code>itemCategory</code>180* Approximate statistics based on the first 1000 samples:181 | | anchor | positive | itemCategory |182 |:--------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|183 | type | string | string | string |184 | details | <ul><li>min: 3 tokens</li><li>mean: 13.19 tokens</li><li>max: 51 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 4.46 tokens</li><li>max: 93 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 3.91 tokens</li><li>max: 11 tokens</li></ul> |185* Samples:186 | anchor | positive | itemCategory |187 |:-----------------------------------------------|:-----------------------------------------|:-------------------------------|188 | <code>mango nos nos small</code> | <code>milk chocolate ganache cake</code> | <code>sweet</code> |189 | <code>lux soap creamy perfection 165 gm</code> | <code>soap</code> | <code>hand soap</code> |190 | <code>grey deo original</code> | <code>classic deodrant</code> | <code>women's deodorant</code> |191* Loss: [<code>MultipleNegativesSymmetricRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativessymmetricrankingloss) with these parameters:192 ```json193 {194 "scale": 20.0,195 "similarity_fct": "cos_sim",196 "gather_across_devices": false197 }198 ```199 200### Evaluation Dataset201 202#### Unnamed Dataset203 204* Size: 9,509 evaluation samples205* Columns: <code>anchor</code>, <code>positive</code>, <code>negative</code>, and <code>itemCategory</code>206* Approximate statistics based on the first 1000 samples:207 | | anchor | positive | negative | itemCategory |208 |:--------|:---------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|209 | type | string | string | string | string |210 | details | <ul><li>min: 3 tokens</li><li>mean: 9.63 tokens</li><li>max: 43 tokens</li></ul> | <ul><li>min: 2 tokens</li><li>mean: 6.53 tokens</li><li>max: 150 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 9.52 tokens</li><li>max: 50 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 3.88 tokens</li><li>max: 10 tokens</li></ul> |211* Samples:212 | anchor | positive | negative | itemCategory |213 |:---------------------------------------------------------------------|:----------------------------------|:-----------------------------------------------------------------------------------------------|:------------------------------------|214 | <code>pilot mechanical pencil progrex h-127 - 0.7 mm</code> | <code>office supplies</code> | <code>scary halloween skull mask</code> | <code>pencil</code> |215 | <code>superior drawing marker -pen - set of 12 colors - 2 nib</code> | <code>superior </code> | <code>coloring and writing book 21 x 29.7 cm 100 gsm 18 pages number subtraction ma4014</code> | <code>marker</code> |216 | <code>first person singular author: haruki murakami</code> | <code>haruki murakami book</code> | <code>buried secrets</code> | <code>literature and fiction</code> |217* Loss: [<code>MultipleNegativesSymmetricRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativessymmetricrankingloss) with these parameters:218 ```json219 {220 "scale": 20.0,221 "similarity_fct": "cos_sim",222 "gather_across_devices": false223 }224 ```225 226### Training Hyperparameters227#### Non-Default Hyperparameters228 229- `eval_strategy`: steps230- `per_device_train_batch_size`: 128231- `per_device_eval_batch_size`: 128232- `learning_rate`: 2e-05233- `weight_decay`: 0.001234- `num_train_epochs`: 5235- `warmup_ratio`: 0.1236- `fp16`: True237- `dataloader_num_workers`: 1238- `dataloader_prefetch_factor`: 2239- `dataloader_persistent_workers`: True240- `push_to_hub`: True241- `hub_model_id`: LamaDiab/MiniLM-V18Data-128ConstantBATCH-SemanticEngine242- `hub_strategy`: all_checkpoints243 244#### All Hyperparameters245<details><summary>Click to expand</summary>246 247- `overwrite_output_dir`: False248- `do_predict`: False249- `eval_strategy`: steps250- `prediction_loss_only`: True251- `per_device_train_batch_size`: 128252- `per_device_eval_batch_size`: 128253- `per_gpu_train_batch_size`: None254- `per_gpu_eval_batch_size`: None255- `gradient_accumulation_steps`: 1256- `eval_accumulation_steps`: None257- `torch_empty_cache_steps`: None258- `learning_rate`: 2e-05259- `weight_decay`: 0.001260- `adam_beta1`: 0.9261- `adam_beta2`: 0.999262- `adam_epsilon`: 1e-08263- `max_grad_norm`: 1.0264- `num_train_epochs`: 5265- `max_steps`: -1266- `lr_scheduler_type`: linear267- `lr_scheduler_kwargs`: {}268- `warmup_ratio`: 0.1269- `warmup_steps`: 0270- `log_level`: passive271- `log_level_replica`: warning272- `log_on_each_node`: True273- `logging_nan_inf_filter`: True274- `save_safetensors`: True275- `save_on_each_node`: False276- `save_only_model`: False277- `restore_callback_states_from_checkpoint`: False278- `no_cuda`: False279- `use_cpu`: False280- `use_mps_device`: False281- `seed`: 42282- `data_seed`: None283- `jit_mode_eval`: False284- `use_ipex`: False285- `bf16`: False286- `fp16`: True287- `fp16_opt_level`: O1288- `half_precision_backend`: auto289- `bf16_full_eval`: False290- `fp16_full_eval`: False291- `tf32`: None292- `local_rank`: 0293- `ddp_backend`: None294- `tpu_num_cores`: None295- `tpu_metrics_debug`: False296- `debug`: []297- `dataloader_drop_last`: False298- `dataloader_num_workers`: 1299- `dataloader_prefetch_factor`: 2300- `past_index`: -1301- `disable_tqdm`: False302- `remove_unused_columns`: True303- `label_names`: None304- `load_best_model_at_end`: False305- `ignore_data_skip`: False306- `fsdp`: []307- `fsdp_min_num_params`: 0308- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}309- `fsdp_transformer_layer_cls_to_wrap`: None310- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}311- `deepspeed`: None312- `label_smoothing_factor`: 0.0313- `optim`: adamw_torch314- `optim_args`: None315- `adafactor`: False316- `group_by_length`: False317- `length_column_name`: length318- `ddp_find_unused_parameters`: None319- `ddp_bucket_cap_mb`: None320- `ddp_broadcast_buffers`: False321- `dataloader_pin_memory`: True322- `dataloader_persistent_workers`: True323- `skip_memory_metrics`: True324- `use_legacy_prediction_loop`: False325- `push_to_hub`: True326- `resume_from_checkpoint`: None327- `hub_model_id`: LamaDiab/MiniLM-V18Data-128ConstantBATCH-SemanticEngine328- `hub_strategy`: all_checkpoints329- `hub_private_repo`: None330- `hub_always_push`: False331- `hub_revision`: None332- `gradient_checkpointing`: False333- `gradient_checkpointing_kwargs`: None334- `include_inputs_for_metrics`: False335- `include_for_metrics`: []336- `eval_do_concat_batches`: True337- `fp16_backend`: auto338- `push_to_hub_model_id`: None339- `push_to_hub_organization`: None340- `mp_parameters`: 341- `auto_find_batch_size`: False342- `full_determinism`: False343- `torchdynamo`: None344- `ray_scope`: last345- `ddp_timeout`: 1800346- `torch_compile`: False347- `torch_compile_backend`: None348- `torch_compile_mode`: None349- `include_tokens_per_second`: False350- `include_num_input_tokens_seen`: False351- `neftune_noise_alpha`: None352- `optim_target_modules`: None353- `batch_eval_metrics`: False354- `eval_on_start`: False355- `use_liger_kernel`: False356- `liger_kernel_config`: None357- `eval_use_gather_object`: False358- `average_tokens_across_devices`: False359- `prompts`: None360- `batch_sampler`: batch_sampler361- `multi_dataset_batch_sampler`: proportional362- `router_mapping`: {}363- `learning_rate_mapping`: {}364 365</details>366 367### Training Logs368| Epoch | Step | Training Loss | Validation Loss | cosine_accuracy |369|:------:|:-----:|:-------------:|:---------------:|:---------------:|370| 0.0002 | 1 | 3.5226 | - | - |371| 0.1813 | 1000 | 2.9981 | 0.5479 | 0.9450 |372| 0.3626 | 2000 | 2.3032 | 0.4921 | 0.9554 |373| 0.5440 | 3000 | 1.8788 | 0.4567 | 0.9591 |374| 0.7253 | 4000 | 1.2997 | 0.4515 | 0.9550 |375| 0.9066 | 5000 | 0.9457 | 0.4435 | 0.9531 |376| 1.0879 | 6000 | 1.2109 | 0.4124 | 0.9660 |377| 1.2693 | 7000 | 1.4479 | 0.4111 | 0.9670 |378| 1.4506 | 8000 | 1.3188 | 0.4127 | 0.9688 |379| 1.6319 | 9000 | 1.1122 | 0.4086 | 0.9656 |380| 1.8132 | 10000 | 0.7841 | 0.4071 | 0.9607 |381| 1.9946 | 11000 | 0.6116 | 0.4164 | 0.9572 |382| 2.1759 | 12000 | 1.198 | 0.3976 | 0.9699 |383| 2.3572 | 13000 | 1.1285 | 0.3976 | 0.9708 |384| 2.5385 | 14000 | 1.0768 | 0.3946 | 0.9692 |385| 2.7199 | 15000 | 0.7841 | 0.3935 | 0.9662 |386| 2.9012 | 16000 | 0.5724 | 0.4049 | 0.9604 |387| 3.0825 | 17000 | 0.7733 | 0.3817 | 0.9729 |388| 3.2638 | 18000 | 1.0369 | 0.3903 | 0.9720 |389| 3.4451 | 19000 | 0.9987 | 0.3902 | 0.9712 |390| 3.6265 | 20000 | 0.8794 | 0.3955 | 0.9678 |391| 3.8078 | 21000 | 0.6143 | 0.4025 | 0.9630 |392| 3.9891 | 22000 | 0.4693 | 0.4097 | 0.9592 |393| 4.1704 | 23000 | 0.9652 | 0.3832 | 0.9727 |394| 4.3518 | 24000 | 0.9589 | 0.3873 | 0.9723 |395| 4.5331 | 25000 | 0.9471 | 0.3861 | 0.9720 |396| 4.7144 | 26000 | 0.7042 | 0.3901 | 0.9675 |397| 4.8957 | 27000 | 0.5195 | 0.3930 | 0.9654 |398 399 400### Framework Versions401- Python: 3.11.13402- Sentence Transformers: 5.1.2403- Transformers: 4.53.3404- PyTorch: 2.6.0+cu124405- Accelerate: 1.9.0406- Datasets: 4.4.1407- Tokenizers: 0.21.2408 409## Citation410 411### BibTeX412 413#### Sentence Transformers414```bibtex415@inproceedings{reimers-2019-sentence-bert,416 title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",417 author = "Reimers, Nils and Gurevych, Iryna",418 booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",419 month = "11",420 year = "2019",421 publisher = "Association for Computational Linguistics",422 url = "https://arxiv.org/abs/1908.10084",423}424```425 426<!--427## Glossary428 429*Clearly define terms in order to be accessible across audiences.*430-->431 432<!--433## Model Card Authors434 435*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*436-->437 438<!--439## Model Card Contact440 441*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*442-->