LamaDiab/MiniLM-V13Data-256BATCH-SemanticEngine
077
1---2tags:3- sentence-transformers4- sentence-similarity5- feature-extraction6- dense7- generated_from_trainer8- dataset_size:6816379- loss:MultipleNegativesSymmetricRankingLoss10base_model: sentence-transformers/all-MiniLM-L6-v211widget:12- source_sentence: essence multi task concealer 15 natural nude13 sentences:14 - one in shower cream sensitive 40 gr fruity15 - natural nude concealer16 - best ab wheel17- source_sentence: 'brain quest workbook author: bridget heos'18 sentences:19 - ' book'20 - double layered tortilla shawerma21 - a to z mysteries, unwilling umpire22- source_sentence: rio mare - salatuna maize with peas, carrots & olives - 160 gr23 sentences:24 - ' shorts'25 - rio mare salatuna26 - french mini raisin swirl27- source_sentence: juliette bundle28 sentences:29 - ' colored pencil'30 - juliette body lotion31 - got2b glued blasting freeze | schwarzkopf32- source_sentence: women summer pajama set cashmere buttoned shirt + pants33 sentences:34 - baguette zircon stone ring35 - side pockets pajama36 - ' bag'37pipeline_tag: sentence-similarity38library_name: sentence-transformers39metrics:40- cosine_accuracy41model-index:42- name: SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v243 results:44 - task:45 type: triplet46 name: Triplet47 dataset:48 name: Unknown49 type: unknown50 metrics:51 - type: cosine_accuracy52 value: 0.969292223453521753 name: Cosine Accuracy54---55 56# SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v257 58This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.59 60## Model Details61 62### Model Description63- **Model Type:** Sentence Transformer64- **Base model:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) <!-- at revision c9745ed1d9f207416be6d2e6f8de32d1f16199bf -->65- **Maximum Sequence Length:** 256 tokens66- **Output Dimensionality:** 384 dimensions67- **Similarity Function:** Cosine Similarity68<!-- - **Training Dataset:** Unknown -->69<!-- - **Language:** Unknown -->70<!-- - **License:** Unknown -->71 72### Model Sources73 74- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)75- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)76- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)77 78### Full Model Architecture79 80```81SentenceTransformer(82 (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})83 (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})84 (2): Normalize()85)86```87 88## Usage89 90### Direct Usage (Sentence Transformers)91 92First install the Sentence Transformers library:93 94```bash95pip install -U sentence-transformers96```97 98Then you can load this model and run inference.99```python100from sentence_transformers import SentenceTransformer101 102# Download from the 🤗 Hub103model = SentenceTransformer("LamaDiab/MiniLM-V13Data-256BATCH-SemanticEngine")104# Run inference105sentences = [106 'women summer pajama set cashmere buttoned shirt + pants',107 'side pockets pajama',108 'baguette zircon stone ring',109]110embeddings = model.encode(sentences)111print(embeddings.shape)112# [3, 384]113 114# Get the similarity scores for the embeddings115similarities = model.similarity(embeddings, embeddings)116print(similarities)117# tensor([[ 1.0000, 0.8069, 0.0497],118# [ 0.8069, 1.0000, -0.0085],119# [ 0.0497, -0.0085, 1.0000]])120```121 122<!--123### Direct Usage (Transformers)124 125<details><summary>Click to see the direct usage in Transformers</summary>126 127</details>128-->129 130<!--131### Downstream Usage (Sentence Transformers)132 133You can finetune this model on your own dataset.134 135<details><summary>Click to expand</summary>136 137</details>138-->139 140<!--141### Out-of-Scope Use142 143*List how the model may foreseeably be misused and address what users ought not to do with the model.*144-->145 146## Evaluation147 148### Metrics149 150#### Triplet151 152* Evaluated with [<code>TripletEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator)153 154| Metric | Value |155|:--------------------|:-----------|156| **cosine_accuracy** | **0.9693** |157 158<!--159## Bias, Risks and Limitations160 161*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*162-->163 164<!--165### Recommendations166 167*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*168-->169 170## Training Details171 172### Training Dataset173 174#### Unnamed Dataset175 176* Size: 681,637 training samples177* Columns: <code>anchor</code> and <code>positive</code>178* Approximate statistics based on the first 1000 samples:179 | | anchor | positive |180 |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|181 | type | string | string |182 | details | <ul><li>min: 3 tokens</li><li>mean: 6.96 tokens</li><li>max: 137 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 7.62 tokens</li><li>max: 118 tokens</li></ul> |183* Samples:184 | anchor | positive |185 |:----------------------------|:------------------------------------------------|186 | <code>men shoe spray</code> | <code>fila restorer spray 200 ml - black</code> |187 | <code>one size dress</code> | <code>fuchsia dress</code> |188 | <code>capsule almond</code> | <code>bristot nespresso caps cremoso</code> |189* Loss: [<code>MultipleNegativesSymmetricRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativessymmetricrankingloss) with these parameters:190 ```json191 {192 "scale": 20.0,193 "similarity_fct": "cos_sim",194 "gather_across_devices": false195 }196 ```197 198### Evaluation Dataset199 200#### Unnamed Dataset201 202* Size: 9,509 evaluation samples203* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>204* Approximate statistics based on the first 1000 samples:205 | | anchor | positive | negative |206 |:--------|:---------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|207 | type | string | string | string |208 | details | <ul><li>min: 3 tokens</li><li>mean: 9.63 tokens</li><li>max: 43 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 6.03 tokens</li><li>max: 150 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 9.43 tokens</li><li>max: 48 tokens</li></ul> |209* Samples:210 | anchor | positive | negative |211 |:---------------------------------------------------------------------|:-----------------------------------|:---------------------------------------------|212 | <code>pilot mechanical pencil progrex h-127 - 0.7 mm</code> | <code>office supplies</code> | <code>banana</code> |213 | <code>superior drawing marker -pen - set of 12 colors - 2 nib</code> | <code>superior </code> | <code>fc 9000 pencil without eraser h</code> |214 | <code>first person singular author: haruki murakami</code> | <code>first person singular</code> | <code>flora vase</code> |215* Loss: [<code>MultipleNegativesSymmetricRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativessymmetricrankingloss) with these parameters:216 ```json217 {218 "scale": 20.0,219 "similarity_fct": "cos_sim",220 "gather_across_devices": false221 }222 ```223 224### Training Hyperparameters225#### Non-Default Hyperparameters226 227- `eval_strategy`: steps228- `per_device_train_batch_size`: 256229- `per_device_eval_batch_size`: 256230- `learning_rate`: 2e-05231- `weight_decay`: 0.01232- `num_train_epochs`: 5233- `warmup_ratio`: 0.2234- `fp16`: True235- `dataloader_num_workers`: 1236- `dataloader_prefetch_factor`: 2237- `dataloader_persistent_workers`: True238- `push_to_hub`: True239- `hub_model_id`: LamaDiab/MiniLM-V13Data-256BATCH-SemanticEngine240- `hub_strategy`: all_checkpoints241- `batch_sampler`: no_duplicates242 243#### All Hyperparameters244<details><summary>Click to expand</summary>245 246- `overwrite_output_dir`: False247- `do_predict`: False248- `eval_strategy`: steps249- `prediction_loss_only`: True250- `per_device_train_batch_size`: 256251- `per_device_eval_batch_size`: 256252- `per_gpu_train_batch_size`: None253- `per_gpu_eval_batch_size`: None254- `gradient_accumulation_steps`: 1255- `eval_accumulation_steps`: None256- `torch_empty_cache_steps`: None257- `learning_rate`: 2e-05258- `weight_decay`: 0.01259- `adam_beta1`: 0.9260- `adam_beta2`: 0.999261- `adam_epsilon`: 1e-08262- `max_grad_norm`: 1.0263- `num_train_epochs`: 5264- `max_steps`: -1265- `lr_scheduler_type`: linear266- `lr_scheduler_kwargs`: {}267- `warmup_ratio`: 0.2268- `warmup_steps`: 0269- `log_level`: passive270- `log_level_replica`: warning271- `log_on_each_node`: True272- `logging_nan_inf_filter`: True273- `save_safetensors`: True274- `save_on_each_node`: False275- `save_only_model`: False276- `restore_callback_states_from_checkpoint`: False277- `no_cuda`: False278- `use_cpu`: False279- `use_mps_device`: False280- `seed`: 42281- `data_seed`: None282- `jit_mode_eval`: False283- `use_ipex`: False284- `bf16`: False285- `fp16`: True286- `fp16_opt_level`: O1287- `half_precision_backend`: auto288- `bf16_full_eval`: False289- `fp16_full_eval`: False290- `tf32`: None291- `local_rank`: 0292- `ddp_backend`: None293- `tpu_num_cores`: None294- `tpu_metrics_debug`: False295- `debug`: []296- `dataloader_drop_last`: False297- `dataloader_num_workers`: 1298- `dataloader_prefetch_factor`: 2299- `past_index`: -1300- `disable_tqdm`: False301- `remove_unused_columns`: True302- `label_names`: None303- `load_best_model_at_end`: False304- `ignore_data_skip`: False305- `fsdp`: []306- `fsdp_min_num_params`: 0307- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}308- `fsdp_transformer_layer_cls_to_wrap`: None309- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}310- `deepspeed`: None311- `label_smoothing_factor`: 0.0312- `optim`: adamw_torch313- `optim_args`: None314- `adafactor`: False315- `group_by_length`: False316- `length_column_name`: length317- `ddp_find_unused_parameters`: None318- `ddp_bucket_cap_mb`: None319- `ddp_broadcast_buffers`: False320- `dataloader_pin_memory`: True321- `dataloader_persistent_workers`: True322- `skip_memory_metrics`: True323- `use_legacy_prediction_loop`: False324- `push_to_hub`: True325- `resume_from_checkpoint`: None326- `hub_model_id`: LamaDiab/MiniLM-V13Data-256BATCH-SemanticEngine327- `hub_strategy`: all_checkpoints328- `hub_private_repo`: None329- `hub_always_push`: False330- `hub_revision`: None331- `gradient_checkpointing`: False332- `gradient_checkpointing_kwargs`: None333- `include_inputs_for_metrics`: False334- `include_for_metrics`: []335- `eval_do_concat_batches`: True336- `fp16_backend`: auto337- `push_to_hub_model_id`: None338- `push_to_hub_organization`: None339- `mp_parameters`: 340- `auto_find_batch_size`: False341- `full_determinism`: False342- `torchdynamo`: None343- `ray_scope`: last344- `ddp_timeout`: 1800345- `torch_compile`: False346- `torch_compile_backend`: None347- `torch_compile_mode`: None348- `include_tokens_per_second`: False349- `include_num_input_tokens_seen`: False350- `neftune_noise_alpha`: None351- `optim_target_modules`: None352- `batch_eval_metrics`: False353- `eval_on_start`: False354- `use_liger_kernel`: False355- `liger_kernel_config`: None356- `eval_use_gather_object`: False357- `average_tokens_across_devices`: False358- `prompts`: None359- `batch_sampler`: no_duplicates360- `multi_dataset_batch_sampler`: proportional361- `router_mapping`: {}362- `learning_rate_mapping`: {}363 364</details>365 366### Training Logs367| Epoch | Step | Training Loss | Validation Loss | cosine_accuracy |368|:------:|:-----:|:-------------:|:---------------:|:---------------:|369| 0.0004 | 1 | 3.7128 | - | - |370| 0.3755 | 1000 | 2.8207 | 1.3596 | 0.9524 |371| 0.7510 | 2000 | 2.2515 | 1.3199 | 0.9578 |372| 1.1265 | 3000 | 1.8906 | 1.3084 | 0.9598 |373| 1.5021 | 4000 | 1.7993 | 1.2729 | 0.9631 |374| 1.8776 | 5000 | 1.6857 | 1.2809 | 0.9652 |375| 2.2531 | 6000 | 1.5251 | 1.2627 | 0.9658 |376| 2.6286 | 7000 | 1.5447 | 1.2668 | 0.9659 |377| 3.0041 | 8000 | 1.4464 | 1.2620 | 0.9660 |378| 3.3796 | 9000 | 1.4583 | 1.2405 | 0.9687 |379| 3.7552 | 10000 | 1.4374 | 1.2410 | 0.9691 |380| 4.1307 | 11000 | 1.3556 | 1.2511 | 0.9687 |381| 4.5062 | 12000 | 1.3997 | 1.2364 | 0.9690 |382| 4.8817 | 13000 | 1.3955 | 1.2428 | 0.9693 |383 384 385### Framework Versions386- Python: 3.11.13387- Sentence Transformers: 5.1.2388- Transformers: 4.53.3389- PyTorch: 2.6.0+cu124390- Accelerate: 1.9.0391- Datasets: 4.4.1392- Tokenizers: 0.21.2393 394## Citation395 396### BibTeX397 398#### Sentence Transformers399```bibtex400@inproceedings{reimers-2019-sentence-bert,401 title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",402 author = "Reimers, Nils and Gurevych, Iryna",403 booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",404 month = "11",405 year = "2019",406 publisher = "Association for Computational Linguistics",407 url = "https://arxiv.org/abs/1908.10084",408}409```410 411<!--412## Glossary413 414*Clearly define terms in order to be accessible across audiences.*415-->416 417<!--418## Model Card Authors419 420*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*421-->422 423<!--424## Model Card Contact425 426*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*427-->