LamaDiab/NewMiniLM-V25Data-256BATCH-SemanticEngine
073
1---2tags:3- sentence-transformers4- sentence-similarity5- feature-extraction6- dense7- generated_from_trainer8- dataset_size:10063859- loss:MultipleNegativesSymmetricRankingLoss10base_model: sentence-transformers/all-MiniLM-L6-v211widget:12- source_sentence: essence multi task concealer 15 natural nude13 sentences:14 - tarte 4 in 1 mini mascara15 - essence16 - face make-up17- source_sentence: granville original one bite original rice crispy squares18 sentences:19 - rice crispy sweetened20 - sweet21 - roasted hazelnut syrup22- source_sentence: sand eel shad soft lure combo eelo 150 25 g ayu/blue23 sentences:24 - fishing25 - fast fishing fishing lure26 - black marble plant pot27- source_sentence: apple cinnamon greek yoghurt28 sentences:29 - dairy30 - gluten free yogurt31 - 1 kg coffee frappe base32- source_sentence: golden olive pouch33 sentences:34 - trio kaftan35 - pouch36 - bag37pipeline_tag: sentence-similarity38library_name: sentence-transformers39metrics:40- cosine_accuracy41model-index:42- name: SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v243 results:44 - task:45 type: triplet46 name: Triplet47 dataset:48 name: Unknown49 type: unknown50 metrics:51 - type: cosine_accuracy52 value: 0.970449030399322553 name: Cosine Accuracy54---55 56# SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v257 58This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.59 60## Model Details61 62### Model Description63- **Model Type:** Sentence Transformer64- **Base model:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) <!-- at revision c9745ed1d9f207416be6d2e6f8de32d1f16199bf -->65- **Maximum Sequence Length:** 256 tokens66- **Output Dimensionality:** 384 dimensions67- **Similarity Function:** Cosine Similarity68<!-- - **Training Dataset:** Unknown -->69<!-- - **Language:** Unknown -->70<!-- - **License:** Unknown -->71 72### Model Sources73 74- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)75- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)76- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)77 78### Full Model Architecture79 80```81SentenceTransformer(82 (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})83 (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})84 (2): Normalize()85)86```87 88## Usage89 90### Direct Usage (Sentence Transformers)91 92First install the Sentence Transformers library:93 94```bash95pip install -U sentence-transformers96```97 98Then you can load this model and run inference.99```python100from sentence_transformers import SentenceTransformer101 102# Download from the 🤗 Hub103model = SentenceTransformer("LamaDiab/NewMiniLM-V25Data-256BATCH-SemanticEngine")104# Run inference105sentences = [106 'golden olive pouch',107 'pouch',108 'trio kaftan',109]110embeddings = model.encode(sentences)111print(embeddings.shape)112# [3, 384]113 114# Get the similarity scores for the embeddings115similarities = model.similarity(embeddings, embeddings)116print(similarities)117# tensor([[1.0000, 0.8075, 0.1342],118# [0.8075, 1.0000, 0.1006],119# [0.1342, 0.1006, 1.0000]])120```121 122<!--123### Direct Usage (Transformers)124 125<details><summary>Click to see the direct usage in Transformers</summary>126 127</details>128-->129 130<!--131### Downstream Usage (Sentence Transformers)132 133You can finetune this model on your own dataset.134 135<details><summary>Click to expand</summary>136 137</details>138-->139 140<!--141### Out-of-Scope Use142 143*List how the model may foreseeably be misused and address what users ought not to do with the model.*144-->145 146## Evaluation147 148### Metrics149 150#### Triplet151 152* Evaluated with [<code>TripletEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator)153 154| Metric | Value |155|:--------------------|:-----------|156| **cosine_accuracy** | **0.9704** |157 158<!--159## Bias, Risks and Limitations160 161*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*162-->163 164<!--165### Recommendations166 167*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*168-->169 170## Training Details171 172### Training Dataset173 174#### Unnamed Dataset175 176* Size: 1,006,385 training samples177* Columns: <code>anchor</code>, <code>positive</code>, and <code>itemCategory</code>178* Approximate statistics based on the first 1000 samples:179 | | anchor | positive | itemCategory |180 |:--------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:--------------------------------------------------------------------------------|181 | type | string | string | string |182 | details | <ul><li>min: 3 tokens</li><li>mean: 11.94 tokens</li><li>max: 72 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 4.57 tokens</li><li>max: 83 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 3.89 tokens</li><li>max: 9 tokens</li></ul> |183* Samples:184 | anchor | positive | itemCategory |185 |:-------------------------------------------------------------------------------------------------|:---------------------------------|:------------------------|186 | <code>lice repellent serum</code> | <code>hair serum</code> | <code>hair serum</code> |187 | <code>vanilla sponge cake with fresh moisturizer and strawberry pieces.<br>serve person.</code> | <code>vanilla tres leches</code> | <code>sweet</code> |188 | <code>wyl chips - kettle cooked sea salt & balsamic vinegar potato chips - gr</code> | <code>snacks</code> | <code>snacks</code> |189* Loss: [<code>MultipleNegativesSymmetricRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativessymmetricrankingloss) with these parameters:190 ```json191 {192 "scale": 20.0,193 "similarity_fct": "cos_sim",194 "gather_across_devices": false195 }196 ```197 198### Evaluation Dataset199 200#### Unnamed Dataset201 202* Size: 9,509 evaluation samples203* Columns: <code>anchor</code>, <code>positive</code>, <code>negative</code>, and <code>itemCategory</code>204* Approximate statistics based on the first 1000 samples:205 | | anchor | positive | negative | itemCategory |206 |:--------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:--------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|207 | type | string | string | string | string |208 | details | <ul><li>min: 3 tokens</li><li>mean: 9.63 tokens</li><li>max: 43 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 6.3 tokens</li><li>max: 150 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 9.5 tokens</li><li>max: 35 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 3.87 tokens</li><li>max: 10 tokens</li></ul> |209* Samples:210 | anchor | positive | negative | itemCategory |211 |:---------------------------------------------------------------------|:-----------------------------------------|:-------------------------------------------------------------------|:------------------------------------|212 | <code>pilot mechanical pencil progrex h-127 - 0.7 mm</code> | <code> pilot pencil </code> | <code>lunch bag colors 22 × 16 × 28 cm must shark 000586181</code> | <code>pencil</code> |213 | <code>superior drawing marker -pen - set of 12 colors - 2 nib</code> | <code> marker pen </code> | <code>staedtler triplus fineliner 10 + 3 pack</code> | <code>marker</code> |214 | <code>first person singular author: haruki murakami</code> | <code> first person singular book</code> | <code>misty grater stainless steel</code> | <code>literature and fiction</code> |215* Loss: [<code>MultipleNegativesSymmetricRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativessymmetricrankingloss) with these parameters:216 ```json217 {218 "scale": 20.0,219 "similarity_fct": "cos_sim",220 "gather_across_devices": false221 }222 ```223 224### Training Hyperparameters225#### Non-Default Hyperparameters226 227- `eval_strategy`: steps228- `per_device_train_batch_size`: 256229- `per_device_eval_batch_size`: 256230- `gradient_accumulation_steps`: 2231- `learning_rate`: 3e-05232- `weight_decay`: 0.001233- `num_train_epochs`: 5234- `warmup_ratio`: 0.1235- `fp16`: True236- `dataloader_num_workers`: 1237- `dataloader_prefetch_factor`: 2238- `dataloader_persistent_workers`: True239- `push_to_hub`: True240- `hub_model_id`: LamaDiab/NewMiniLM-V25Data-256BATCH-SemanticEngine241- `hub_strategy`: all_checkpoints242 243#### All Hyperparameters244<details><summary>Click to expand</summary>245 246- `overwrite_output_dir`: False247- `do_predict`: False248- `eval_strategy`: steps249- `prediction_loss_only`: True250- `per_device_train_batch_size`: 256251- `per_device_eval_batch_size`: 256252- `per_gpu_train_batch_size`: None253- `per_gpu_eval_batch_size`: None254- `gradient_accumulation_steps`: 2255- `eval_accumulation_steps`: None256- `torch_empty_cache_steps`: None257- `learning_rate`: 3e-05258- `weight_decay`: 0.001259- `adam_beta1`: 0.9260- `adam_beta2`: 0.999261- `adam_epsilon`: 1e-08262- `max_grad_norm`: 1.0263- `num_train_epochs`: 5264- `max_steps`: -1265- `lr_scheduler_type`: linear266- `lr_scheduler_kwargs`: {}267- `warmup_ratio`: 0.1268- `warmup_steps`: 0269- `log_level`: passive270- `log_level_replica`: warning271- `log_on_each_node`: True272- `logging_nan_inf_filter`: True273- `save_safetensors`: True274- `save_on_each_node`: False275- `save_only_model`: False276- `restore_callback_states_from_checkpoint`: False277- `no_cuda`: False278- `use_cpu`: False279- `use_mps_device`: False280- `seed`: 42281- `data_seed`: None282- `jit_mode_eval`: False283- `use_ipex`: False284- `bf16`: False285- `fp16`: True286- `fp16_opt_level`: O1287- `half_precision_backend`: auto288- `bf16_full_eval`: False289- `fp16_full_eval`: False290- `tf32`: None291- `local_rank`: 0292- `ddp_backend`: None293- `tpu_num_cores`: None294- `tpu_metrics_debug`: False295- `debug`: []296- `dataloader_drop_last`: False297- `dataloader_num_workers`: 1298- `dataloader_prefetch_factor`: 2299- `past_index`: -1300- `disable_tqdm`: False301- `remove_unused_columns`: True302- `label_names`: None303- `load_best_model_at_end`: False304- `ignore_data_skip`: False305- `fsdp`: []306- `fsdp_min_num_params`: 0307- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}308- `fsdp_transformer_layer_cls_to_wrap`: None309- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}310- `deepspeed`: None311- `label_smoothing_factor`: 0.0312- `optim`: adamw_torch313- `optim_args`: None314- `adafactor`: False315- `group_by_length`: False316- `length_column_name`: length317- `ddp_find_unused_parameters`: None318- `ddp_bucket_cap_mb`: None319- `ddp_broadcast_buffers`: False320- `dataloader_pin_memory`: True321- `dataloader_persistent_workers`: True322- `skip_memory_metrics`: True323- `use_legacy_prediction_loop`: False324- `push_to_hub`: True325- `resume_from_checkpoint`: None326- `hub_model_id`: LamaDiab/NewMiniLM-V25Data-256BATCH-SemanticEngine327- `hub_strategy`: all_checkpoints328- `hub_private_repo`: None329- `hub_always_push`: False330- `hub_revision`: None331- `gradient_checkpointing`: False332- `gradient_checkpointing_kwargs`: None333- `include_inputs_for_metrics`: False334- `include_for_metrics`: []335- `eval_do_concat_batches`: True336- `fp16_backend`: auto337- `push_to_hub_model_id`: None338- `push_to_hub_organization`: None339- `mp_parameters`: 340- `auto_find_batch_size`: False341- `full_determinism`: False342- `torchdynamo`: None343- `ray_scope`: last344- `ddp_timeout`: 1800345- `torch_compile`: False346- `torch_compile_backend`: None347- `torch_compile_mode`: None348- `include_tokens_per_second`: False349- `include_num_input_tokens_seen`: False350- `neftune_noise_alpha`: None351- `optim_target_modules`: None352- `batch_eval_metrics`: False353- `eval_on_start`: False354- `use_liger_kernel`: False355- `liger_kernel_config`: None356- `eval_use_gather_object`: False357- `average_tokens_across_devices`: False358- `prompts`: None359- `batch_sampler`: batch_sampler360- `multi_dataset_batch_sampler`: proportional361- `router_mapping`: {}362- `learning_rate_mapping`: {}363 364</details>365 366### Training Logs367| Epoch | Step | Training Loss | Validation Loss | cosine_accuracy |368|:------:|:----:|:-------------:|:---------------:|:---------------:|369| 0.0005 | 1 | 3.5148 | - | - |370| 0.5086 | 1000 | 2.3332 | 0.4183 | 0.9514 |371| 1.0173 | 2000 | 1.1848 | 0.3631 | 0.9598 |372| 1.5257 | 3000 | 1.2233 | 0.3502 | 0.9645 |373| 2.0341 | 4000 | 1.0983 | 0.3489 | 0.9663 |374| 2.5425 | 5000 | 0.9919 | 0.3366 | 0.9687 |375| 3.0508 | 6000 | 0.9487 | 0.3369 | 0.9696 |376| 3.5592 | 7000 | 0.8887 | 0.3311 | 0.9696 |377| 4.0676 | 8000 | 0.8724 | 0.3361 | 0.9703 |378| 4.5760 | 9000 | 0.8366 | 0.3336 | 0.9704 |379 380 381### Framework Versions382- Python: 3.11.13383- Sentence Transformers: 5.1.2384- Transformers: 4.53.3385- PyTorch: 2.6.0+cu124386- Accelerate: 1.9.0387- Datasets: 4.4.1388- Tokenizers: 0.21.2389 390## Citation391 392### BibTeX393 394#### Sentence Transformers395```bibtex396@inproceedings{reimers-2019-sentence-bert,397 title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",398 author = "Reimers, Nils and Gurevych, Iryna",399 booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",400 month = "11",401 year = "2019",402 publisher = "Association for Computational Linguistics",403 url = "https://arxiv.org/abs/1908.10084",404}405```406 407<!--408## Glossary409 410*Clearly define terms in order to be accessible across audiences.*411-->412 413<!--414## Model Card Authors415 416*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*417-->418 419<!--420## Model Card Contact421 422*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*423-->