LamaDiab/MiniLM-V9Data-256BATCH-SemanticEngine
072
1---2tags:3- sentence-transformers4- sentence-similarity5- feature-extraction6- dense7- generated_from_trainer8- dataset_size:5540309- loss:MultipleNegativesSymmetricRankingLoss10base_model: sentence-transformers/all-MiniLM-L6-v211widget:12- source_sentence: pacman smoked turkey13 sentences:14 - omelette with fresh basil & cherry tomatoes15 - mozzarella pacman16 - ' tote '17- source_sentence: mfk 140 static kite - pulpy18 sentences:19 - kite for young children20 - 'leather wrap skirt available in two colors white and black. outside materials:21 leather.'22 - adult long-sleeved thermal football base layer top keepcomfort 100 - black23- source_sentence: large zk diffuser - pack 724 sentences:25 - ' wrap'26 - zk diffuser27 - leo28- source_sentence: emerald green double-face drape pajama (short pants)29 sentences:30 - fiber cushion31 - 'the double-faced design pajama of the fabric ensures that both sides have a glossy32 finish, providing a stunning look and feel. inside and outside material: double33 face satin'34 - sky blue seashell set35- source_sentence: to - do - dahab36 sentences:37 - notebook ruled glue binding soft cover 14.2 x 20.8 cm 160 sheets 80 gsm leather38 cover heeton no a25-83539 - ' notebook'40 - ' advance repair lotion'41pipeline_tag: sentence-similarity42library_name: sentence-transformers43metrics:44- cosine_accuracy45model-index:46- name: SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v247 results:48 - task:49 type: triplet50 name: Triplet51 dataset:52 name: Unknown53 type: unknown54 metrics:55 - type: cosine_accuracy56 value: 0.959705412387847957 name: Cosine Accuracy58---59 60# SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v261 62This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.63 64## Model Details65 66### Model Description67- **Model Type:** Sentence Transformer68- **Base model:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) <!-- at revision c9745ed1d9f207416be6d2e6f8de32d1f16199bf -->69- **Maximum Sequence Length:** 256 tokens70- **Output Dimensionality:** 384 dimensions71- **Similarity Function:** Cosine Similarity72<!-- - **Training Dataset:** Unknown -->73<!-- - **Language:** Unknown -->74<!-- - **License:** Unknown -->75 76### Model Sources77 78- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)79- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)80- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)81 82### Full Model Architecture83 84```85SentenceTransformer(86 (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})87 (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})88 (2): Normalize()89)90```91 92## Usage93 94### Direct Usage (Sentence Transformers)95 96First install the Sentence Transformers library:97 98```bash99pip install -U sentence-transformers100```101 102Then you can load this model and run inference.103```python104from sentence_transformers import SentenceTransformer105 106# Download from the 🤗 Hub107model = SentenceTransformer("LamaDiab/MiniLM-V9Data-256BATCH-SemanticEngine")108# Run inference109sentences = [110 'to - do - dahab',111 ' notebook',112 'notebook ruled glue binding soft cover 14.2 x 20.8 cm 160 sheets 80 gsm leather cover heeton no a25-835',113]114embeddings = model.encode(sentences)115print(embeddings.shape)116# [3, 384]117 118# Get the similarity scores for the embeddings119similarities = model.similarity(embeddings, embeddings)120print(similarities)121# tensor([[1.0000, 0.3143, 0.2632],122# [0.3143, 1.0000, 0.7027],123# [0.2632, 0.7027, 1.0000]])124```125 126<!--127### Direct Usage (Transformers)128 129<details><summary>Click to see the direct usage in Transformers</summary>130 131</details>132-->133 134<!--135### Downstream Usage (Sentence Transformers)136 137You can finetune this model on your own dataset.138 139<details><summary>Click to expand</summary>140 141</details>142-->143 144<!--145### Out-of-Scope Use146 147*List how the model may foreseeably be misused and address what users ought not to do with the model.*148-->149 150## Evaluation151 152### Metrics153 154#### Triplet155 156* Evaluated with [<code>TripletEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator)157 158| Metric | Value |159|:--------------------|:-----------|160| **cosine_accuracy** | **0.9597** |161 162<!--163## Bias, Risks and Limitations164 165*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*166-->167 168<!--169### Recommendations170 171*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*172-->173 174## Training Details175 176### Training Dataset177 178#### Unnamed Dataset179 180* Size: 554,030 training samples181* Columns: <code>anchor</code> and <code>positive</code>182* Approximate statistics based on the first 1000 samples:183 | | anchor | positive |184 |:--------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|185 | type | string | string |186 | details | <ul><li>min: 3 tokens</li><li>mean: 7.19 tokens</li><li>max: 44 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 8.03 tokens</li><li>max: 58 tokens</li></ul> |187* Samples:188 | anchor | positive |189 |:---------------------------------------|:------------------------------------|190 | <code>grass fed butter basbousa</code> | <code>coconut flour basbousa</code> |191 | <code>silver printer tape</code> | <code>printer labels</code> |192 | <code>top</code> | <code>charcoal tee</code> |193* Loss: [<code>MultipleNegativesSymmetricRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativessymmetricrankingloss) with these parameters:194 ```json195 {196 "scale": 20.0,197 "similarity_fct": "cos_sim",198 "gather_across_devices": false199 }200 ```201 202### Evaluation Dataset203 204#### Unnamed Dataset205 206* Size: 9,505 evaluation samples207* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>208* Approximate statistics based on the first 1000 samples:209 | | anchor | positive | negative |210 |:--------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|211 | type | string | string | string |212 | details | <ul><li>min: 3 tokens</li><li>mean: 9.63 tokens</li><li>max: 43 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 6.2 tokens</li><li>max: 150 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 9.58 tokens</li><li>max: 34 tokens</li></ul> |213* Samples:214 | anchor | positive | negative |215 |:---------------------------------------------------------------------|:-----------------------------------------|:--------------------------------------------------------------------------|216 | <code>pilot mechanical pencil progrex h-127 - 0.7 mm</code> | <code> progrex pencil </code> | <code>canvas frame 100% cotton 380 gsm 2040 cm rectangular m e5305</code> |217 | <code>superior drawing marker -pen - set of 12 colors - 2 nib</code> | <code> marker pen </code> | <code>blue to-do list</code> |218 | <code>first person singular author: haruki murakami</code> | <code> first person singular book</code> | <code>sesame street 5-minute stories</code> |219* Loss: [<code>MultipleNegativesSymmetricRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativessymmetricrankingloss) with these parameters:220 ```json221 {222 "scale": 20.0,223 "similarity_fct": "cos_sim",224 "gather_across_devices": false225 }226 ```227 228### Training Hyperparameters229#### Non-Default Hyperparameters230 231- `eval_strategy`: steps232- `per_device_train_batch_size`: 256233- `per_device_eval_batch_size`: 256234- `learning_rate`: 2e-05235- `weight_decay`: 0.001236- `num_train_epochs`: 5237- `warmup_steps`: 1082238- `fp16`: True239- `dataloader_num_workers`: 1240- `dataloader_prefetch_factor`: 2241- `dataloader_persistent_workers`: True242- `push_to_hub`: True243- `hub_model_id`: LamaDiab/MiniLM-V9Data-256BATCH-SemanticEngine244- `hub_strategy`: all_checkpoints245- `batch_sampler`: no_duplicates246 247#### All Hyperparameters248<details><summary>Click to expand</summary>249 250- `overwrite_output_dir`: False251- `do_predict`: False252- `eval_strategy`: steps253- `prediction_loss_only`: True254- `per_device_train_batch_size`: 256255- `per_device_eval_batch_size`: 256256- `per_gpu_train_batch_size`: None257- `per_gpu_eval_batch_size`: None258- `gradient_accumulation_steps`: 1259- `eval_accumulation_steps`: None260- `torch_empty_cache_steps`: None261- `learning_rate`: 2e-05262- `weight_decay`: 0.001263- `adam_beta1`: 0.9264- `adam_beta2`: 0.999265- `adam_epsilon`: 1e-08266- `max_grad_norm`: 1.0267- `num_train_epochs`: 5268- `max_steps`: -1269- `lr_scheduler_type`: linear270- `lr_scheduler_kwargs`: {}271- `warmup_ratio`: 0272- `warmup_steps`: 1082273- `log_level`: passive274- `log_level_replica`: warning275- `log_on_each_node`: True276- `logging_nan_inf_filter`: True277- `save_safetensors`: True278- `save_on_each_node`: False279- `save_only_model`: False280- `restore_callback_states_from_checkpoint`: False281- `no_cuda`: False282- `use_cpu`: False283- `use_mps_device`: False284- `seed`: 42285- `data_seed`: None286- `jit_mode_eval`: False287- `use_ipex`: False288- `bf16`: False289- `fp16`: True290- `fp16_opt_level`: O1291- `half_precision_backend`: auto292- `bf16_full_eval`: False293- `fp16_full_eval`: False294- `tf32`: None295- `local_rank`: 0296- `ddp_backend`: None297- `tpu_num_cores`: None298- `tpu_metrics_debug`: False299- `debug`: []300- `dataloader_drop_last`: False301- `dataloader_num_workers`: 1302- `dataloader_prefetch_factor`: 2303- `past_index`: -1304- `disable_tqdm`: False305- `remove_unused_columns`: True306- `label_names`: None307- `load_best_model_at_end`: False308- `ignore_data_skip`: False309- `fsdp`: []310- `fsdp_min_num_params`: 0311- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}312- `fsdp_transformer_layer_cls_to_wrap`: None313- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}314- `deepspeed`: None315- `label_smoothing_factor`: 0.0316- `optim`: adamw_torch317- `optim_args`: None318- `adafactor`: False319- `group_by_length`: False320- `length_column_name`: length321- `ddp_find_unused_parameters`: None322- `ddp_bucket_cap_mb`: None323- `ddp_broadcast_buffers`: False324- `dataloader_pin_memory`: True325- `dataloader_persistent_workers`: True326- `skip_memory_metrics`: True327- `use_legacy_prediction_loop`: False328- `push_to_hub`: True329- `resume_from_checkpoint`: None330- `hub_model_id`: LamaDiab/MiniLM-V9Data-256BATCH-SemanticEngine331- `hub_strategy`: all_checkpoints332- `hub_private_repo`: None333- `hub_always_push`: False334- `hub_revision`: None335- `gradient_checkpointing`: False336- `gradient_checkpointing_kwargs`: None337- `include_inputs_for_metrics`: False338- `include_for_metrics`: []339- `eval_do_concat_batches`: True340- `fp16_backend`: auto341- `push_to_hub_model_id`: None342- `push_to_hub_organization`: None343- `mp_parameters`: 344- `auto_find_batch_size`: False345- `full_determinism`: False346- `torchdynamo`: None347- `ray_scope`: last348- `ddp_timeout`: 1800349- `torch_compile`: False350- `torch_compile_backend`: None351- `torch_compile_mode`: None352- `include_tokens_per_second`: False353- `include_num_input_tokens_seen`: False354- `neftune_noise_alpha`: None355- `optim_target_modules`: None356- `batch_eval_metrics`: False357- `eval_on_start`: False358- `use_liger_kernel`: False359- `liger_kernel_config`: None360- `eval_use_gather_object`: False361- `average_tokens_across_devices`: False362- `prompts`: None363- `batch_sampler`: no_duplicates364- `multi_dataset_batch_sampler`: proportional365- `router_mapping`: {}366- `learning_rate_mapping`: {}367 368</details>369 370### Training Logs371| Epoch | Step | Training Loss | Validation Loss | cosine_accuracy |372|:------:|:----:|:-------------:|:---------------:|:---------------:|373| 0.0005 | 1 | 3.6183 | - | - |374| 0.4619 | 1000 | - | 1.2476 | 0.9518 |375| 0.9238 | 2000 | - | 1.1979 | 0.9533 |376| 1.0 | 2165 | 2.4107 | - | - |377| 1.3857 | 3000 | - | 1.1493 | 0.9572 |378| 1.8476 | 4000 | - | 1.1210 | 0.9597 |379| 2.0 | 4330 | 1.8868 | - | - |380 381 382### Framework Versions383- Python: 3.11.13384- Sentence Transformers: 5.1.2385- Transformers: 4.53.3386- PyTorch: 2.6.0+cu124387- Accelerate: 1.9.0388- Datasets: 4.4.1389- Tokenizers: 0.21.2390 391## Citation392 393### BibTeX394 395#### Sentence Transformers396```bibtex397@inproceedings{reimers-2019-sentence-bert,398 title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",399 author = "Reimers, Nils and Gurevych, Iryna",400 booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",401 month = "11",402 year = "2019",403 publisher = "Association for Computational Linguistics",404 url = "https://arxiv.org/abs/1908.10084",405}406```407 408<!--409## Glossary410 411*Clearly define terms in order to be accessible across audiences.*412-->413 414<!--415## Model Card Authors416 417*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*418-->419 420<!--421## Model Card Contact422 423*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*424-->