LamaDiab/Finetunningv2MiniLM-V22Data-128ConstantBATCH-SemanticEngine
013
1---2tags:3- sentence-transformers4- sentence-similarity5- feature-extraction6- dense7- generated_from_trainer8- dataset_size:6472369- loss:MultipleNegativesSymmetricRankingLoss10base_model: LamaDiab/v2MiniLM-V22Data-128ConstantBATCH-SemanticEngine11widget:12- source_sentence: essence multi task concealer 15 natural nude13 sentences:14 - pure oxygen 20 vol15 - essence16 - face make-up17- source_sentence: faber castell jumbo colored pencil, metallic copper18 sentences:19 - ' faber castell colored pencil'20 - pencil21 - a4 photographic paper, 5 colors, 100 sheets, 80 gsm22- source_sentence: gedo & the champ23 sentences:24 - children book25 - ' book'26 - diary of a wimpy kid do-it-youself book27- source_sentence: green track suit28 sentences:29 - outfit30 - green track suit31 - tres32- source_sentence: must kindergarten backpack mermazing 2 cases33 sentences:34 - crescent stand with 3 dates plate gold35 - school supplies36 - bag37pipeline_tag: sentence-similarity38library_name: sentence-transformers39metrics:40- cosine_accuracy41model-index:42- name: SentenceTransformer based on LamaDiab/v2MiniLM-V22Data-128ConstantBATCH-SemanticEngine43 results:44 - task:45 type: triplet46 name: Triplet47 dataset:48 name: Unknown49 type: unknown50 metrics:51 - type: cosine_accuracy52 value: 0.969607710838317953 name: Cosine Accuracy54---55 56# SentenceTransformer based on LamaDiab/v2MiniLM-V22Data-128ConstantBATCH-SemanticEngine57 58This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [LamaDiab/v2MiniLM-V22Data-128ConstantBATCH-SemanticEngine](https://huggingface.co/LamaDiab/v2MiniLM-V22Data-128ConstantBATCH-SemanticEngine). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.59 60## Model Details61 62### Model Description63- **Model Type:** Sentence Transformer64- **Base model:** [LamaDiab/v2MiniLM-V22Data-128ConstantBATCH-SemanticEngine](https://huggingface.co/LamaDiab/v2MiniLM-V22Data-128ConstantBATCH-SemanticEngine) <!-- at revision 0ccfe4263e8d73e36dfe2fa65847e71dbfda1e3b -->65- **Maximum Sequence Length:** 256 tokens66- **Output Dimensionality:** 384 dimensions67- **Similarity Function:** Cosine Similarity68<!-- - **Training Dataset:** Unknown -->69<!-- - **Language:** Unknown -->70<!-- - **License:** Unknown -->71 72### Model Sources73 74- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)75- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)76- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)77 78### Full Model Architecture79 80```81SentenceTransformer(82 (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})83 (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})84 (2): Normalize()85)86```87 88## Usage89 90### Direct Usage (Sentence Transformers)91 92First install the Sentence Transformers library:93 94```bash95pip install -U sentence-transformers96```97 98Then you can load this model and run inference.99```python100from sentence_transformers import SentenceTransformer101 102# Download from the 🤗 Hub103model = SentenceTransformer("LamaDiab/Finetunningv2MiniLM-V22Data-128ConstantBATCH-SemanticEngine")104# Run inference105sentences = [106 'must kindergarten backpack mermazing 2 cases',107 'school supplies',108 'crescent stand with 3 dates plate gold',109]110embeddings = model.encode(sentences)111print(embeddings.shape)112# [3, 384]113 114# Get the similarity scores for the embeddings115similarities = model.similarity(embeddings, embeddings)116print(similarities)117# tensor([[ 1.0000, 0.6193, -0.2278],118# [ 0.6193, 1.0000, -0.1204],119# [-0.2278, -0.1204, 1.0000]])120```121 122<!--123### Direct Usage (Transformers)124 125<details><summary>Click to see the direct usage in Transformers</summary>126 127</details>128-->129 130<!--131### Downstream Usage (Sentence Transformers)132 133You can finetune this model on your own dataset.134 135<details><summary>Click to expand</summary>136 137</details>138-->139 140<!--141### Out-of-Scope Use142 143*List how the model may foreseeably be misused and address what users ought not to do with the model.*144-->145 146## Evaluation147 148### Metrics149 150#### Triplet151 152* Evaluated with [<code>TripletEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator)153 154| Metric | Value |155|:--------------------|:-----------|156| **cosine_accuracy** | **0.9696** |157 158<!--159## Bias, Risks and Limitations160 161*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*162-->163 164<!--165### Recommendations166 167*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*168-->169 170## Training Details171 172### Training Dataset173 174#### Unnamed Dataset175 176* Size: 647,236 training samples177* Columns: <code>anchor</code>, <code>positive</code>, and <code>itemCategory</code>178* Approximate statistics based on the first 1000 samples:179 | | anchor | positive | itemCategory |180 |:--------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:--------------------------------------------------------------------------------|181 | type | string | string | string |182 | details | <ul><li>min: 3 tokens</li><li>mean: 11.56 tokens</li><li>max: 50 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 4.55 tokens</li><li>max: 12 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 3.91 tokens</li><li>max: 9 tokens</li></ul> |183* Samples:184 | anchor | positive | itemCategory |185 |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------|:-------------------------|186 | <code>petrol samsung galaxy</code> | <code>smart phone</code> | <code>smart phone</code> |187 | <code>must trolley bag must true football 4 cases</code> | <code>wheels cover backpack</code> | <code>bag</code> |188 | <code>sanpellegrino chino is a bold and refreshing italian beverage with a unique bittersweet flavor made from herbal extracts and citrus best served chilled for a distinctive taste experience</code> | <code>chino can drink</code> | <code>beverage</code> |189* Loss: [<code>MultipleNegativesSymmetricRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativessymmetricrankingloss) with these parameters:190 ```json191 {192 "scale": 20.0,193 "similarity_fct": "cos_sim",194 "gather_across_devices": false195 }196 ```197 198### Evaluation Dataset199 200#### Unnamed Dataset201 202* Size: 9,509 evaluation samples203* Columns: <code>anchor</code>, <code>positive</code>, <code>negative</code>, and <code>itemCategory</code>204* Approximate statistics based on the first 1000 samples:205 | | anchor | positive | negative | itemCategory |206 |:--------|:---------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|207 | type | string | string | string | string |208 | details | <ul><li>min: 3 tokens</li><li>mean: 9.63 tokens</li><li>max: 43 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 6.61 tokens</li><li>max: 150 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 9.58 tokens</li><li>max: 46 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 3.88 tokens</li><li>max: 10 tokens</li></ul> |209* Samples:210 | anchor | positive | negative | itemCategory |211 |:---------------------------------------------------------------------|:----------------------------------|:-------------------------------------------------------------------------------------------------------------------|:------------------------------------|212 | <code>pilot mechanical pencil progrex h-127 - 0.7 mm</code> | <code> pencil </code> | <code>artist pen brush tip 1.5m gold no.250</code> | <code>pencil</code> |213 | <code>superior drawing marker -pen - set of 12 colors - 2 nib</code> | <code>superior </code> | <code>notte 11-101 a5 stapled squared notebook, 60 sheets, cardboard cover, 60 grams, 148 x 210 mm, turkish</code> | <code>marker</code> |214 | <code>first person singular author: haruki murakami</code> | <code>haruki murakami book</code> | <code>yellow dinosaur assembling game</code> | <code>literature and fiction</code> |215* Loss: [<code>MultipleNegativesSymmetricRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativessymmetricrankingloss) with these parameters:216 ```json217 {218 "scale": 20.0,219 "similarity_fct": "cos_sim",220 "gather_across_devices": false221 }222 ```223 224### Training Hyperparameters225#### Non-Default Hyperparameters226 227- `eval_strategy`: steps228- `per_device_train_batch_size`: 128229- `per_device_eval_batch_size`: 128230- `weight_decay`: 0.001231- `warmup_ratio`: 0.1232- `fp16`: True233- `dataloader_num_workers`: 1234- `dataloader_prefetch_factor`: 2235- `dataloader_persistent_workers`: True236- `push_to_hub`: True237- `hub_model_id`: Finetunningv2MiniLM-V22Data-128ConstantBATCH-SemanticEngine238- `hub_strategy`: all_checkpoints239 240#### All Hyperparameters241<details><summary>Click to expand</summary>242 243- `overwrite_output_dir`: False244- `do_predict`: False245- `eval_strategy`: steps246- `prediction_loss_only`: True247- `per_device_train_batch_size`: 128248- `per_device_eval_batch_size`: 128249- `per_gpu_train_batch_size`: None250- `per_gpu_eval_batch_size`: None251- `gradient_accumulation_steps`: 1252- `eval_accumulation_steps`: None253- `torch_empty_cache_steps`: None254- `learning_rate`: 5e-05255- `weight_decay`: 0.001256- `adam_beta1`: 0.9257- `adam_beta2`: 0.999258- `adam_epsilon`: 1e-08259- `max_grad_norm`: 1.0260- `num_train_epochs`: 3261- `max_steps`: -1262- `lr_scheduler_type`: linear263- `lr_scheduler_kwargs`: {}264- `warmup_ratio`: 0.1265- `warmup_steps`: 0266- `log_level`: passive267- `log_level_replica`: warning268- `log_on_each_node`: True269- `logging_nan_inf_filter`: True270- `save_safetensors`: True271- `save_on_each_node`: False272- `save_only_model`: False273- `restore_callback_states_from_checkpoint`: False274- `no_cuda`: False275- `use_cpu`: False276- `use_mps_device`: False277- `seed`: 42278- `data_seed`: None279- `jit_mode_eval`: False280- `use_ipex`: False281- `bf16`: False282- `fp16`: True283- `fp16_opt_level`: O1284- `half_precision_backend`: auto285- `bf16_full_eval`: False286- `fp16_full_eval`: False287- `tf32`: None288- `local_rank`: 0289- `ddp_backend`: None290- `tpu_num_cores`: None291- `tpu_metrics_debug`: False292- `debug`: []293- `dataloader_drop_last`: False294- `dataloader_num_workers`: 1295- `dataloader_prefetch_factor`: 2296- `past_index`: -1297- `disable_tqdm`: False298- `remove_unused_columns`: True299- `label_names`: None300- `load_best_model_at_end`: False301- `ignore_data_skip`: False302- `fsdp`: []303- `fsdp_min_num_params`: 0304- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}305- `fsdp_transformer_layer_cls_to_wrap`: None306- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}307- `deepspeed`: None308- `label_smoothing_factor`: 0.0309- `optim`: adamw_torch310- `optim_args`: None311- `adafactor`: False312- `group_by_length`: False313- `length_column_name`: length314- `ddp_find_unused_parameters`: None315- `ddp_bucket_cap_mb`: None316- `ddp_broadcast_buffers`: False317- `dataloader_pin_memory`: True318- `dataloader_persistent_workers`: True319- `skip_memory_metrics`: True320- `use_legacy_prediction_loop`: False321- `push_to_hub`: True322- `resume_from_checkpoint`: None323- `hub_model_id`: Finetunningv2MiniLM-V22Data-128ConstantBATCH-SemanticEngine324- `hub_strategy`: all_checkpoints325- `hub_private_repo`: None326- `hub_always_push`: False327- `hub_revision`: None328- `gradient_checkpointing`: False329- `gradient_checkpointing_kwargs`: None330- `include_inputs_for_metrics`: False331- `include_for_metrics`: []332- `eval_do_concat_batches`: True333- `fp16_backend`: auto334- `push_to_hub_model_id`: None335- `push_to_hub_organization`: None336- `mp_parameters`: 337- `auto_find_batch_size`: False338- `full_determinism`: False339- `torchdynamo`: None340- `ray_scope`: last341- `ddp_timeout`: 1800342- `torch_compile`: False343- `torch_compile_backend`: None344- `torch_compile_mode`: None345- `include_tokens_per_second`: False346- `include_num_input_tokens_seen`: False347- `neftune_noise_alpha`: None348- `optim_target_modules`: None349- `batch_eval_metrics`: False350- `eval_on_start`: False351- `use_liger_kernel`: False352- `liger_kernel_config`: None353- `eval_use_gather_object`: False354- `average_tokens_across_devices`: False355- `prompts`: None356- `batch_sampler`: batch_sampler357- `multi_dataset_batch_sampler`: proportional358- `router_mapping`: {}359- `learning_rate_mapping`: {}360 361</details>362 363### Training Logs364| Epoch | Step | Training Loss | Validation Loss | cosine_accuracy |365|:------:|:-----:|:-------------:|:---------------:|:---------------:|366| 0.0002 | 1 | 0.8464 | - | - |367| 0.1977 | 1000 | 0.8746 | 0.4013 | 0.9700 |368| 0.3955 | 2000 | 0.8834 | 0.4074 | 0.9702 |369| 0.5932 | 3000 | 0.7973 | 0.4106 | 0.9674 |370| 0.7910 | 4000 | 0.5365 | 0.3833 | 0.9680 |371| 0.9887 | 5000 | 0.4558 | 0.3746 | 0.9673 |372| 1.1864 | 6000 | 0.6229 | 0.3872 | 0.9699 |373| 1.3841 | 7000 | 0.5929 | 0.3837 | 0.9710 |374| 1.5817 | 8000 | 0.5784 | 0.3874 | 0.9697 |375| 1.7794 | 9000 | 0.5687 | 0.3881 | 0.9694 |376| 1.9771 | 10000 | 0.5546 | 0.3854 | 0.9701 |377| 2.1747 | 11000 | 0.5081 | 0.3918 | 0.9696 |378| 2.3724 | 12000 | 0.4974 | 0.3988 | 0.9681 |379| 2.5701 | 13000 | 0.4847 | 0.3989 | 0.9687 |380| 2.7677 | 14000 | 0.4906 | 0.3968 | 0.9687 |381| 2.9654 | 15000 | 0.4831 | 0.3913 | 0.9696 |382 383 384### Framework Versions385- Python: 3.11.13386- Sentence Transformers: 5.1.2387- Transformers: 4.53.3388- PyTorch: 2.6.0+cu124389- Accelerate: 1.9.0390- Datasets: 4.4.1391- Tokenizers: 0.21.2392 393## Citation394 395### BibTeX396 397#### Sentence Transformers398```bibtex399@inproceedings{reimers-2019-sentence-bert,400 title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",401 author = "Reimers, Nils and Gurevych, Iryna",402 booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",403 month = "11",404 year = "2019",405 publisher = "Association for Computational Linguistics",406 url = "https://arxiv.org/abs/1908.10084",407}408```409 410<!--411## Glossary412 413*Clearly define terms in order to be accessible across audiences.*414-->415 416<!--417## Model Card Authors418 419*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*420-->421 422<!--423## Model Card Contact424 425*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*426-->