cassador/2bs32lr2
095
1---2base_model: indobenchmark/indobert-base-p23datasets:4- afaji/indonli5language:6- id7library_name: sentence-transformers8metrics:9- pearson_cosine10- spearman_cosine11- pearson_manhattan12- spearman_manhattan13- pearson_euclidean14- spearman_euclidean15- pearson_dot16- spearman_dot17- pearson_max18- spearman_max19pipeline_tag: sentence-similarity20tags:21- sentence-transformers22- sentence-similarity23- feature-extraction24- generated_from_trainer25- dataset_size:691526- loss:SoftmaxLoss27widget:28- source_sentence: Pesta Olahraga Asia Tenggara atau Southeast Asian Games, biasa29 disingkat SEA Games, adalah ajang olahraga yang diadakan setiap dua tahun dan30 melibatkan 11 negara Asia Tenggara.31 sentences:32 - Sekarang tahun 2017.33 - Warna kulit tidak mempengaruhi waktu berjemur yang baik untuk mengatifkan pro-vitamin34 D3.35 - Pesta Olahraga Asia Tenggara diadakan setiap tahun.36- source_sentence: Menjalani aktivitas Ramadhan di tengah wabah Corona tentunya tidak37 mudah.38 sentences:39 - Tidak ada observasi yang pernah dilansir oleh Business Insider.40 - Wabah Corona membuat aktivitas Ramadhan tidak mudah dijalani.41 - Piala Sudirman pertama digelar pada tahun 1989.42- source_sentence: Dalam bidang politik, partai ini memperjuangkan agar kekuasaan43 sepenuhnya berada di tangan rakyat.44 sentences:45 - Galileo tidak berhasil mengetes hasil dari Hukum Inert.46 - Kudeta 14 Februari 1946 gagal merebut kekuasaan Belanda.47 - Partai ini berusaha agar kekuasaan sepenuhnya berada di tangan rakyat.48- source_sentence: Keluarga mendiang Prince menuduh layanan musik streaming Tidal49 memasukkan karya milik sang penyanyi legendaris tanpa izin .50 sentences:51 - Rosier adalah pelayan setia Lord Voldemort.52 - Bangunan ini digunakan untuk penjualan.53 - Keluarga mendiang Prince sudah memberi izin kepada TImbal untuk menggunakan lagu54 milik Prince.55- source_sentence: Tujuan dari acara dengar pendapat CRTC adalah untuk mengumpulkan56 respons dari pada pemangku kepentingan industri ini dan dari masyarakat umum.57 sentences:58 - Pembuat Rooms hanya bisa membuat meeting yang terbuka.59 - Masyarakat umum dilibatkan untuk memberikan respon dalam acara dengar pendapat60 CRTC.61 - Eminem dirasa tidak akan memulai kembali kariernya tahun ini.62model-index:63- name: SentenceTransformer based on indobenchmark/indobert-base-p264 results:65 - task:66 type: semantic-similarity67 name: Semantic Similarity68 dataset:69 name: sts dev70 type: sts-dev71 metrics:72 - type: pearson_cosine73 value: 0.582989883623505574 name: Pearson Cosine75 - type: spearman_cosine76 value: 0.560488088021162777 name: Spearman Cosine78 - type: pearson_manhattan79 value: 0.570353499281212680 name: Pearson Manhattan81 - type: spearman_manhattan82 value: 0.549998936416694783 name: Spearman Manhattan84 - type: pearson_euclidean85 value: 0.575332363098834186 name: Pearson Euclidean87 - type: spearman_euclidean88 value: 0.55244296975475589 name: Spearman Euclidean90 - type: pearson_dot91 value: 0.562011347371809592 name: Pearson Dot93 - type: spearman_dot94 value: 0.562432432530972695 name: Spearman Dot96 - type: pearson_max97 value: 0.582989883623505598 name: Pearson Max99 - type: spearman_max100 value: 0.5624324325309726101 name: Spearman Max102 - task:103 type: semantic-similarity104 name: Semantic Similarity105 dataset:106 name: sts test107 type: sts-test108 metrics:109 - type: pearson_cosine110 value: 0.27661444766220145111 name: Pearson Cosine112 - type: spearman_cosine113 value: 0.25397061268923804114 name: Spearman Cosine115 - type: pearson_manhattan116 value: 0.22893950626786405117 name: Pearson Manhattan118 - type: spearman_manhattan119 value: 0.2295445814901059120 name: Spearman Manhattan121 - type: pearson_euclidean122 value: 0.23773763148887356123 name: Pearson Euclidean124 - type: spearman_euclidean125 value: 0.23225044424139019126 name: Spearman Euclidean127 - type: pearson_dot128 value: 0.2930559400528471129 name: Pearson Dot130 - type: spearman_dot131 value: 0.28163535345836893132 name: Spearman Dot133 - type: pearson_max134 value: 0.2930559400528471135 name: Pearson Max136 - type: spearman_max137 value: 0.28163535345836893138 name: Spearman Max139---140 141# SentenceTransformer based on indobenchmark/indobert-base-p2142 143This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [indobenchmark/indobert-base-p2](https://huggingface.co/indobenchmark/indobert-base-p2) on the [afaji/indonli](https://huggingface.co/datasets/afaji/indonli) dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.144 145## Model Details146 147### Model Description148- **Model Type:** Sentence Transformer149- **Base model:** [indobenchmark/indobert-base-p2](https://huggingface.co/indobenchmark/indobert-base-p2) <!-- at revision 94b4e0a82081fa57f227fcc2024d1ea89b57ac1f -->150- **Maximum Sequence Length:** 512 tokens151- **Output Dimensionality:** 768 tokens152- **Similarity Function:** Cosine Similarity153- **Training Dataset:**154 - [afaji/indonli](https://huggingface.co/datasets/afaji/indonli)155- **Language:** id156<!-- - **License:** Unknown -->157 158### Model Sources159 160- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)161- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)162- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)163 164### Full Model Architecture165 166```167SentenceTransformer(168 (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 169 (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})170)171```172 173## Usage174 175### Direct Usage (Sentence Transformers)176 177First install the Sentence Transformers library:178 179```bash180pip install -U sentence-transformers181```182 183Then you can load this model and run inference.184```python185from sentence_transformers import SentenceTransformer186 187# Download from the 🤗 Hub188model = SentenceTransformer("cassador/2bs32lr2")189# Run inference190sentences = [191 'Tujuan dari acara dengar pendapat CRTC adalah untuk mengumpulkan respons dari pada pemangku kepentingan industri ini dan dari masyarakat umum.',192 'Masyarakat umum dilibatkan untuk memberikan respon dalam acara dengar pendapat CRTC.',193 'Pembuat Rooms hanya bisa membuat meeting yang terbuka.',194]195embeddings = model.encode(sentences)196print(embeddings.shape)197# [3, 768]198 199# Get the similarity scores for the embeddings200similarities = model.similarity(embeddings, embeddings)201print(similarities.shape)202# [3, 3]203```204 205<!--206### Direct Usage (Transformers)207 208<details><summary>Click to see the direct usage in Transformers</summary>209 210</details>211-->212 213<!--214### Downstream Usage (Sentence Transformers)215 216You can finetune this model on your own dataset.217 218<details><summary>Click to expand</summary>219 220</details>221-->222 223<!--224### Out-of-Scope Use225 226*List how the model may foreseeably be misused and address what users ought not to do with the model.*227-->228 229## Evaluation230 231### Metrics232 233#### Semantic Similarity234* Dataset: `sts-dev`235* Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)236 237| Metric | Value |238|:--------------------|:-----------|239| pearson_cosine | 0.583 |240| **spearman_cosine** | **0.5605** |241| pearson_manhattan | 0.5704 |242| spearman_manhattan | 0.55 |243| pearson_euclidean | 0.5753 |244| spearman_euclidean | 0.5524 |245| pearson_dot | 0.562 |246| spearman_dot | 0.5624 |247| pearson_max | 0.583 |248| spearman_max | 0.5624 |249 250#### Semantic Similarity251* Dataset: `sts-test`252* Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)253 254| Metric | Value |255|:--------------------|:----------|256| pearson_cosine | 0.2766 |257| **spearman_cosine** | **0.254** |258| pearson_manhattan | 0.2289 |259| spearman_manhattan | 0.2295 |260| pearson_euclidean | 0.2377 |261| spearman_euclidean | 0.2323 |262| pearson_dot | 0.2931 |263| spearman_dot | 0.2816 |264| pearson_max | 0.2931 |265| spearman_max | 0.2816 |266 267<!--268## Bias, Risks and Limitations269 270*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*271-->272 273<!--274### Recommendations275 276*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*277-->278 279## Training Details280 281### Training Dataset282 283#### afaji/indonli284 285* Dataset: [afaji/indonli](https://huggingface.co/datasets/afaji/indonli)286* Size: 6,915 training samples287* Columns: <code>premise</code>, <code>hypothesis</code>, and <code>label</code>288* Approximate statistics based on the first 1000 samples:289 | | premise | hypothesis | label |290 |:--------|:------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:------------------------------------------------|291 | type | string | string | int |292 | details | <ul><li>min: 12 tokens</li><li>mean: 29.26 tokens</li><li>max: 135 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 12.13 tokens</li><li>max: 36 tokens</li></ul> | <ul><li>0: ~51.00%</li><li>1: ~49.00%</li></ul> |293* Samples:294 | premise | hypothesis | label |295 |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------|:---------------|296 | <code>Presiden Joko Widodo (Jokowi) menyampaikan prediksi bahwa wabah virus Corona (COVID-19) di Indonesia akan selesai akhir tahun ini.</code> | <code>Prediksi akhir wabah tidak disampaikan Jokowi.</code> | <code>0</code> |297 | <code>Meski biasanya hanya digunakan di fasilitas kesehatan, saat ini masker dan sarung tangan sekali pakai banyak dipakai di tingkat rumah tangga.</code> | <code>Masker sekali pakai banyak dipakai di tingkat rumah tangga.</code> | <code>1</code> |298 | <code>Seperti namanya, paket internet sahur Telkomsel ini ditujukan bagi pengguna yang menginginkan kuota ekstra, untuk menemani momen sahur sepanjang bulan puasa.</code> | <code>Paket internet sahur tidak ditujukan untuk saat sahur.</code> | <code>0</code> |299* Loss: [<code>SoftmaxLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#softmaxloss)300 301### Evaluation Dataset302 303#### afaji/indonli304 305* Dataset: [afaji/indonli](https://huggingface.co/datasets/afaji/indonli)306* Size: 1,556 evaluation samples307* Columns: <code>premise</code>, <code>hypothesis</code>, and <code>label</code>308* Approximate statistics based on the first 1000 samples:309 | | premise | hypothesis | label |310 |:--------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:------------------------------------------------|311 | type | string | string | int |312 | details | <ul><li>min: 9 tokens</li><li>mean: 28.07 tokens</li><li>max: 179 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 12.15 tokens</li><li>max: 25 tokens</li></ul> | <ul><li>0: ~47.90%</li><li>1: ~52.10%</li></ul> |313* Samples:314 | premise | hypothesis | label |315 |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------|:---------------|316 | <code>Manuskrip tersebut berisi tiga catatan yang menceritakan bagaimana peristiwa jatuhnya meteorit serta laporan kematian akibat kejadian tersebut seperti dilansir dari Science Alert, Sabtu (25/4/2020).</code> | <code>Manuskrip tersebut tidak mencatat laporan kematian.</code> | <code>0</code> |317 | <code>Dilansir dari Business Insider, menurut observasi dari Mauna Loa Observatory di Hawaii pada karbon dioksida (CO2) di level mencapai 410 ppm tidak langsung memberikan efek pada pernapasan, karena tubuh manusia juga masih membutuhkan CO2 dalam kadar tertentu.</code> | <code>Tidak ada observasi yang pernah dilansir oleh Business Insider.</code> | <code>0</code> |318 | <code>Seorang wanita asal New York mengaku sangat benci air putih.</code> | <code>Tidak ada orang dari New York yang membenci air putih.</code> | <code>0</code> |319* Loss: [<code>SoftmaxLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#softmaxloss)320 321### Training Hyperparameters322#### Non-Default Hyperparameters323 324- `eval_strategy`: epoch325- `per_device_train_batch_size`: 32326- `per_device_eval_batch_size`: 32327- `learning_rate`: 2e-05328- `num_train_epochs`: 2329- `warmup_ratio`: 0.1330- `fp16`: True331 332#### All Hyperparameters333<details><summary>Click to expand</summary>334 335- `overwrite_output_dir`: False336- `do_predict`: False337- `eval_strategy`: epoch338- `prediction_loss_only`: True339- `per_device_train_batch_size`: 32340- `per_device_eval_batch_size`: 32341- `per_gpu_train_batch_size`: None342- `per_gpu_eval_batch_size`: None343- `gradient_accumulation_steps`: 1344- `eval_accumulation_steps`: None345- `learning_rate`: 2e-05346- `weight_decay`: 0.0347- `adam_beta1`: 0.9348- `adam_beta2`: 0.999349- `adam_epsilon`: 1e-08350- `max_grad_norm`: 1.0351- `num_train_epochs`: 2352- `max_steps`: -1353- `lr_scheduler_type`: linear354- `lr_scheduler_kwargs`: {}355- `warmup_ratio`: 0.1356- `warmup_steps`: 0357- `log_level`: passive358- `log_level_replica`: warning359- `log_on_each_node`: True360- `logging_nan_inf_filter`: True361- `save_safetensors`: True362- `save_on_each_node`: False363- `save_only_model`: False364- `restore_callback_states_from_checkpoint`: False365- `no_cuda`: False366- `use_cpu`: False367- `use_mps_device`: False368- `seed`: 42369- `data_seed`: None370- `jit_mode_eval`: False371- `use_ipex`: False372- `bf16`: False373- `fp16`: True374- `fp16_opt_level`: O1375- `half_precision_backend`: auto376- `bf16_full_eval`: False377- `fp16_full_eval`: False378- `tf32`: None379- `local_rank`: 0380- `ddp_backend`: None381- `tpu_num_cores`: None382- `tpu_metrics_debug`: False383- `debug`: []384- `dataloader_drop_last`: False385- `dataloader_num_workers`: 0386- `dataloader_prefetch_factor`: None387- `past_index`: -1388- `disable_tqdm`: False389- `remove_unused_columns`: True390- `label_names`: None391- `load_best_model_at_end`: False392- `ignore_data_skip`: False393- `fsdp`: []394- `fsdp_min_num_params`: 0395- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}396- `fsdp_transformer_layer_cls_to_wrap`: None397- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}398- `deepspeed`: None399- `label_smoothing_factor`: 0.0400- `optim`: adamw_torch401- `optim_args`: None402- `adafactor`: False403- `group_by_length`: False404- `length_column_name`: length405- `ddp_find_unused_parameters`: None406- `ddp_bucket_cap_mb`: None407- `ddp_broadcast_buffers`: False408- `dataloader_pin_memory`: True409- `dataloader_persistent_workers`: False410- `skip_memory_metrics`: True411- `use_legacy_prediction_loop`: False412- `push_to_hub`: False413- `resume_from_checkpoint`: None414- `hub_model_id`: None415- `hub_strategy`: every_save416- `hub_private_repo`: False417- `hub_always_push`: False418- `gradient_checkpointing`: False419- `gradient_checkpointing_kwargs`: None420- `include_inputs_for_metrics`: False421- `eval_do_concat_batches`: True422- `fp16_backend`: auto423- `push_to_hub_model_id`: None424- `push_to_hub_organization`: None425- `mp_parameters`: 426- `auto_find_batch_size`: False427- `full_determinism`: False428- `torchdynamo`: None429- `ray_scope`: last430- `ddp_timeout`: 1800431- `torch_compile`: False432- `torch_compile_backend`: None433- `torch_compile_mode`: None434- `dispatch_batches`: None435- `split_batches`: None436- `include_tokens_per_second`: False437- `include_num_input_tokens_seen`: False438- `neftune_noise_alpha`: None439- `optim_target_modules`: None440- `batch_eval_metrics`: False441- `batch_sampler`: batch_sampler442- `multi_dataset_batch_sampler`: proportional443 444</details>445 446### Training Logs447| Epoch | Step | Training Loss | loss | sts-dev_spearman_cosine | sts-test_spearman_cosine |448|:------:|:----:|:-------------:|:------:|:-----------------------:|:------------------------:|449| 0 | 0 | - | - | 0.1277 | - |450| 0.4608 | 100 | 0.5694 | - | - | - |451| 0.9217 | 200 | 0.4754 | - | - | - |452| 1.0 | 217 | - | 0.4349 | 0.5410 | - |453| 1.3825 | 300 | 0.3829 | - | - | - |454| 1.8433 | 400 | 0.3507 | - | - | - |455| 2.0 | 434 | - | 0.4254 | 0.5605 | 0.2540 |456 457 458### Framework Versions459- Python: 3.10.12460- Sentence Transformers: 3.0.1461- Transformers: 4.41.2462- PyTorch: 2.3.0+cu121463- Accelerate: 0.31.0464- Datasets: 2.20.0465- Tokenizers: 0.19.1466 467## Citation468 469### BibTeX470 471#### Sentence Transformers and SoftmaxLoss472```bibtex473@inproceedings{reimers-2019-sentence-bert,474 title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",475 author = "Reimers, Nils and Gurevych, Iryna",476 booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",477 month = "11",478 year = "2019",479 publisher = "Association for Computational Linguistics",480 url = "https://arxiv.org/abs/1908.10084",481}482```483 484<!--485## Glossary486 487*Clearly define terms in order to be accessible across audiences.*488-->489 490<!--491## Model Card Authors492 493*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*494-->495 496<!--497## Model Card Contact498 499*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*500-->