ethanteh/bge-base-insurance-matryoshka

BGE base Health Insurance Matryoshka

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5 on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• —Model Type: Sentence Transformer
• —Base model: BAAI/bge-base-en-v1.5 <!-- at revision a5beb1e3e68b9ab74eb54cfd186867f64f240e1a -->
• —Maximum Sequence Length: 512 tokens
• —Output Dimensionality: 768 dimensions
• —Similarity Function: Cosine Similarity
• —Training Dataset:
• —json
• —Language: en
• —License: apache-2.0

Model Sources

• —Documentation: Sentence Transformers Documentation
• —Repository: Sentence Transformers on GitHub
• —Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ethanteh/bge-base-insurance-matryoshka")
# Run inference
sentences = [
    'Be in control with Active Pricing\n\nSam, age 35 years, purchases PRUMillion Med Active medical plan. With Active Pricing, Sam pays less premiums when he claims less.\n\nSam will enjoy an instant discount of 15% on the medical insurance charges, paying a monthly premium of only RM229 throughout the policy term if there are no claims made and approved.\n\nIn the event Sam makes a claim of less than RM5,000\n\nStack-Up Level Year 8: Premium temporarily increases to RM250 Base Level Year 7: Sam makes a claim of RM 250 Year 9 & onwards: discounted RM4,800 and is approved premium continues at RM229 Discount Level RM 229 RM 229 RM 229 RM 229 RM 229 RM 229 RM 229 RM 229 ...... Year 1 2 3... 7 8 9 10 11 12...\n\nIn the event Sam makes a claim of RM5,000 or more\n\nYear 4 & 5: Premiums temporarily increase to RM287 Year 6: Premium lowers down to RM250 Stack-Up Level RM 287 RM 287 Base Level Year 3: Sam was hospitalised with a serious illness and made a claim of RM80,000 and is approved RM 250 Year 7 & onwards: discounted premium continues at RM229 Discount Level RM 229 RM 229 RM 229 RM 229 RM 229 RM 229 Year 1 2 3 4 5 6 7 8 9... ...',
    "What happens to Sam's premium if he makes a claim of less than RM5,000 with PRUMillion Med Active?",
    'What are the premiums for a RM1,000 sum assured for a 28-year-old male non-smoker?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

Evaluation

Metrics

Information Retrieval

• —Datasets: dim_768, dim_512, dim_256, dim_128 and dim_64
• —Evaluated with <code>InformationRetrievalEvaluator</code>

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

json

• —Dataset: json
• —Size: 739 training samples
• —Columns: <code>positive</code> and <code>anchor</code>
• —Approximate statistics based on the first 739 samples: | | positive | anchor | |:--------|:-------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------| | type | string | string | | details | <ul><li>min: 13 tokens</li><li>mean: 264.01 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 9 tokens</li><li>mean: 20.37 tokens</li><li>max: 60 tokens</li></ul> |
• —Samples: | positive | anchor | |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------| | <code>For your attention<br><br>1. This brochure contains only a summary of the main features of this product and is not exhaustive. It does not constitute a policy. You are advised to refer to the Sales Illustration and Product Disclosure Sheet for more details of the product before purchasing a policy and refer to the terms and conditions in the policy for details of the features and benefits, exclusions and waiting periods under the policy.<br><br>2. You should satisfy yourself that this plan will best serve your needs and that the premium payable under this policy is an amount you can afford.<br><br>If you cancel your policy within the free-look period of 15 days, we will refund to you the full premium less any expenses which may have been incurred for any medical examination (if any).<br><br>3.<br><br>4. The premium for A-Life Essential Critical Care and A-Plus Recover is not guaranteed and AIA Bhd. may revise the premium by giving you 30 days’ prior notice in advance. The premium for A-Plus Life Cover is guaranteed...</code> | <code>Is the premium for all AIA Bhd. plans guaranteed?</code> | | <code>For the mother<br><br>has existed prior to the risk effective date;<br><br>is caused directly or indirectly by self-inflicted injuries, while sane or insane;<br><br>is resulted from the mother committing, attempting or provoking an assault or a felony or from any violation of law by the mother;<br><br>is caused while under the influence of alcohol or drugs unless taken as prescribed by a doctor. For the avoidance of doubt, a person is considered as under the influence of alcohol if the breath, blood or urine test result is over the<br><br>35 mcg of alcohol per 100ml of breath<br><br>80 mg of alcohol per 100ml of blood<br><br>107 mg alcohol per 100ml of urine;<br><br>is caused directly or indirectly by the existence of Acquired Immune Deficiency Syndrome (AIDS) or by the presence of any Human Immuno-deficiency Virus (HIV) infection. The Company reserves the right to require the mother to undergo a<br><br>is resulted from the mother choosing to have a termination of pregnancy other than for medical reasons;<br><br>is caused by any unlawful, criminal o...</code> | <code>What are the specific blood alcohol content (BAC) levels that would lead to a life insurance claim denial for the mother?</code> | | <code>FOR YOUR ATTENTION<br><br>1. A-Plus Health 2 is an optional rider attachable to regular premium investment-linked plans, underwritten by AIA Bhd.<br><br>2. This brochure contains only a summary of the main features of the rider and is not exhaustive. It does not constitute a policy. You are advised to refer to the Sales Illustration and Product Disclosure Sheet for more details of the rider before purchasing, and refer to the terms and conditions in the policy for details of the features and benefits, exclusions and waiting periods under the policy.<br><br>3. Buying life insurance is a long-term financial commitment. You should satisfy yourself that the policy (including riders, if any) will best serve your needs and that the premium payable under the policy is an amount you can afford. To achieve this, we recommend that you speak to your Life Planner to perform a needs analysis and assist you in making an informed decision. You may also contact AIA Bhd. directly for more information.<br><br>If you cancel the...</code> | <code>What company underwrites the A-Plus Health 2 rider?</code> |
• —Loss: <code>MatryoshkaLoss</code> with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

Training Hyperparameters

Non-Default Hyperparameters

• —eval_strategy: epoch
• —gradient_accumulation_steps: 16
• —learning_rate: 2e-05
• —num_train_epochs: 10
• —lr_scheduler_type: cosine
• —warmup_ratio: 0.1
• —fp16: True
• —tf32: False
• —load_best_model_at_end: True
• —optim: adamwtorchfused
• —batch_sampler: no_duplicates

All Hyperparameters

<details><summary>Click to expand</summary>

• —overwrite_output_dir: False
• —do_predict: False
• —eval_strategy: epoch
• —prediction_loss_only: True
• —per_device_train_batch_size: 8
• —per_device_eval_batch_size: 8
• —per_gpu_train_batch_size: None
• —per_gpu_eval_batch_size: None
• —gradient_accumulation_steps: 16
• —eval_accumulation_steps: None
• —torch_empty_cache_steps: None
• —learning_rate: 2e-05
• —weight_decay: 0.0
• —adam_beta1: 0.9
• —adam_beta2: 0.999
• —adam_epsilon: 1e-08
• —max_grad_norm: 1.0
• —num_train_epochs: 10
• —max_steps: -1
• —lr_scheduler_type: cosine
• —lr_scheduler_kwargs: {}
• —warmup_ratio: 0.1
• —warmup_steps: 0
• —log_level: passive
• —log_level_replica: warning
• —log_on_each_node: True
• —logging_nan_inf_filter: True
• —save_safetensors: True
• —save_on_each_node: False
• —save_only_model: False
• —restore_callback_states_from_checkpoint: False
• —no_cuda: False
• —use_cpu: False
• —use_mps_device: False
• —seed: 42
• —data_seed: None
• —jit_mode_eval: False
• —use_ipex: False
• —bf16: False
• —fp16: True
• —fp16_opt_level: O1
• —half_precision_backend: auto
• —bf16_full_eval: False
• —fp16_full_eval: False
• —tf32: False
• —local_rank: 0
• —ddp_backend: None
• —tpu_num_cores: None
• —tpu_metrics_debug: False
• —debug: []
• —dataloader_drop_last: False
• —dataloader_num_workers: 0
• —dataloader_prefetch_factor: None
• —past_index: -1
• —disable_tqdm: False
• —remove_unused_columns: True
• —label_names: None
• —load_best_model_at_end: True
• —ignore_data_skip: False
• —fsdp: []
• —fsdp_min_num_params: 0
• —fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
• —fsdp_transformer_layer_cls_to_wrap: None
• —accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
• —deepspeed: None
• —label_smoothing_factor: 0.0
• —optim: adamwtorchfused
• —optim_args: None
• —adafactor: False
• —group_by_length: False
• —length_column_name: length
• —ddp_find_unused_parameters: None
• —ddp_bucket_cap_mb: None
• —ddp_broadcast_buffers: False
• —dataloader_pin_memory: True
• —dataloader_persistent_workers: False
• —skip_memory_metrics: True
• —use_legacy_prediction_loop: False
• —push_to_hub: False
• —resume_from_checkpoint: None
• —hub_model_id: None
• —hub_strategy: every_save
• —hub_private_repo: None
• —hub_always_push: False
• —gradient_checkpointing: False
• —gradient_checkpointing_kwargs: None
• —include_inputs_for_metrics: False
• —include_for_metrics: []
• —eval_do_concat_batches: True
• —fp16_backend: auto
• —push_to_hub_model_id: None
• —push_to_hub_organization: None
• —mp_parameters:
• —auto_find_batch_size: False
• —full_determinism: False
• —torchdynamo: None
• —ray_scope: last
• —ddp_timeout: 1800
• —torch_compile: False
• —torch_compile_backend: None
• —torch_compile_mode: None
• —dispatch_batches: None
• —split_batches: None
• —include_tokens_per_second: False
• —include_num_input_tokens_seen: False
• —neftune_noise_alpha: None
• —optim_target_modules: None
• —batch_eval_metrics: False
• —eval_on_start: False
• —use_liger_kernel: False
• —eval_use_gather_object: False
• —average_tokens_across_devices: False
• —prompts: None
• —batch_sampler: no_duplicates
• —multi_dataset_batch_sampler: proportional

</details>

Training Logs

• —The bold row denotes the saved checkpoint.

Framework Versions

• —Python: 3.11.11
• —Sentence Transformers: 3.4.1
• —Transformers: 4.48.2
• —PyTorch: 2.5.1+cu124
• —Accelerate: 1.2.1
• —Datasets: 2.19.1
• —Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->