meandyou200175/sp_chatbot_query

SentenceTransformer based on vinai/phobert-base-v2

This is a sentence-transformers model finetuned from vinai/phobert-base-v2. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• —Model Type: Sentence Transformer
• —Base model: vinai/phobert-base-v2 <!-- at revision e2375d266bdf39c6e8e9a87af16a5da3190b0cc8 -->
• —Maximum Sequence Length: 256 tokens
• —Output Dimensionality: 768 dimensions
• —Similarity Function: Cosine Similarity <!-- - Training Dataset: Unknown --> <!-- - Language: Unknown --> <!-- - License: Unknown -->

Model Sources

• —Documentation: Sentence Transformers Documentation
• —Repository: Sentence Transformers on GitHub
• —Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: RobertaModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("meandyou200175/sp_chatbot_query")
# Run inference
sentences = [
    'cho tôi máy hút bụi công suất hút trên 8kPa và pin chạy ít nhất 40 phút',
    'Robot hút bụi Xiaomi Vacuum X10, lực hút 20000Pa (20kPa), pin 60 phút, Giá: 11.900.000',
    'Lò nướng Sunhouse SHD4260, dung tích 45L, công suất 1600W, Giá 1.150.000',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

Evaluation

Metrics

Information Retrieval

• —Evaluated with <code>InformationRetrievalEvaluator</code>

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

Unnamed Dataset

• —Size: 14,462 training samples
• —Columns: <code>query</code> and <code>positive</code>
• —Approximate statistics based on the first 1000 samples: | | query | positive | |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | | details | <ul><li>min: 5 tokens</li><li>mean: 14.29 tokens</li><li>max: 27 tokens</li></ul> | <ul><li>min: 17 tokens</li><li>mean: 35.68 tokens</li><li>max: 133 tokens</li></ul> |
• —Samples: | query | positive | |:--------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>Có cân điện tử y tế dưới 1.068.000.000 VNĐ không</code> | <code>Omron HN-286 - . Cân điện tử sức khỏe, hiển thị cân nặng & BMI, mặt kính chịu lực, pin AA x 2, thiết kế gọn nhẹ, vận hành êm. Kích thước: 300 x 300 x 25 mm. Trọng lượng: 2 kg. Giá: 890.000 VNĐ</code> | | <code>cần nồi cơm điện công suất trên 700W</code> | <code>Nồi cơm điện Sharp KS-11ETV, Công suất 750W, Dung tích 1.1L, Giá: 1.050.000</code> | | <code>cho tôi màn hình máy tính kích thước tối thiểu 23 inch</code> | <code>Màn hình LG UltraGear 27GN950, 27 inch, 4K UHD, 144Hz, Nano IPS, Giá: 16.800.000</code> |
• —Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }

Evaluation Dataset

Unnamed Dataset

• —Size: 1,607 evaluation samples
• —Columns: <code>query</code> and <code>positive</code>
• —Approximate statistics based on the first 1000 samples: | | query | positive | |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | | details | <ul><li>min: 5 tokens</li><li>mean: 14.33 tokens</li><li>max: 27 tokens</li></ul> | <ul><li>min: 18 tokens</li><li>mean: 37.23 tokens</li><li>max: 139 tokens</li></ul> |
• —Samples: | query | positive | |:-------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>mình cần máy lạnh giá trong khoảng 15 đến 20 triệu, công suất trên 12.000 BTU và tiết kiệm điện 5 sao</code> | <code>Điều hòa Panasonic Inverter 1.5HP, Công suất 12.700 BTU, Công nghệ NanoeX, Giá: 18.200.000</code> | | <code>tôi muốn mua máy chiếu độ sáng trên 3.500 lumen và giá nhỏ hơn 19 triệu</code> | <code>Máy chiếu Epson EB-X51, Độ sáng 3.700 lumen, Độ phân giải XGA, Giá: 14.200.000</code> | | <code>Có bộ đồ trang điểm dưới 1.741.500.000 VNĐ không</code> | <code>Sephora Basics Kit - . Bao gồm 12 màu phấn mắt, 4 màu má hồng, 2 màu son, cọ trang điểm, hộp gọn nhẹ, thích hợp đi du lịch, chất liệu an toàn cho da. Kích thước: 300 x 200 x 50 mm. Trọng lượng: 0.8 kg. Giá: 1.290.000 VNĐ</code> |
• —Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }

Training Hyperparameters

Non-Default Hyperparameters

• —eval_strategy: steps
• —per_device_train_batch_size: 2
• —per_device_eval_batch_size: 2
• —learning_rate: 2e-05
• —num_train_epochs: 6
• —warmup_ratio: 0.1
• —fp16: True
• —batch_sampler: no_duplicates

All Hyperparameters

<details><summary>Click to expand</summary>

• —overwrite_output_dir: False
• —do_predict: False
• —eval_strategy: steps
• —prediction_loss_only: True
• —per_device_train_batch_size: 2
• —per_device_eval_batch_size: 2
• —per_gpu_train_batch_size: None
• —per_gpu_eval_batch_size: None
• —gradient_accumulation_steps: 1
• —eval_accumulation_steps: None
• —torch_empty_cache_steps: None
• —learning_rate: 2e-05
• —weight_decay: 0.0
• —adam_beta1: 0.9
• —adam_beta2: 0.999
• —adam_epsilon: 1e-08
• —max_grad_norm: 1.0
• —num_train_epochs: 6
• —max_steps: -1
• —lr_scheduler_type: linear
• —lr_scheduler_kwargs: {}
• —warmup_ratio: 0.1
• —warmup_steps: 0
• —log_level: passive
• —log_level_replica: warning
• —log_on_each_node: True
• —logging_nan_inf_filter: True
• —save_safetensors: True
• —save_on_each_node: False
• —save_only_model: False
• —restore_callback_states_from_checkpoint: False
• —no_cuda: False
• —use_cpu: False
• —use_mps_device: False
• —seed: 42
• —data_seed: None
• —jit_mode_eval: False
• —use_ipex: False
• —bf16: False
• —fp16: True
• —fp16_opt_level: O1
• —half_precision_backend: auto
• —bf16_full_eval: False
• —fp16_full_eval: False
• —tf32: None
• —local_rank: 0
• —ddp_backend: None
• —tpu_num_cores: None
• —tpu_metrics_debug: False
• —debug: []
• —dataloader_drop_last: False
• —dataloader_num_workers: 0
• —dataloader_prefetch_factor: None
• —past_index: -1
• —disable_tqdm: False
• —remove_unused_columns: True
• —label_names: None
• —load_best_model_at_end: False
• —ignore_data_skip: False
• —fsdp: []
• —fsdp_min_num_params: 0
• —fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
• —fsdp_transformer_layer_cls_to_wrap: None
• —accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
• —deepspeed: None
• —label_smoothing_factor: 0.0
• —optim: adamw_torch
• —optim_args: None
• —adafactor: False
• —group_by_length: False
• —length_column_name: length
• —ddp_find_unused_parameters: None
• —ddp_bucket_cap_mb: None
• —ddp_broadcast_buffers: False
• —dataloader_pin_memory: True
• —dataloader_persistent_workers: False
• —skip_memory_metrics: True
• —use_legacy_prediction_loop: False
• —push_to_hub: False
• —resume_from_checkpoint: None
• —hub_model_id: None
• —hub_strategy: every_save
• —hub_private_repo: None
• —hub_always_push: False
• —gradient_checkpointing: False
• —gradient_checkpointing_kwargs: None
• —include_inputs_for_metrics: False
• —include_for_metrics: []
• —eval_do_concat_batches: True
• —fp16_backend: auto
• —push_to_hub_model_id: None
• —push_to_hub_organization: None
• —mp_parameters:
• —auto_find_batch_size: False
• —full_determinism: False
• —torchdynamo: None
• —ray_scope: last
• —ddp_timeout: 1800
• —torch_compile: False
• —torch_compile_backend: None
• —torch_compile_mode: None
• —include_tokens_per_second: False
• —include_num_input_tokens_seen: False
• —neftune_noise_alpha: None
• —optim_target_modules: None
• —batch_eval_metrics: False
• —eval_on_start: False
• —use_liger_kernel: False
• —eval_use_gather_object: False
• —average_tokens_across_devices: False
• —prompts: None
• —batch_sampler: no_duplicates
• —multi_dataset_batch_sampler: proportional

</details>

Training Logs

<details><summary>Click to expand</summary>

</details>

Framework Versions

• —Python: 3.11.13
• —Sentence Transformers: 4.1.0
• —Transformers: 4.52.4
• —PyTorch: 2.6.0+cu124
• —Accelerate: 1.8.1
• —Datasets: 3.6.0
• —Tokenizers: 0.21.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->