CoolFace
Modelpublic

Detomo/cl-nagoya-sup-simcse-ja-nss-v0_9_15

sourceHugging Faceupdated 1y agoView on Hugging Face
1likes21downloads
Model Card

SentenceTransformer

This is a sentence-transformers model trained. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • —Model Type: Sentence Transformer <!-- - Base model: Unknown -->
  • —Maximum Sequence Length: 512 tokens
  • —Output Dimensionality: 768 dimensions
  • —Similarity Function: Cosine Similarity <!-- - Training Dataset: Unknown --> <!-- - Language: Unknown --> <!-- - License: Unknown -->

Model Sources

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

bash
pip install -U sentence-transformers

Then you can load this model and run inference.

python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Detomo/cl-nagoya-sup-simcse-ja-nss-v0_9_15")
# Run inference
sentences = [
    '科目：塗装。名称：PCa保護塗り（細幅物）。',
    '科目：塗装。名称：PCa面塗り（細幅物）。',
    '科目：塗装。名称：PCa面塗り（細幅物）。',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

Unnamed Dataset
  • —Size: 7,598 training samples
  • —Columns: <code>sentence</code> and <code>label</code>
  • —Approximate statistics based on the first 1000 samples: | | sentence | label | |:--------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | type | string | int | | details | <ul><li>min: 11 tokens</li><li>mean: 17.2 tokens</li><li>max: 29 tokens</li></ul> | <ul><li>0: ~0.30%</li><li>1: ~0.30%</li><li>2: ~0.30%</li><li>3: ~0.30%</li><li>4: ~0.30%</li><li>5: ~0.30%</li><li>6: ~0.30%</li><li>7: ~0.30%</li><li>8: ~0.30%</li><li>9: ~0.30%</li><li>10: ~0.30%</li><li>11: ~0.40%</li><li>12: ~0.30%</li><li>13: ~0.30%</li><li>14: ~0.30%</li><li>15: ~0.30%</li><li>16: ~0.30%</li><li>17: ~0.30%</li><li>18: ~0.50%</li><li>19: ~0.30%</li><li>20: ~0.30%</li><li>21: ~0.30%</li><li>22: ~0.30%</li><li>23: ~0.30%</li><li>24: ~0.30%</li><li>25: ~0.30%</li><li>26: ~0.30%</li><li>27: ~0.30%</li><li>28: ~0.30%</li><li>29: ~0.30%</li><li>30: ~0.30%</li><li>31: ~0.30%</li><li>32: ~0.30%</li><li>33: ~0.30%</li><li>34: ~0.30%</li><li>35: ~0.30%</li><li>36: ~0.30%</li><li>37: ~0.30%</li><li>38: ~0.30%</li><li>39: ~0.30%</li><li>40: ~0.40%</li><li>41: ~0.30%</li><li>42: ~0.30%</li><li>43: ~0.30%</li><li>44: ~0.60%</li><li>45: ~0.70%</li><li>46: ~0.30%</li><li>47: ~0.30%</li><li>48: ~0.30%</li><li>49: ~0.30%</li><li>50: ~0.30%</li><li>51: ~0.30%</li><li>52: ~0.30%</li><li>53: ~0.30%</li><li>54: ~0.30%</li><li>55: ~0.30%</li><li>56: ~0.30%</li><li>57: ~0.80%</li><li>58: ~0.30%</li><li>59: ~0.30%</li><li>60: ~0.60%</li><li>61: ~0.30%</li><li>62: ~0.30%</li><li>63: ~0.30%</li><li>64: ~0.50%</li><li>65: ~0.30%</li><li>66: ~0.30%</li><li>67: ~0.30%</li><li>68: ~0.30%</li><li>69: ~0.30%</li><li>70: ~0.60%</li><li>71: ~0.30%</li><li>72: ~0.30%</li><li>73: ~0.30%</li><li>74: ~0.30%</li><li>75: ~0.30%</li><li>76: ~0.30%</li><li>77: ~0.30%</li><li>78: ~0.30%</li><li>79: ~0.30%</li><li>80: ~0.30%</li><li>81: ~0.30%</li><li>82: ~0.30%</li><li>83: ~0.30%</li><li>84: ~0.80%</li><li>85: ~0.60%</li><li>86: ~0.50%</li><li>87: ~0.30%</li><li>88: ~0.30%</li><li>89: ~16.30%</li><li>90: ~0.30%</li><li>91: ~0.30%</li><li>92: ~0.30%</li><li>93: ~0.30%</li><li>94: ~0.30%</li><li>95: ~0.30%</li><li>96: ~0.30%</li><li>97: ~0.30%</li><li>98: ~0.50%</li><li>99: ~0.30%</li><li>100: ~0.30%</li><li>101: ~0.30%</li><li>102: ~0.30%</li><li>103: ~0.30%</li><li>104: ~0.30%</li><li>105: ~0.30%</li><li>106: ~1.20%</li><li>107: ~0.70%</li><li>108: ~0.30%</li><li>109: ~3.20%</li><li>110: ~0.30%</li><li>111: ~2.30%</li><li>112: ~0.30%</li><li>113: ~0.30%</li><li>114: ~0.50%</li><li>115: ~0.50%</li><li>116: ~0.50%</li><li>117: ~0.30%</li><li>118: ~0.30%</li><li>119: ~0.30%</li><li>120: ~0.80%</li><li>121: ~0.30%</li><li>122: ~0.30%</li><li>123: ~0.30%</li><li>124: ~0.30%</li><li>125: ~0.30%</li><li>126: ~0.30%</li><li>127: ~0.30%</li><li>128: ~0.30%</li><li>129: ~0.30%</li><li>130: ~0.30%</li><li>131: ~0.40%</li><li>132: ~0.30%</li><li>133: ~0.30%</li><li>134: ~0.30%</li><li>135: ~0.30%</li><li>136: ~0.30%</li><li>137: ~0.30%</li><li>138: ~0.30%</li><li>139: ~0.30%</li><li>140: ~0.30%</li><li>141: ~0.30%</li><li>142: ~0.40%</li><li>143: ~0.30%</li><li>144: ~0.30%</li><li>145: ~0.30%</li><li>146: ~0.30%</li><li>147: ~0.30%</li><li>148: ~0.30%</li><li>149: ~0.70%</li><li>150: ~0.30%</li><li>151: ~0.30%</li><li>152: ~0.30%</li><li>153: ~1.30%</li><li>154: ~0.30%</li><li>155: ~0.30%</li><li>156: ~0.30%</li><li>157: ~0.30%</li><li>158: ~0.30%</li><li>159: ~1.30%</li><li>160: ~0.30%</li><li>161: ~0.30%</li><li>162: ~0.30%</li><li>163: ~0.30%</li><li>164: ~0.30%</li><li>165: ~0.30%</li><li>166: ~0.30%</li><li>167: ~1.50%</li><li>168: ~0.30%</li><li>169: ~0.30%</li><li>170: ~7.90%</li><li>171: ~0.30%</li><li>172: ~1.00%</li><li>173: ~0.30%</li><li>174: ~0.30%</li><li>175: ~0.30%</li><li>176: ~1.80%</li><li>177: ~0.30%</li><li>178: ~0.50%</li><li>179: ~0.70%</li><li>180: ~0.30%</li><li>181: ~0.30%</li><li>182: ~0.30%</li><li>183: ~0.30%</li><li>184: ~0.30%</li><li>185: ~0.30%</li><li>186: ~0.30%</li><li>187: ~0.30%</li><li>188: ~2.50%</li></ul> |
  • —Samples: | sentence | label | |:-----------------------------------------|:---------------| | <code>科目：コンクリート。名称：免震基礎天端ｸﾞﾗｳﾄ注入。</code> | <code>0</code> | | <code>科目：コンクリート。名称：免震基礎天端ｸﾞﾗｳﾄ注入。</code> | <code>0</code> | | <code>科目：コンクリート。名称：免震基礎天端ｸﾞﾗｳﾄ注入。</code> | <code>0</code> |
  • —Loss: <code>sentencetransformerlib.custombatchalltriploss.CustomBatchAllTripletLoss</code>

Training Hyperparameters

Non-Default Hyperparameters
  • —per_device_train_batch_size: 512
  • —per_device_eval_batch_size: 512
  • —learning_rate: 1e-05
  • —weight_decay: 0.01
  • —num_train_epochs: 250
  • —warmup_ratio: 0.1
  • —fp16: True
  • —batch_sampler: groupbylabel
All Hyperparameters

<details><summary>Click to expand</summary>

  • —overwrite_output_dir: False
  • —do_predict: False
  • —eval_strategy: no
  • —prediction_loss_only: True
  • —per_device_train_batch_size: 512
  • —per_device_eval_batch_size: 512
  • —per_gpu_train_batch_size: None
  • —per_gpu_eval_batch_size: None
  • —gradient_accumulation_steps: 1
  • —eval_accumulation_steps: None
  • —torch_empty_cache_steps: None
  • —learning_rate: 1e-05
  • —weight_decay: 0.01
  • —adam_beta1: 0.9
  • —adam_beta2: 0.999
  • —adam_epsilon: 1e-08
  • —max_grad_norm: 1.0
  • —num_train_epochs: 250
  • —max_steps: -1
  • —lr_scheduler_type: linear
  • —lr_scheduler_kwargs: {}
  • —warmup_ratio: 0.1
  • —warmup_steps: 0
  • —log_level: passive
  • —log_level_replica: warning
  • —log_on_each_node: True
  • —logging_nan_inf_filter: True
  • —save_safetensors: True
  • —save_on_each_node: False
  • —save_only_model: False
  • —restore_callback_states_from_checkpoint: False
  • —no_cuda: False
  • —use_cpu: False
  • —use_mps_device: False
  • —seed: 42
  • —data_seed: None
  • —jit_mode_eval: False
  • —use_ipex: False
  • —bf16: False
  • —fp16: True
  • —fp16_opt_level: O1
  • —half_precision_backend: auto
  • —bf16_full_eval: False
  • —fp16_full_eval: False
  • —tf32: None
  • —local_rank: 0
  • —ddp_backend: None
  • —tpu_num_cores: None
  • —tpu_metrics_debug: False
  • —debug: []
  • —dataloader_drop_last: False
  • —dataloader_num_workers: 0
  • —dataloader_prefetch_factor: None
  • —past_index: -1
  • —disable_tqdm: False
  • —remove_unused_columns: True
  • —label_names: None
  • —load_best_model_at_end: False
  • —ignore_data_skip: False
  • —fsdp: []
  • —fsdp_min_num_params: 0
  • —fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
  • —tp_size: 0
  • —fsdp_transformer_layer_cls_to_wrap: None
  • —accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
  • —deepspeed: None
  • —label_smoothing_factor: 0.0
  • —optim: adamw_torch
  • —optim_args: None
  • —adafactor: False
  • —group_by_length: False
  • —length_column_name: length
  • —ddp_find_unused_parameters: None
  • —ddp_bucket_cap_mb: None
  • —ddp_broadcast_buffers: False
  • —dataloader_pin_memory: True
  • —dataloader_persistent_workers: False
  • —skip_memory_metrics: True
  • —use_legacy_prediction_loop: False
  • —push_to_hub: False
  • —resume_from_checkpoint: None
  • —hub_model_id: None
  • —hub_strategy: every_save
  • —hub_private_repo: None
  • —hub_always_push: False
  • —gradient_checkpointing: False
  • —gradient_checkpointing_kwargs: None
  • —include_inputs_for_metrics: False
  • —include_for_metrics: []
  • —eval_do_concat_batches: True
  • —fp16_backend: auto
  • —push_to_hub_model_id: None
  • —push_to_hub_organization: None
  • —mp_parameters:
  • —auto_find_batch_size: False
  • —full_determinism: False
  • —torchdynamo: None
  • —ray_scope: last
  • —ddp_timeout: 1800
  • —torch_compile: False
  • —torch_compile_backend: None
  • —torch_compile_mode: None
  • —dispatch_batches: None
  • —split_batches: None
  • —include_tokens_per_second: False
  • —include_num_input_tokens_seen: False
  • —neftune_noise_alpha: None
  • —optim_target_modules: None
  • —batch_eval_metrics: False
  • —eval_on_start: False
  • —use_liger_kernel: False
  • —eval_use_gather_object: False
  • —average_tokens_across_devices: False
  • —prompts: None
  • —batch_sampler: groupbylabel
  • —multi_dataset_batch_sampler: proportional

</details>

Training Logs

EpochStepTraining Loss
0.6667100.0662
1.3333200.0
2.0300.0
2.6667400.0
3.3333500.0
4.0600.0
4.6667700.0
5.3333800.0
6.0900.0
6.66671000.0
7.33331100.0
8.01200.0
8.66671300.0
9.33331400.0
10.01500.0
10.0102.7711
20.0201.2115
30.0300.3753
40.0400.1646
50.0500.0876
60.0600.0559
70.0700.0344
80.0800.0262
90.0900.0194
100.01000.0218
110.01100.0214
120.01200.014
130.01300.0231
140.01400.0132
150.01500.0146
3.75761000.0701
7.75762000.0747
11.75763000.0709
15.75764000.0689
19.75765000.0622
23.75766000.0639
27.75767000.063
31.75768000.0605
35.75769000.061
39.757610000.0602
43.757611000.0609
47.757612000.0596
51.757613000.0568
55.757614000.0593
59.757615000.058
63.757616000.0613
67.757617000.0515
71.757618000.0511
75.757619000.0538
79.757620000.0559
83.757621000.0482
87.757622000.0511
91.757623000.0553
95.757624000.0522
99.757625000.0534
103.757626000.0477
107.757627000.052
111.757628000.0518
115.757629000.047
119.757630000.0503
123.757631000.0494
127.757632000.0488
131.757633000.052
135.757634000.0459
139.757635000.0467
143.757636000.0493
147.757637000.0453
151.757638000.0457
155.757639000.0462
159.757640000.0451
163.757641000.0446
167.757642000.0438
171.757643000.0398
175.757644000.0414
179.757645000.045
183.757646000.0448
187.757647000.0426
191.757648000.0427
195.757649000.0434
199.757650000.039
203.757651000.0381
207.757652000.0434
211.757653000.041
215.757654000.0463
219.757655000.0386
223.757656000.0453
227.757657000.0412
231.757658000.0373
235.757659000.0393
239.757660000.0362
243.757661000.0363
247.757662000.0372

Framework Versions

  • —Python: 3.11.12
  • —Sentence Transformers: 3.4.1
  • —Transformers: 4.50.3
  • —PyTorch: 2.6.0+cu124
  • —Accelerate: 1.5.2
  • —Datasets: 3.5.0
  • —Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers
bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
CustomBatchAllTripletLoss
bibtex
@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->