swardiantara/bert-tiny-yelp-k3-fixed-euclidean

SentenceTransformer based on google/bertuncasedL-2H-128A-2

This is a sentence-transformers model finetuned from google/bert_uncased_L-2_H-128_A-2. It maps sentences & paragraphs to a 128-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• —Model Type: Sentence Transformer
• —Base model: google/bert_uncased_L-2_H-128_A-2 <!-- at revision 30b0a37ccaaa32f332884b96992754e246e48c5f -->
• —Maximum Sequence Length: 128 tokens
• —Output Dimensionality: 128 dimensions
• —Similarity Function: Cosine Similarity
• —Supported Modality: Text <!-- - Training Dataset: Unknown --> <!-- - Language: Unknown --> <!-- - License: Unknown -->

Model Sources

• —Documentation: Sentence Transformers Documentation
• —Repository: Sentence Transformers on GitHub
• —Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 128, 'pooling_mode': 'mean', 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("swardiantara/bert-tiny-yelp-k3-fixed-euclidean")
# Run inference
sentences = [
    "This place is absolute garbage...  Half of the tees are not available, including all the grass tees.  It is cash only, and they sell the last bucket at 8, despite having lights.  And if you finish even a minute after 8, don't plan on getting a drink.  The vending machines are sold out (of course) and they sell drinks inside, but close the drawers at 8 on the dot.  There are weeds grown all over the place.  I noticed some sort of batting cage, but it looks like those are out of order as well.  Someone should buy this place and turn it into what it should be.",
    '\\"Absolute Perfect Dining!\\"\\n\\nBoth my husband and I ate at this restaurant on the fourth of July and were amazed at the views and all the firework displays all over Las Vegas. What a spectacular view this restaurant provided. The food, ambiance and service were superb. If you want to try something fun for dessert, try the three scoops of sorbet (raspberry, mango and orange) with mint and crystalized sugar. What a treat! Make sure you share this dessert as it is very large. In fact, everything they serve is large so make sure you have an appetite. We enjoyed ourselves and will definitely return. \\n\\n\\nVisited July 2013',
    "This is an older retail store.  I normally have a loving relationship with Eat 'n Park, but this one just didn't hit home for me.  I came during the Sunday Brunch Buffet; my friend partook, I abstained after seeing the fare.  I ordered two eggs over-easy with a side of toast.  Getting beyond the murky, dated feel of the store, it was actually fairly clean on the interior and in the restrooms.  \\n\\nI received my food and the eggs were cooked perfectly.  Maybe not a big deal for some, but you would be surprised at how often over-easy comes out over-hard or, worse, uncooked.  I think the real reason I enjoy Eat 'n Park most of the time is the normal salad bar items.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 128]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9956, 0.9968],
#         [0.9956, 1.0000, 0.9930],
#         [0.9968, 0.9930, 1.0000]])

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

Unnamed Dataset

• —Size: 7,605,030 training samples
• —Columns: <code>texta</code>, <code>textb</code>, and <code>label</code>
• —Approximate statistics based on the first 100 samples: | | texta | textb | label | |:---------|:-------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|:-----------------------------------| | type | string | string | list | | modality | text | text | | | details | <ul><li>min: 62 tokens</li><li>mean: 104.38 tokens</li><li>max: 128 tokens</li></ul> | <ul><li>min: 52 tokens</li><li>mean: 115.28 tokens</li><li>max: 128 tokens</li></ul> | <ul><li>size: 2 elements</li></ul> |
• —Samples: | texta | textb | label | |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------| | <code>dr. goldberg offers everything i look for in a general practitioner. he's nice and easy to talk to without being patronizing; he's always on time in seeing his patients; he's affiliated with a top-notch hospital (nyu) which my parents have explained to me is very important in case something happens and you need surgery; and you can get referrals to see specialists without having to see him first. really, what more do you need? i'm sitting here trying to think of any complaints i have about him, but i'm really drawing a blank.</code> | <code>I might be in love. This place may have just blown my last crush away! I had driven by this rather nondescript place a million times thinking, \"I really need to stop in there and check it out\". I also had a fellow, respected cocktail-connoisseur tell me I had to go here for the amazing cocktails and ambiance...that was over a year ago. So I finally mosied in on a Thursday night and was IMPRESSED. Shady's has great tunes coming out of the jukebox, great bartenders that made fantastic beverages, and quite a friendly cool crowd. For all the beer people they also have a good variety of beers. Oh and did I mention it is INEXPENSIVE! I thought they screwed something up when the bill arrived because it was so FREAKING cheap. I went back Friday and Saturday just to confirm: Shady's is indeed AWESOME.</code> | <code>[1.0, 0.0]</code> | | <code>dr. goldberg offers everything i look for in a general practitioner. he's nice and easy to talk to without being patronizing; he's always on time in seeing his patients; he's affiliated with a top-notch hospital (nyu) which my parents have explained to me is very important in case something happens and you need surgery; and you can get referrals to see specialists without having to see him first. really, what more do you need? i'm sitting here trying to think of any complaints i have about him, but i'm really drawing a blank.</code> | <code>No, it's not worth it! Bellagio overcharged me and their accounting department never got back to me (I asked them for an itemized receipt 2 weeks ago). When I called them, I was (twice!) put on this music-less \"dead\" hold for 30-40 minutes. They're so full of themselves, they don't provide good customer service. Their wi-fi needed fixing when I stayed there. And by the way, the woman at the buffet was super rude to me. Finally, the famous waterworks never happened while I was standing outside in the cold waiting (though i asked the staff and they said they're up and running). No, don't waste your money! I won't and won't let my family and friends do that either.</code> | <code>[0.0, 1.0]</code> | | <code>dr. goldberg offers everything i look for in a general practitioner. he's nice and easy to talk to without being patronizing; he's always on time in seeing his patients; he's affiliated with a top-notch hospital (nyu) which my parents have explained to me is very important in case something happens and you need surgery; and you can get referrals to see specialists without having to see him first. really, what more do you need? i'm sitting here trying to think of any complaints i have about him, but i'm really drawing a blank.</code> | <code>Lidia, you should be ashamed! This spot wouldn't have lasted a year in NYC. Citizens of Pittsburgh, don't be fooled. Lidia's is average at best. I ate here when it opened,a and again 4 years later. Nothing has changed. We did have some great bar nibbles, quality cocktails, and a good bottle of wine off of a pretty good list, but the food and service lacked. For starters we were served ice cold meatballs and limp soggy salads. The took away everything and only brought back fresh salads (mind you, we were one of about 10 tables being served that evening!). The entrees were dull and lacked seasoning. The servers were confused (how many times does it take to ask fro fresh pepper?) and our waiter seemed to hand off our table to another server. It's tough to find anything good here. Just another 'poser'.</code> | <code>[0.0, 1.0]</code> |
• —Loss: <code>_main_.OrdinalProxyContrastiveLoss</code>

Training Hyperparameters

Non-Default Hyperparameters

• —per_device_train_batch_size: 1024
• —learning_rate: 1e-05
• —load_best_model_at_end: True

All Hyperparameters

<details><summary>Click to expand</summary>

• —per_device_train_batch_size: 1024
• —num_train_epochs: 3
• —max_steps: -1
• —learning_rate: 1e-05
• —lr_scheduler_type: linear
• —lr_scheduler_kwargs: None
• —warmup_steps: 0
• —optim: adamw_torch
• —optim_args: None
• —weight_decay: 0.0
• —adam_beta1: 0.9
• —adam_beta2: 0.999
• —adam_epsilon: 1e-08
• —optim_target_modules: None
• —gradient_accumulation_steps: 1
• —average_tokens_across_devices: True
• —max_grad_norm: 1.0
• —label_smoothing_factor: 0.0
• —bf16: False
• —fp16: False
• —bf16_full_eval: False
• —fp16_full_eval: False
• —tf32: None
• —gradient_checkpointing: False
• —gradient_checkpointing_kwargs: None
• —torch_compile: False
• —torch_compile_backend: None
• —torch_compile_mode: None
• —use_liger_kernel: False
• —liger_kernel_config: None
• —use_cache: False
• —neftune_noise_alpha: None
• —torch_empty_cache_steps: None
• —auto_find_batch_size: False
• —log_on_each_node: True
• —logging_nan_inf_filter: True
• —include_num_input_tokens_seen: no
• —log_level: passive
• —log_level_replica: warning
• —disable_tqdm: False
• —project: huggingface
• —trackio_space_id: None
• —trackio_bucket_id: None
• —trackio_static_space_id: None
• —per_device_eval_batch_size: 8
• —prediction_loss_only: True
• —eval_on_start: False
• —eval_do_concat_batches: True
• —eval_use_gather_object: False
• —eval_accumulation_steps: None
• —include_for_metrics: []
• —batch_eval_metrics: False
• —save_only_model: False
• —save_on_each_node: False
• —enable_jit_checkpoint: False
• —push_to_hub: False
• —hub_private_repo: None
• —hub_model_id: None
• —hub_strategy: every_save
• —hub_always_push: False
• —hub_revision: None
• —load_best_model_at_end: True
• —ignore_data_skip: False
• —restore_callback_states_from_checkpoint: False
• —full_determinism: False
• —seed: 42
• —data_seed: None
• —use_cpu: False
• —accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
• —parallelism_config: None
• —dataloader_drop_last: False
• —dataloader_num_workers: 0
• —dataloader_pin_memory: True
• —dataloader_persistent_workers: False
• —dataloader_prefetch_factor: None
• —remove_unused_columns: True
• —label_names: None
• —train_sampling_strategy: random
• —length_column_name: length
• —ddp_find_unused_parameters: None
• —ddp_bucket_cap_mb: None
• —ddp_broadcast_buffers: False
• —ddp_static_graph: None
• —ddp_backend: None
• —ddp_timeout: 1800
• —fsdp: None
• —fsdp_config: None
• —deepspeed: None
• —debug: []
• —skip_memory_metrics: True
• —do_predict: False
• —resume_from_checkpoint: None
• —warmup_ratio: None
• —local_rank: -1
• —prompts: None
• —batch_sampler: batch_sampler
• —multi_dataset_batch_sampler: proportional
• —router_mapping: {}
• —learning_rate_mapping: {}

</details>

Training Logs

• —The bold row denotes the saved checkpoint.

Training Time

• —Training: 1.7 hours
• —Evaluation: 34.7 seconds
• —Total: 1.7 hours

Framework Versions

• —Python: 3.12.4
• —Sentence Transformers: 5.5.1
• —Transformers: 5.11.0
• —PyTorch: 2.5.1+cu121
• —Accelerate: 1.13.0
• —Datasets: 2.21.0
• —Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->