Modelpublic

Alessio-Borgi/all-mpnet-base-v2-margin-based-triplet-loss-finetuned-culture-5-epochs-enhanced

sourceHugging Faceupdated 1y agoView on Hugging Face

0likes18downloads

Model Card

SentenceTransformer based on sentence-transformers/all-mpnet-base-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-mpnet-base-v2. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

—Model Type: Sentence Transformer
—Base model: sentence-transformers/all-mpnet-base-v2
—Maximum Sequence Length: 384 tokens
—Output Dimensionality: 768 dimensions
—Similarity Function: Cosine Similarity

Model Sources

—Documentation: Sentence Transformers Documentation
—Repository: Sentence Transformers on GitHub
—Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 384, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

bash

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Alessio-Borgi/all-mpnet-base-v2-margin-based-triplet-loss-finetuned-culture-5-epochs-enhanced")
# Run inference
sentences = [
    'music of Japan overview of musical traditions in Japan In Japan, music includes a wide array of distinct genres, both traditional and modern. The word for "music" in Japanese is 音楽 (ongaku), combining the kanji 音 on (sound) with the kanji 楽 gaku (music, comfort). Japan is the world\'s largest market for music on physical media and the second-largest overall music market, with a retail value of US$2.7 billion in 2017. {\'aliases\': [\'music in Japan\', \'Japanese music\', \'Japan music\']} {\'instance of\': \'music genre\', \'country of origin\': \'United States\', \'described by source\': \'Brockhaus and Efron Encyclopedic Dictionary\', \'subclass of\': \'music genre\', \'on focus list of Wikimedia project\': \'WikiProject African diaspora\', \'inception\': \'{"time": "+1990-01-01T00:00:00Z", "timezone": 0, "before": 0, "after": 0, "precision": 9, "calendarmodel": "http://www.wikidata.org/entity/Q1985727"}\'}',
    'CoMix Wave Films Japanese animation studio CoMix Wave Films, Inc. (Japanese: 株式会社コミックス・ウェーブ・フィルム, Hepburn: Kabushiki-gaisha Komikkusu Uēbu Firumu) is a Japanese animation film studio and distribution company based in Chiyoda, Tokyo, Japan. The studio is known for its anime feature films, short films, and television commercials, particularly those made by director Makoto Shinkai. It was founded in March 2007 when it split from CoMix Wave Inc., which was initially formed in 1998 from Itochu Corporation, ASATSU (now ADK), and other companies. On October 15, 2024, Toho announced that they\'d acquired 45 shares or 6.09% of CoMix Wave Films. {\'name\': \'CoMix Wave Films, Inc.\', \'native_name\': \'株式会社コミックス・ウェーブ・フィルム\', \'native_name_lang\': \'ja\', \'romanized_name\': \'Kabushiki-gaisha Komikkusu Uēbu Firumu\', \'logo\': \'CoMix Wave Films logo.svg\', \'logo_size\': \'150px\', \'image_caption\': \'Headquarters in Suginami, Tokyo\', \'type\': \'Kabushiki kaisha\', \'foundation\': \'{{start date and age|2007|03}}\', \'location\': \'Ogikubo\', \'location_city\': \'Suginami, Tokyo\', \'location_country\': \'Japan\', \'key_people\': \'Noritaka Kawaguchi (CEO)<br/>Kazuki Sunama (board member)<br/>Tomohiro Tokunaga (board member)\', \'industry\': \'Motion pictures (anime)<br>TV commercials\', \'predecessor\': \'CoMix Wave Inc.\', \'assets\': \'¥95.1 million (2017)\', \'owner\': \'Toho (6.09%)\', \'products\': "Animated feature films (\'\'anime\'\'), short films, commercials", \'homepage\': \'{{URL|http://www.cwfilms.jp/}}\', \'aliases\': [\'Comix Wave Films\', \'CoMix Wave Films Inc.\', \'CoMix Wave Films, Inc.\', \'Kabushiki-gaisha Komikkusu Uēbu Firumu\']} {\'instance of\': \'animation studio\', \'industry\': \'anime industry\', \'country\': \'Japan\', \'product or material produced\': \'anime\', \'legal form\': \'kabushiki gaisha\', \'headquarters location\': \'Suginami-ku\'}',
    "white label record type of vinyl record labeling A white label record is a vinyl record with white labels attached. There are several variations each with a different purpose. Variations include test pressings, white label promos, and plain white labels. {'note': 'infobox not present in Wikipedia'} {'instance of': 'type of record label', 'subclass of': 'record label', 'country': 'Germany'}",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

Unnamed Dataset

—Size: 6,551 training samples
—Columns: <code>sentence0</code>, <code>sentence1</code>, and <code>sentence_2</code>
—Approximate statistics based on the first 1000 samples: | | sentence0 | sentence1 | sentence_2 | |:--------|:-------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------| | type | string | string | string | | details | <ul><li>min: 46 tokens</li><li>mean: 302.15 tokens</li><li>max: 384 tokens</li></ul> | <ul><li>min: 59 tokens</li><li>mean: 296.32 tokens</li><li>max: 384 tokens</li></ul> | <ul><li>min: 59 tokens</li><li>mean: 299.25 tokens</li><li>max: 384 tokens</li></ul> |
—Samples: | sentence0 | sentence1 | sentence2 | |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>Carolyn Carlson American dancer Carolyn Carlson (born 7 March 1943) is an American born French nationalized contemporary dance choreographer, performer, and poet. She is of Finnish descent. She is the director of the Centre Chorégraphique National in Roubaix and of the Atelier de Paris at La Cartoucherie de Vincennes in Paris. Carlson was awarded the title of Chevalier des Arts et des Lettres of the French Republic. {'birthdate': '{{birth date and age|1943|03|07}}', 'education': 'San Francisco School of Ballet; University of Utah'} {'instance of': 'human', 'occupation': 'choreographer', 'sex or gender': 'male', 'described by source': 'Obálky knih', 'country of citizenship': 'France', 'languages spoken, written or signed': 'English', 'genre': 'opera'}</code> | <code>Palace of Versailles palace in Versailles, France and location of the Museum of the History of France The Palace of Versailles ( vair-SY, vur-SY; French: château de Versailles [ʃɑto d(ə) vɛʁsɑj] ) is a former royal residence commissioned by King Louis XIV located in Versailles, about 18 kilometres (11 mi) west of Paris, in the Yvelines Department of Île-de-France region in France. The palace is owned by the government of France and since 1995 has been managed, under the direction of the French Ministry of Culture, by the Public Establishment of the Palace, Museum and National Estate of Versailles. About 15,000,000 people visit the palace, park, or gardens of Versailles every year, making it one of the most popular tourist attractions in the world. Louis XIII built a hunting lodge at Versailles in 1623. His successor, Louis XIV, expanded the château into a palace that went through several expansions in phases from 1661 to 1715. It was a favourite residence for both kings, and in 1682, L...</code> | <code>socks and sandals wearing socks and sandals together Wearing socks and sandals together is a controversial fashion combination and social phenomenon that is discussed in various countries and cultures. In some places it is considered a fashion faux pas. {'aliases': ['socks in sandals']} {}</code> | | <code>religious nationalism relationship between national identity and religion Religious nationalism can be understood in a number of ways, such as nationalism as a religion itself, a position articulated by Carlton Hayes in his text Nationalism: A Religion, or as the relationship of nationalism to a particular religious belief, dogma, ideology, or affiliation. This relationship can be broken down into two aspects: the politicisation of religion and the influence of religion on politics. In the former aspect, a shared religion can be seen to contribute to a sense of national unity, a common bond among the citizens of the nation. Another political aspect of religion is the support of a national identity, similar to a shared ethnicity, language, or culture. The influence of religion on politics is more ideological, where current interpretations of religious ideas inspire political activism and action; for example, laws are passed to foster stricter religious adherence. Ideologically-driven re...</code> | <code>Jugo species of plant Juglans regia, known by various common names including the common walnut, English walnut, or Persian walnut amongst other names, is a species of walnut. It is native to Eurasia in at least southwest and central Asia and southeast Europe, but its exact natural area is obscure due to its long history of cultivation. The species has numerous cultivars which produce the edible walnut consumed around the world and produced predominately in China. It is widely cultivated across temperate regions throughout the world including those of Eurasia, Australia, and the Americas. {'imagecaption': 'Mature walnut tree', 'status': 'LC', 'statussystem': 'IUCN3.1', 'genus': 'Juglans', 'parent': 'Juglans sect. Juglans', 'species': 'regia', 'authority': 'L.', 'rangemap': 'Juglans regia range.svg', 'rangemapcaption': 'Distribution map', 'synonymsref': '{{citation needed|reason|=|Always supply a suitable, single citation for synonym lists|date|=|November 2018}}', 'synonyms': "''J....</code> | <code>Mococa municipality in the state of São Paulo in Brazil Mococa Portuguese pronunciation: [moˈkɔkɐ] is a municipality in the state of São Paulo in Brazil. The population is 68,980 (2020 est.) in an area of 855 km². The elevation is 645 m. The local government is made up of a mayor (in Brazil, Prefeito) and a municipal council (in Brazil, Câmara Municipal.) The current mayor is Eduardo Ribeiro Barison, from the PSD, who was elected in the 2024 municipal elections. {'officialname': 'Mococa', 'imageskyline': 'Montagem fotográfica de Mococa.jpg', 'imagesize': '200px', 'imageflag': 'Flagmococa.jpg', 'imageseal': 'Brasaomococa.jpg', 'imagemap': 'SaoPaulo Municip Mococa.svg', 'mapsize': '200px', 'subdivisiontype': 'Country', 'subdivisiontype1': 'Region', 'subdivisiontype2': 'State', 'subdivisionname': 'Brazil', 'subdivisionname1': 'Southeast', 'subdivisionname2': 'São Paulo', 'areatotalkm2': '856.39', 'populationasof': '2013', 'populationtotal': '80,629', 'timezone': 'UTC-3', '...</code> | | <code>Hyundai N high-performance brand by Hyundai Hyundai N (Korean: 현대 N) is a sub-brand of high-performance cars, engines, and related technologies established in 2012 by Hyundai. Hyundai claims the "N" refers to two elements. First, the Namyang district in South Korea, home of Hyundai's Global Research & Development Center where the brand 'N' was founded; second, the over 20 km long Nordschleife northern loop of the Nürburgring racetrack in Germany, home to Hyundai's European Technical Center and where all the 'N' models are tested - and many other brands, too. The 'N' logo was inspired by the shape of chicanes in racing circuits. The first 'N'-branded vehicle produced was the i30 N, which debuted in 2016. {'name': 'Hyundai N', 'logo': 'Hyundai N Logo.png', 'producttype': '{{ubl |Performance engines and cars |Automotive sports accessories}}', 'currentowner': 'Hyundai Motor Group', 'producedby': 'Hyundai', 'introduced': '2016', 'markets': 'Worldwide', 'website': '[https://www.hyundai-n.com...</code> | <code>Anna Sui American fashion designer Anna Sui (Chinese: 蕭志美; pinyin: Xiāo Zhìměi; born August 4, 1964) is an American fashion designer. Her brand categories include several fashion lines, footwear, cosmetics, fragrances, eyewear, jewelry, accessories and home goods. Sui was named one of the "Top 5 Fashion Icons of the Decade", and in 2009 earned the Geoffrey Beene Lifetime Achievement Award from the Council of Fashion Designers of America (CFDA), joining the ranks of Yves Saint Laurent, Giorgio Armani, Ralph Lauren, and Diane von Furstenberg. {'caption': 'Sui in 2009', 'name': 'Anna Sui', 'birthdate': '{{Birth date and age|1964|8|4}}', 'birthplace': 'Detroit, Michigan, U.S.', 'education': 'Parsons School of Design', 'label_name': '{{plainlist|\n Anna Sui\n Dolly Girl by Anna Sui|ref| name="DollyGirl"|{{cite web |url=http://www.onward.co.jp/dollygirl/ |title=Dolly Girl by Anna Sui |date=2016 |website=onward.co.jp |publisher=Onward Kashiyama Co., Ltd. |access-date=October 6, 2016 |url-...</code> | <code>speechwriter person who writes speeches that will be delivered by another person A speechwriter is a person who is hired to prepare and write speeches to be delivered by another person. Speechwriters are employed by many senior-level elected officials and executives in the government and private sectors. They can also be employed to write for weddings and other social occasions. {'note': 'infobox not present in Wikipedia'} {'subclass of': 'writer', 'instance of': 'profession', 'ISCO-88 occupation class': '2451', 'ISCO-08 occupation class': '2641', 'different from': 'playwright'}</code> |
—Loss: <code>TripletLoss</code> with these parameters:

json

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 0.5
  }

Training Hyperparameters

Non-Default Hyperparameters

—num_train_epochs: 5
—fp16: True
—multi_dataset_batch_sampler: round_robin

All Hyperparameters

<details><summary>Click to expand</summary>

—overwrite_output_dir: False
—do_predict: False
—eval_strategy: no
—prediction_loss_only: True
—per_device_train_batch_size: 8
—per_device_eval_batch_size: 8
—per_gpu_train_batch_size: None
—per_gpu_eval_batch_size: None
—gradient_accumulation_steps: 1
—eval_accumulation_steps: None
—torch_empty_cache_steps: None
—learning_rate: 5e-05
—weight_decay: 0.0
—adam_beta1: 0.9
—adam_beta2: 0.999
—adam_epsilon: 1e-08
—max_grad_norm: 1
—num_train_epochs: 5
—max_steps: -1
—lr_scheduler_type: linear
—lr_scheduler_kwargs: {}
—warmup_ratio: 0.0
—warmup_steps: 0
—log_level: passive
—log_level_replica: warning
—log_on_each_node: True
—logging_nan_inf_filter: True
—save_safetensors: True
—save_on_each_node: False
—save_only_model: False
—restore_callback_states_from_checkpoint: False
—no_cuda: False
—use_cpu: False
—use_mps_device: False
—seed: 42
—data_seed: None
—jit_mode_eval: False
—use_ipex: False
—bf16: False
—fp16: True
—fp16_opt_level: O1
—half_precision_backend: auto
—bf16_full_eval: False
—fp16_full_eval: False
—tf32: None
—local_rank: 0
—ddp_backend: None
—tpu_num_cores: None
—tpu_metrics_debug: False
—debug: []
—dataloader_drop_last: False
—dataloader_num_workers: 0
—dataloader_prefetch_factor: None
—past_index: -1
—disable_tqdm: False
—remove_unused_columns: True
—label_names: None
—load_best_model_at_end: False
—ignore_data_skip: False
—fsdp: []
—fsdp_min_num_params: 0
—fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
—tp_size: 0
—fsdp_transformer_layer_cls_to_wrap: None
—accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
—deepspeed: None
—label_smoothing_factor: 0.0
—optim: adamw_torch
—optim_args: None
—adafactor: False
—group_by_length: False
—length_column_name: length
—ddp_find_unused_parameters: None
—ddp_bucket_cap_mb: None
—ddp_broadcast_buffers: False
—dataloader_pin_memory: True
—dataloader_persistent_workers: False
—skip_memory_metrics: True
—use_legacy_prediction_loop: False
—push_to_hub: False
—resume_from_checkpoint: None
—hub_model_id: None
—hub_strategy: every_save
—hub_private_repo: None
—hub_always_push: False
—gradient_checkpointing: False
—gradient_checkpointing_kwargs: None
—include_inputs_for_metrics: False
—include_for_metrics: []
—eval_do_concat_batches: True
—fp16_backend: auto
—push_to_hub_model_id: None
—push_to_hub_organization: None
—mp_parameters:
—auto_find_batch_size: False
—full_determinism: False
—torchdynamo: None
—ray_scope: last
—ddp_timeout: 1800
—torch_compile: False
—torch_compile_backend: None
—torch_compile_mode: None
—include_tokens_per_second: False
—include_num_input_tokens_seen: False
—neftune_noise_alpha: None
—optim_target_modules: None
—batch_eval_metrics: False
—eval_on_start: False
—use_liger_kernel: False
—eval_use_gather_object: False
—average_tokens_across_devices: False
—prompts: None
—batch_sampler: batch_sampler
—multi_dataset_batch_sampler: round_robin

</details>

Training Logs

Epoch	Step	Training Loss
0.6105	500	0.2708
1.2210	1000	0.1498
1.8315	1500	0.0962
2.4420	2000	0.0522
3.0525	2500	0.0383
3.6630	3000	0.0215
4.2735	3500	0.0173
4.8840	4000	0.0087

Framework Versions

—Python: 3.11.12
—Sentence Transformers: 3.4.1
—Transformers: 4.51.3
—PyTorch: 2.6.0+cu124
—Accelerate: 1.5.2
—Datasets: 3.5.0
—Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

bibtex

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->