Chandar/sv-subject-based-matryoshka-mpnet-base

SentenceTransformer based on microsoft/mpnet-base

This is a sentence-transformers model finetuned from microsoft/mpnet-base. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• —Model Type: Sentence Transformer
• —Base model: microsoft/mpnet-base <!-- at revision 6996ce1e91bd2a9c7d7f61daec37463394f73f09 -->
• —Maximum Sequence Length: 512 tokens
• —Output Dimensionality: 768 dimensions
• —Similarity Function: Cosine Similarity <!-- - Training Dataset: Unknown --> <!-- - Language: Unknown --> <!-- - License: Unknown -->

Model Sources

• —Documentation: Sentence Transformers Documentation
• —Repository: Sentence Transformers on GitHub
• —Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Chandar/sv-subject-based-matryoshka-mpnet-base")
# Run inference
sentences = [
    'channels. Cl- ions enter the cell and hyperpolarizes the membrane, making the neuron less likely to fire\nan action potential.\nOnce neurotransmission has occurred, the neurotransmitter must be removed from the synaptic\ncleft so the postsynaptic membrane can “reset” and be ready to receive another signal. This can be\naccomplished in three ways: the neurotransmitter can diffuse away from the synaptic cleft, it can be\ndegraded by enzymes in the synaptic cleft, or it can be recycled (sometimes called reuptake) by the\npresynaptic neuron. Several drugs act at this step of neurotransmission. For example, some drugs that\nare given to Alzheimer’s patients work by inhibiting acetylcholinesterase, the enzyme that degrades\nacetylcholine. This inhibition of the enzyme essentially increases neurotransmission at synapses that\nrelease acetylcholine. Once released, the acetylcholine stays in the cleft and can continually bind and\nunbind to postsynaptic receptors.\nNeurotransmitter Function and Location\nNeurotransmitter Example Location\nAcetylcholine — CNS and/or\nPNS\nBiogenic amine Dopamine, serotonin, norepinephrine CNS and/or\nPNS\nAmino acid Glycine, glutamate, aspartate, gamma aminobutyric\nacidCNS\nNeuropeptide Substance P, endorphins CNS and/or\nPNS\nTable 35.2\nElectrical Synapse\nWhile electrical synapses are fewer in number than chemical synapses, they are found in all nervous\nsystems and play important and unique roles. The mode of neurotransmission in electrical synapses is\nquite different from that in chemical synapses. In an electrical synapse, the presynaptic and postsynaptic\nmembranes are very close together and are actually physically connected by channel proteins forming\ngap junctions. Gap junctions allow current to pass directly from one cell to the next. In addition to the\nions that carry this current, other molecules, such as ATP, can diffuse through the large gap junction\npores.\nThere are key differences between chemical and electrical synapses. Because chemical synapses\ndepend on the release of neurotransmitter molecules from synaptic vesicles to pass on their signal, there\nis an approximately one millisecond delay between when the axon potential reaches the presynaptic\nterminal and when the neurotransmitter leads to opening of postsynaptic ion channels. Additionally, this\nsignaling is unidirectional. Signaling in electrical synapses, in contrast, is virtually instantaneous (which\nis important for synapses involved in key reflexes), and some electrical synapses are bidirectional.\nElectrical synapses are also more reliable as they are less likely to be blocked, and they are important\nfor synchronizing the electrical activity of a group of neurons. For example, electrical synapses in the\nthalamus are thought to regulate slow-wave sleep, and disruption of these synapses can cause seizures.\nSignal Summation\nSometimes a single EPSP is strong enough to induce an action potential in the postsynaptic neuron,\nbut often multiple presynaptic inputs must create EPSPs around the same time for the postsynaptic\nneuron to be sufficiently depolarized to fire an action potential. This process is calledsummation and\noccurs at the axon hillock, as illustrated inFigure 35.16. Additionally, one neuron often has inputs\nfrom many presynaptic neurons—some excitatory and some inhibitory—so IPSPs can cancel out EPSPs\nand vice versa. It is the net change in postsynaptic membrane voltage that determines whether the\npostsynaptic cell has reached its threshold of excitation needed to fire an action potential. Together,\nsynaptic summation and the threshold for excitation act as a filter so that random “noise” in the system\nis not transmitted as important information.\n1004 CHAPTER 35 | THE NERVOUS SYSTEM\nThis content is available for free at http://cnx.org/content/col11448/1.9',
    'WHERESELENIUMPOISONING\nIncertainportionsoftheNorthCentralgreatplains,plantsabsorbenoughsele-niumfromthesoiltoinjureanimalsthatfeeduponthem.Thepoisoningmayresultinaslowdiseaseknownas"blindstaggers"oras"alkalidisease",oritmaybequicklyfatal.Asaresultoftheselenium,thejointsoftheleg-bonesbecomebadlyeroded.Thehoofsdevelopabnormalitiesordropoff.Locomotionisimpaired.Theeffectoftheseleniumper-sists,fortheanimalsdonotusuallyrecoverevenifre-movedfromsucharegionandfedagoodrationOCCURSironisanessentialconstituentofthehemoglobin(seepage205).Coppercompoundsaregenerallypoisonoustomostkindsofprotoplasm;yetforsomespeciescopperisnecessaryinsmallamounts.Copperisanessentialelementinthebluishoxygen-carrierhemocyaninofthekingcrabandthelobster.InsomeoftheWesternstatesthesoilcontainstheelementselenium.Thiselementispresentalsoinplantsgrowinginsuchsoil,althoughitdoesnotappeartoaffecttheminanyway.Butanimalsthatfeeduponsuchplantsareoftenseriouslypoisoned(seeillustrationopposite).Inotherregionsvariationintheamountoffluorineinthesoilmaybeimportanttous.Theelementfluorine,whichisverywidelybutunevenlydistributed,seemstoplayaroleintheassimilationofcalciumandphosphorus,andsoaffectstheformationoftheteeth.Astudyof7000girlsandboysofhigh-schoolageinvariousmiddleandsouthwesternstatesbroughtoutthefactthattherewasmuchmoretoothdecay,orcaries^incommunitieswhosewatersupplieswerefreeoffluorinethanincommunitiesusingwaterwith0.5ormorepartsfluorinepermillionpartswater.Thus,thepopulationofacertainpartofTexas,DeafSmithCounty,wasfoundtohaveanexceptionallylownumberofdecayedteeth;andthisrelativefreedomfromdentalcariesisasso-ciatedwithmorethanusualamountsoffluorineinthelocalwaters.Inotherregionsunusualamountsoffluorineinthesoilandsoilwatersapparentlybringaboutthedevelopmentof"mottledteeth"amongthechildrenlivingthere.Nobodywantsblotchyteeth,butnobodywantscaries102',
    "INDEX 275\nSenescence\n1\n, 27-30,46,7078\ntheoiietiof,43f>0\nSenilechangesinneivecells,27-29\nSenilityat,causeofdeath,1011\ninplants,44,71,75\nSepiK.rniia,231,282\nSeitna,252,258\nMeium,influenceontissuecultme,\n70,77\nSe\\organs,107,IOS,111,121-125,\n217219\nSexualiepioduction,3741\nShell,J, 20,27\nSkeletalsyhtem,107,108,112,127,\n128\nskin,107,H)8,no,112,331,132\nSlonakei,,T I?,212,213,218,228,\n2U7\nSloiopolhki,B, 33,2(>7\nSnow,12 C,179-383,225,267\nSofteningofthe-hiain,231,2.12\nSoma,40\nSomaticcella,ranmntalityof,5878\nSpan,174,175\nSpiegelberg,W, 87\nSpmtuahsm,18-20\nSpleen,61\nSponges,62\nStatine,174,175\nStomach,M,217-210,207\nStenuntomum,35,36\nSievenflon,THC.,206-208,2(>7\nStillbirths,205\n$ ongyloocnt')otuspitrpmnIus,55,\n56\nHummaiyofreaultH,223-227\nf-hxi'vivorfiluplinesofDrosophilo,\n188,192,195\nSyphihH,123\nTable,life,70-82\nTempeiatuic,208-217\nTethehn,70,220222\nTbeoiiesofdeath,4350\nTheoiyofpopulationgiowtli,249\nThyioulgland,01\nTissuecnltmet Ditto,5878\nTianaplantationoftumois,04,(51\nTubeicuIomH,101,204,208,230,2!1,\n238\nTuiuoiti.uihpliinlation,61, 65\nTyphoidfcvei,230,2,11,2.J5,2,'iO\nUnitedStates,giowtliol,2502,12,\n254-257\nLi'iostylayiandis,72\nVanBuien,GII, 11J,2UO\nVariation,genetic,190\nVeneiwildiseases,123,124\nVeilmlHt,PF,249,2(57\nVerwoin,M,,44,207\nVienna,245,246\nVoumoll,217\nWallei,AD,216,207\nWalwoith,BH,152,267\nWar,243\nWedekmd,33,267\nWeiimann,A, 26,43,65,207\nWlialc,longevityof,22\nWilaon,HV,62,267\nWittalom,99\nWoodruff,LL,30,33,72,73,267,\n268\nWoodn,FA, 38,3,208\nYellowfever,240,21-2\nYoung,TTO,,23-25,268",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

Evaluation

Metrics

Binary Classification

• —Evaluated with <code>BinaryClassificationEvaluator</code> with these parameters:

  {
      "truncate_dim": 768
  }

Binary Classification

• —Evaluated with <code>BinaryClassificationEvaluator</code> with these parameters:

  {
      "truncate_dim": 512
  }

Binary Classification

• —Evaluated with <code>BinaryClassificationEvaluator</code> with these parameters:

  {
      "truncate_dim": 256
  }

Binary Classification

• —Evaluated with <code>BinaryClassificationEvaluator</code> with these parameters:

  {
      "truncate_dim": 128
  }

Binary Classification

• —Evaluated with <code>BinaryClassificationEvaluator</code> with these parameters:

  {
      "truncate_dim": 64
  }

Binary Classification

• —Evaluated with <code>BinaryClassificationEvaluator</code> with these parameters:

  {
      "truncate_dim": 32
  }

Binary Classification

• —Evaluated with <code>BinaryClassificationEvaluator</code> with these parameters:

  {
      "truncate_dim": 16
  }

Binary Classification

• —Evaluated with <code>BinaryClassificationEvaluator</code> with these parameters:

  {
      "truncate_dim": 8
  }

Binary Classification

• —Evaluated with <code>BinaryClassificationEvaluator</code> with these parameters:

  {
      "truncate_dim": 4
  }

Binary Classification

• —Evaluated with <code>BinaryClassificationEvaluator</code> with these parameters:

  {
      "truncate_dim": 2
  }

Binary Classification

• —Evaluated with <code>BinaryClassificationEvaluator</code> with these parameters:

  {
      "truncate_dim": 1
  }

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

Unnamed Dataset

• —Size: 1,124,250 training samples
• —Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>label</code>
• —Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | label | |:--------|:-------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|:------------------------------------------------| | type | string | string | int | | details | <ul><li>min: 512 tokens</li><li>mean: 512.0 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 23 tokens</li><li>mean: 432.19 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>0: ~50.10%</li><li>1: ~49.90%</li></ul> |
• —Samples: | sentence1 | sentence2 | label | |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------| | <code>channels. Cl- ions enter the cell and hyperpolarizes the membrane, making the neuron less likely to fire<br>an action potential.<br>Once neurotransmission has occurred, the neurotransmitter must be removed from the synaptic<br>cleft so the postsynaptic membrane can “reset” and be ready to receive another signal. This can be<br>accomplished in three ways: the neurotransmitter can diffuse away from the synaptic cleft, it can be<br>degraded by enzymes in the synaptic cleft, or it can be recycled (sometimes called reuptake) by the<br>presynaptic neuron. Several drugs act at this step of neurotransmission. For example, some drugs that<br>are given to Alzheimer’s patients work by inhibiting acetylcholinesterase, the enzyme that degrades<br>acetylcholine. This inhibition of the enzyme essentially increases neurotransmission at synapses that<br>release acetylcholine. Once released, the acetylcholine stays in the cleft and can continually bind and<br>unbind to postsynaptic receptors.<br>Neurotransmitter Function and Location<br>N...</code> | <code>Figure 25.6 This table shows the major divisions of green plants.<br>Which of the following statements about plant divisions is false?<br>a. Lycophytes and pterophytes are seedless vascular plants.<br>b. All vascular plants produce seeds.<br>c. All nonvascular embryophytes are bryophytes.<br>d. Seed plants include angiosperms and gymnosperms.<br>25.2 | Green Algae: Precursors of Land Plants<br>By the end of this section, you will be able to:<br>• Describe the traits shared by green algae and land plants<br>• Explain the reasons why Charales are considered the closest relative to land plants<br>• Understand that current phylogenetic relationships are reshaped by comparative analysis of<br>DNA sequences<br>Streptophytes<br>Until recently, all photosynthetic eukaryotes were considered members of the kingdom Plantae. The<br>brown, red, and gold algae, however, have been reassigned to the Protista kingdom. This is because<br>apart from their ability to capture light energy and fix CO2, they lack many structural and biochemical<br>traits ...</code> | <code>1</code> | | <code>channels. Cl- ions enter the cell and hyperpolarizes the membrane, making the neuron less likely to fire<br>an action potential.<br>Once neurotransmission has occurred, the neurotransmitter must be removed from the synaptic<br>cleft so the postsynaptic membrane can “reset” and be ready to receive another signal. This can be<br>accomplished in three ways: the neurotransmitter can diffuse away from the synaptic cleft, it can be<br>degraded by enzymes in the synaptic cleft, or it can be recycled (sometimes called reuptake) by the<br>presynaptic neuron. Several drugs act at this step of neurotransmission. For example, some drugs that<br>are given to Alzheimer’s patients work by inhibiting acetylcholinesterase, the enzyme that degrades<br>acetylcholine. This inhibition of the enzyme essentially increases neurotransmission at synapses that<br>release acetylcholine. Once released, the acetylcholine stays in the cleft and can continually bind and<br>unbind to postsynaptic receptors.<br>Neurotransmitter Function and Location<br>N...</code> | <code>NATURALDEATH,PUBLICHEALTH229<br>andpublichealthhave,byallodds,thebestcasewhen<br>measuredintermsofaccomplishment.Man'sexpecta-<br>tionoflifehasincreasedashehascomedownthrough<br>thecenturies(cf.PearsonandMacdonell.)Alarge<br>partofthisimprovementmustsurelybecreditedtohis<br>improvedunderstandingofhowtocopewithanalways<br>moreorlessinimicalenvironmentandassuageitsasper-<br>itiestohisgreatercomfortandwell-being.Tqfailto<br>givethiscreditwouldbemanifestlyabsurd.<br>Butitwouldbeequallyabsurdtoattempttomain-<br>tainthatalldeclineinthedeath-ratewhichhasoccurred<br>liasbeenduetotheeffortsofhealthofficials,whether<br>consciousorunconscious,asisoftenassertedandstill<br>moreoftenimpliedintheimpassionedoutpouringsof<br>zealouspropagandists.Theopen-mindedstudentofthe<br>naturalhistoryofdiseaseknqwsperfectlywellthata<br>largepartoftheimprovementintherateofmortality<br>cannotpossiblyhavebeenduetoanysuchefforts.To<br>illustratethepoint,Ihavepreparedaseriesofillustra-<br>tionsdealingwithconditionsintheEegistrationArea<br>oftheUnitedStatesintheimmediatepast.Allthe...</code> | <code>1</code> | | <code>channels. Cl- ions enter the cell and hyperpolarizes the membrane, making the neuron less likely to fire<br>an action potential.<br>Once neurotransmission has occurred, the neurotransmitter must be removed from the synaptic<br>cleft so the postsynaptic membrane can “reset” and be ready to receive another signal. This can be<br>accomplished in three ways: the neurotransmitter can diffuse away from the synaptic cleft, it can be<br>degraded by enzymes in the synaptic cleft, or it can be recycled (sometimes called reuptake) by the<br>presynaptic neuron. Several drugs act at this step of neurotransmission. For example, some drugs that<br>are given to Alzheimer’s patients work by inhibiting acetylcholinesterase, the enzyme that degrades<br>acetylcholine. This inhibition of the enzyme essentially increases neurotransmission at synapses that<br>release acetylcholine. Once released, the acetylcholine stays in the cleft and can continually bind and<br>unbind to postsynaptic receptors.<br>Neurotransmitter Function and Location<br>N...</code> | <code>through a mass which is now formed out of sugar and is now dissolved again<br>into sugar. For it appears that this soluble form of carbo-hydrate, taken into the<br>liver from the intestine, is there, when not immediately needed, stored up in the<br>form of glycogen, ready to be re-dissolved and carried into the system either for<br>immediate use or for re-deposit as glycogen at the places where it is presently to<br>be consumed: the great deposit in the liver and the minor deposits in the muscles<br>being, to use the simile of Prof. Michael Foster, analogous in their functions to a<br>central bank and branch banks.<br>An instructive parallelism may be noted between these processes carried on in<br>the animal organism and those carried on in the vegetal organism. For the carbo-<br>hydrates named, easily made to assume the soluble or the insoluble form by the<br>addition or subtraction of a molecule of water, and thus fitted sometimes for<br>distribution and sometimes for accumulation, are similarly dealt with in the two<br>c...</code> | <code>1</code> |
• —Loss: <code>MatryoshkaLoss</code> with these parameters:

  {
      "loss": "CoSENTLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64,
          32,
          16,
          8,
          4,
          2,
          1
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

Training Hyperparameters

Non-Default Hyperparameters

• —per_device_train_batch_size: 16
• —per_device_eval_batch_size: 32
• —learning_rate: 2e-05
• —weight_decay: 0.01
• —max_steps: 2000

All Hyperparameters

<details><summary>Click to expand</summary>

• —overwrite_output_dir: False
• —do_predict: False
• —eval_strategy: no
• —prediction_loss_only: True
• —per_device_train_batch_size: 16
• —per_device_eval_batch_size: 32
• —per_gpu_train_batch_size: None
• —per_gpu_eval_batch_size: None
• —gradient_accumulation_steps: 1
• —eval_accumulation_steps: None
• —torch_empty_cache_steps: None
• —learning_rate: 2e-05
• —weight_decay: 0.01
• —adam_beta1: 0.9
• —adam_beta2: 0.999
• —adam_epsilon: 1e-08
• —max_grad_norm: 1.0
• —num_train_epochs: 3.0
• —max_steps: 2000
• —lr_scheduler_type: linear
• —lr_scheduler_kwargs: {}
• —warmup_ratio: 0.0
• —warmup_steps: 0
• —log_level: passive
• —log_level_replica: warning
• —log_on_each_node: True
• —logging_nan_inf_filter: True
• —save_safetensors: True
• —save_on_each_node: False
• —save_only_model: False
• —restore_callback_states_from_checkpoint: False
• —no_cuda: False
• —use_cpu: False
• —use_mps_device: False
• —seed: 42
• —data_seed: None
• —jit_mode_eval: False
• —use_ipex: False
• —bf16: False
• —fp16: False
• —fp16_opt_level: O1
• —half_precision_backend: auto
• —bf16_full_eval: False
• —fp16_full_eval: False
• —tf32: None
• —local_rank: 0
• —ddp_backend: None
• —tpu_num_cores: None
• —tpu_metrics_debug: False
• —debug: []
• —dataloader_drop_last: False
• —dataloader_num_workers: 0
• —dataloader_prefetch_factor: None
• —past_index: -1
• —disable_tqdm: False
• —remove_unused_columns: True
• —label_names: None
• —load_best_model_at_end: False
• —ignore_data_skip: False
• —fsdp: []
• —fsdp_min_num_params: 0
• —fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
• —fsdp_transformer_layer_cls_to_wrap: None
• —accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
• —deepspeed: None
• —label_smoothing_factor: 0.0
• —optim: adamw_torch
• —optim_args: None
• —adafactor: False
• —group_by_length: False
• —length_column_name: length
• —ddp_find_unused_parameters: None
• —ddp_bucket_cap_mb: None
• —ddp_broadcast_buffers: False
• —dataloader_pin_memory: True
• —dataloader_persistent_workers: False
• —skip_memory_metrics: True
• —use_legacy_prediction_loop: False
• —push_to_hub: False
• —resume_from_checkpoint: None
• —hub_model_id: None
• —hub_strategy: every_save
• —hub_private_repo: None
• —hub_always_push: False
• —hub_revision: None
• —gradient_checkpointing: False
• —gradient_checkpointing_kwargs: None
• —include_inputs_for_metrics: False
• —include_for_metrics: []
• —eval_do_concat_batches: True
• —fp16_backend: auto
• —push_to_hub_model_id: None
• —push_to_hub_organization: None
• —mp_parameters:
• —auto_find_batch_size: False
• —full_determinism: False
• —torchdynamo: None
• —ray_scope: last
• —ddp_timeout: 1800
• —torch_compile: False
• —torch_compile_backend: None
• —torch_compile_mode: None
• —include_tokens_per_second: False
• —include_num_input_tokens_seen: False
• —neftune_noise_alpha: None
• —optim_target_modules: None
• —batch_eval_metrics: False
• —eval_on_start: False
• —use_liger_kernel: False
• —liger_kernel_config: None
• —eval_use_gather_object: False
• —average_tokens_across_devices: False
• —prompts: None
• —batch_sampler: batch_sampler
• —multi_dataset_batch_sampler: proportional

</details>

Training Logs

Framework Versions

• —Python: 3.12.9
• —Sentence Transformers: 4.1.0
• —Transformers: 4.53.0
• —PyTorch: 2.7.1
• —Accelerate: 1.8.1
• —Datasets: 3.6.0
• —Tokenizers: 0.21.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

CoSENTLoss

@online{kexuefm-8847,
    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
    author={Su Jianlin},
    year={2022},
    month={Jan},
    url={https://kexue.fm/archives/8847},
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->