mariakrissmer/alias_demo_model

SentenceTransformer based on NeuML/pubmedbert-base-embeddings

This is a sentence-transformers model finetuned from NeuML/pubmedbert-base-embeddings. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• —Model Type: Sentence Transformer
• —Base model: NeuML/pubmedbert-base-embeddings <!-- at revision d6eaca8254bc229f3ca42749a5510ae287eb3486 -->
• —Maximum Sequence Length: 512 tokens
• —Output Dimensionality: 768 dimensions
• —Similarity Function: Cosine Similarity <!-- - Training Dataset: Unknown --> <!-- - Language: Unknown --> <!-- - License: Unknown -->

Model Sources

• —Documentation: Sentence Transformers Documentation
• —Repository: Sentence Transformers on GitHub
• —Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("mariakrissmer/alias_demo_model")
# Run inference
sentences = [
    'FTL, FTH1, TMSB4X, B2M, MALAT1, LYZ, ACTB, TMSB10, RHOG, S100A11, TYROBP, S100A6, CST3, EEF1A1, PFN1, AIF1, CFL1, TPT1, CD52, S100A4, SH3BGRL3, HLA-DRA, UBA52, LILRB2, FCER1G, CD74, FCN1, PSAP, CYBA, PTMA, DUSP1, GNB2L1, SAT1, COTL1, OAZ1, VIM, H3F3B, HLA-DPA1, MYL6, SRSF5, NPC2, ZFP36, HLA-B, NACA, EIF1, ACTG1, LGALS1, CTSS, ARPC2, HLA-E expression pattern defines this as a CD14+ Monocytes cell.',
    'The expression of FTL, FTH1, S100A9, TMSB4X, TMSB10, S100A4, B2M, S100A8, LYZ, EEF1A1, ACTB, FOS, CTSS, EIF1, OAZ1, S100A11, S100A6, TKT, CD74, GNB2L1, MALAT1, TPT1, CYBA, JUNB, TYROBP, TYMP, FCER1G, PTMA, HLA-C, LST1, PPDPF, IER2, SH3BGRL3, LAPTM5, TXNIP, ID2, GPX1, GPSM3, LAMTOR4, SAT1, KLF6, VIM, PSAP, GAPDH, ARHGDIB, ALDOA, PFN1, ASGR1, CD68, CST3 aligns with a CD14+ Monocytes identity.',
    'A transcriptome with MALAT1, B2M, TMSB4X, RARS, TPT1, ARPC5, EEF1B2, EEF1A1, H3F3B, MYL12B, PTMA, UBE2D3, ACTB, STK17A, GIMAP7, HNRNPK, CFL1, ARHGDIB, EIF1, PSMA7, UBA52, AL592183.1, TRAF3IP3, SRSF7, CALM2, WIPF1, UBE2E3, CNBP, LAP3, FAM175A, H2AFZ, OSTC, CCDC109B, COX7C, DUSP1, HLA-E, HLA-C, PIM1, CYCS, PPIA, SLC25A6, RBM3, EEF1D, PDCD4-AS1, FAU, ATP5L, PFDN5, HNRNPA1, BTG1, CALM1 points toward CD4 T cells identity.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9911, 0.3080],
#         [0.9911, 1.0000, 0.3525],
#         [0.3080, 0.3525, 1.0000]])

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

Evaluation

Metrics

Triplet

• —Dataset: triplet_eval_scrna
• —Evaluated with <code>TripletEvaluator</code>

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

Unnamed Dataset

• —Size: 10,676 training samples
• —Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>negative</code>
• —Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | negative | |:--------|:--------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------| | type | string | string | string | | details | <ul><li>min: 164 tokens</li><li>mean: 181.83 tokens</li><li>max: 204 tokens</li></ul> | <ul><li>min: 164 tokens</li><li>mean: 181.5 tokens</li><li>max: 204 tokens</li></ul> | <ul><li>min: 164 tokens</li><li>mean: 181.71 tokens</li><li>max: 204 tokens</li></ul> |
• —Samples: | sentence1 | sentence2 | negative | |:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>Expression of FTL, TMSB4X, FTH1, B2M, ACTB, MALAT1, CCT7, COTL1, SPG7, OAZ1, RAB4B, CST3, AIF1, S100A4, PFN1, LGALS1, LST1, EEF1A1, CYBA, TMSB10, FAU, NACA, GNB2L1, HLA-DRA, HLA-DPA1, VIM, S100A6, PTMA, TKT, IFITM3, LYZ, SAT1, SH3BGRL3, FCER1G, TIMP1, LAS1L, ARPC3, TYROBP, HLA-C, FCN1, IFITM2, GAPDH, TPT1, CD52, YBX1, FCGR3A, CD74, PABPC1, STXBP2, HLA-B suggests FCGR3A+ Monocytes lineage commitment.</code> | <code>The transcriptome suggests a FCGR3A+ Monocytes type, with expression of B2M, FTH1, TMSB4X, FTL, ACTB, MALAT1, JUNB, TNFSF10, LAMTOR4, HDAC5, LYZ, OAZ1, TYROBP, CTSS, FCGR3A, AIF1, FCER1G, TMSB10, HLA-C, IFITM2, SAT1, CST3, EEF1A1, NACA, PFN1, CD74, GNB2L1, HLA-DPA1, NCKAP1L, COTL1, EIF1, ARPC2, LST1, ARHGDIB, SH3BGRL3, PTMA, SERPINA1, CYBA, ACTG1, EMP3, CD52, NCF2, HLA-A, CHCHD2, ARPC1B, EEF1D, PFDN5, HNRNPA1, ARPC3, ALDOA.</code> | <code>CD4 T cells cells are known to express: MALAT1, TMSB4X, B2M, EEF1A1, JUN, JUNB, HLA-A, TMSB10, ACTB, TPT1, FAU, CXCR4, EEF1D, H3F3B, ZFP36L2, TMA7, HLA-C, HNRNPA1, PFN1, EIF1, FTL, TXNIP, DUSP1, GNB2L1, ARHGDIB, PFDN5, FOS, SRP14, MYL12A, EEF2, EIF3K, ZFP36, CD52, LAPTM5, S100A4, CD48, ARPC2, ARL6IP5, COX7C, HNRNPA0, HLA-B, LTB, ANXA1, ATP6V1G1, VIM, LDHB, MYL6, BTG1, ARL6IP4, IL32.</code> | | <code>With genes like ACTB, TMSB4X, B2M, GAPDH, PTMA, ABRACL, TMSB10, PPP1CA, ACTG1, CFL1, THOC7, RARRES3, TUBA1B, PFN1, EEF1A1, H2AFZ, HNRNPA2B1, HMGB1, HNRNPA1, RAN, NPM1, PPIA, CORO1A, SRRM1, LDHA, TPI1, NACA, HSP90AA1, EIF4A1, ENO1, TUBB, CHCHD2, FTH1, ARHGDIB, MYL6, COTL1, EIF4A3, YBX1, HMGB2, VIM, FAU, MALAT1, ATP5G2, CALM1, COX4I1, FTL, ACTR3, CD74, GNB2L1, HLA-C active, this cell is identified as a CD8 T cells.</code> | <code>Observed top genes: MALAT1, TMSB4X, B2M, JUNB, PGK1, ACTB, LTB, TPT1, EIF1, MX2, S100A4, PTMA, ACTG1, S100A6, EEF1A1, MYL12A, UBA52, NPM1, HLA-C, CALM1, CD52, TXNIP, ID2, DUSP1, GNB2L1, HLA-A, CD99, VIM, FAU, CFL1, HSPA8, NACA, SLC25A3, FOS, IL32, PFN1, EIF3K, GLTSCR2, FTL, CD2, HCLS1, PLAC8, GZMK, GRK6, HLA-F, CITED2, ARPC1B, IL2RG, HNRNPK, CASP4.</code> | <code>A cell that expresses the following genes: FTL, FTH1, TMSB4X, LYZ, S100A4, MALAT1, S100A9, PTMA, TMSB10, VIM, PFN1, FAU, GAPDH, TPT1, S100A8, S100A6, B2M, EIF1, LGALS2, LGALS1, CTSS, S100A10, S100A11, CD74, DUSP1, NACA, CST3, OAZ1, TYROBP, SH3BGRL3, EEF1B2, GNB2L1, HLA-B, LST1, AIF1, EEF1A1, ACTB, FOS, H3F3B, UBA52, ISG15, LAPTM5, FCER1G, RAC1, NCF1, PABPC1, KLF6, CFL1, PFDN5, MT2A is likely a CD14+ Monocytes cell.</code> | | <code>This cell likely originates from the CD4 T cells family, based on expression of MALAT1, B2M, TMSB4X, TPT1, EEF1A1, FAU, TMSB10, JUNB, PTMA, HLA-C, EEF1D, GNB2L1, ABRACL, ACTB, PABPC1, DUSP1, FTH1, NACA, HLA-E, BTG1, HLA-B, TOMM7, CFL1, FOS, PFN1, EIF1, DDX5, SH3BGRL3, ARPC5, EEF1B2, PLAC8, ANAPC16, LSP1, HNRNPA1, HMGB1, IL32, COX4I1, CCR7, LIMD2, FTL, RBBP4, CXCR4, CCNI, COX7C, HLA-A, LTB, SRSF3, NDUFA1, SPOCK2, EIF3F.</code> | <code>MALAT1, ALDOA, TMSB4X, B2M, EEF1A1, ACTB, TPT1, TMSB10, FTH1, HLA-C, C6orf48, FOS, UBA52, TAGLN2, PTMA, GNB2L1, BTG1, EIF1, H3F3B, FTL, CD52, CXCR4, TMEM66, VIM, NACA, CIRBP, JUN, S100A10, EEF1B2, NPM1, HLA-A, UCP2, TMEM123, LDHB, HNRNPA1, GLTSCR2, SUN2, SH3BGRL3, ZCCHC11, DUSP1, HLA-E, HLA-B, EIF3F, CD44, ARHGDIB, ARPC3, SRP14, IL32, UBB, DDX5 define the expression landscape of this cell.</code> | <code>MALAT1, B2M, ACTB, TMSB4X, SUV420H2, HLA-C, NKG7, COMMD10, CCL5, HLA-A, HLA-B, TMSB10, GZMB, PFN1, TXNIP, GNLY, RFNG, ARPC2, MSN, FKBP11, COMMD6, UBB, EIF1, H3F3B, ACTG1, PPDPF, FTL, RAC2, S100A4, FCGR3A, FGFBP2, GZMA, HLA-E, CLIC1, EEF1D, TYROBP, CNBP, SPON2, AGA, BTF3, NDUFA4, GIMAP7, IFITM2, HNRNPA1, MYL6, UBC, TPT1, SERF2, GPATCH8, CD7 reflect the unique expression profile of NK cells cells.</code> |
• —Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }

Evaluation Dataset

Unnamed Dataset

• —Size: 1,187 evaluation samples
• —Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>negative</code>
• —Approximate statistics based on the first 1000 samples: | | sentence1 | sentence2 | negative | |:--------|:--------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------| | type | string | string | string | | details | <ul><li>min: 164 tokens</li><li>mean: 181.73 tokens</li><li>max: 202 tokens</li></ul> | <ul><li>min: 164 tokens</li><li>mean: 181.64 tokens</li><li>max: 205 tokens</li></ul> | <ul><li>min: 164 tokens</li><li>mean: 181.52 tokens</li><li>max: 205 tokens</li></ul> |
• —Samples: | sentence1 | sentence2 | negative | |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>B2M, MALAT1, SCAPER, TMSB4X, IFI35, GLTSCR2, JUNB, TMSB10, NAA20, PMAIP1, JUN, PTMA, HLA-C, ACTB, H3F3B, S100A4, EEF1A1, BTG1, IL32, DUSP1, HLA-A, HLA-B, VIM, FTH1, FAU, NACA, CD52, EEF1D, HNRNPA1, FOS, UBA52, FTL, HLA-E, ARHGDIB, EIF1, SELL, SRSF7, ARPC2, SP110, LTB, PPIA, LINC-PINT, VPS28, ANXA1, PFDN5, UBC, HMGB1, DAD1, SRSF5, CALM1 are the top expressed genes in this cell.</code> | <code>Cells expressing MALAT1, B2M, TMSB4X, JUNB, EEF1A1, TMSB10, PTMA, FTH1, TPT1, EIF1, HLA-A, NACA, FOS, JUN, HNRNPA1, FTL, ID2, DUSP1, HLA-C, PABPC1, UBC, SRSF5, KLF2, GNB2L1, HLA-B, TMEM66, FAU, GLTSCR2, TRAF3IP3, ZFP36L2, EEF1B2, BTF3, ACTB, EEF1D, CFL1, SERF2, IL32, CD52, CD53, CXCR4, NCL, NPM1, LTB, BRD2, PNRC1, RAC1, TSC22D3, ATP6V1G1, EIF4G2, NUCB2 often belong to the CD4 T cells lineage.</code> | <code>This transcriptomic profile — with genes like B2M, MALAT1, TMSB4X, ACTB, SNRPE, CEBPD, HLA-A, HLA-C, EEF1D, EEF1A1, CCL5, EIF1, CD52, RGS2, HLA-B, KLF6, NACA, TPT1, PFN1, PTMA, NPM1, TOMM7, IL2RG, ATP5G2, H3F3B, UBA52, S100A4, DUSP1, PABPC1, FAU, CFL1, UBC, FOS, PPIB, ACTG1, JUNB, S100A6, TRAF3IP3, GUK1, OST4, EIF4A2, HOPX, IL7R, GZMK, GNB2L1, LTB, GIMAP1, CCDC107, PRF1, FTH1 — resembles that of a CD8 T cells.</code> | | <code>Observed top genes: MALAT1, B2M, TMSB4X, TMSB10, YWHAB, EEF1A1, IL32, NKG7, GZMH, H3F3B, PPDPF, SH3BGRL3, S100A6, EEF1B2, GNB2L1, ACTB, FTH1, NACA, CCL5, EEF2, FXYD5, TMEM50A, CD52, LAPTM5, CD53, S100A10, DCAF8, UFC1, TRAF3IP3, IMMT, GCC2, ARPC2, PTMA, HINT1, SQSTM1, HLA-C, HLA-B, RBM3, SLC25A5, EEF1D, SRGN, IFITM1, TPT1, SRSF5, FOS, CALM1, HMOX2, CIRBP, CLPP, PRMT2.</code> | <code>This cell likely originates from the CD8 T cells family, based on expression of MALAT1, B2M, TMSB4X, RHOG, NKG7, HLA-C, CCL5, ACTB, HLA-A, HLA-B, TXNIP, TMSB10, EEF1A1, PTPRCAP, EIF1, PFN1, S100A4, FTH1, CFL1, CTSW, UCP2, NACA, SH3BGRL3, PTMA, CLIC1, HLA-DPB1, PRF1, FAU, ARHGDIB, TPT1, UBB, MYL12A, CST7, PPDPF, SSR2, IAH1, ARPC2, HLA-E, SLC25A6, CD3D, TPI1, UBC, CALM1, IDH2, H3F3B, C19orf43, FTL, CRELD2, CD52, S100A6.</code> | <code>CD4 T cells cells typically express genes such as: MALAT1, B2M, TMSB4X, TMSB10, EEF1A1, PTMA, FOS, JUN, ACTB, JUNB, CEBPB, TPT1, GCH1, EIF1, HLA-C, FAU, CD52, LPIN1, HLA-A, GNB2L1, FTH1, CFL1, BTG1, UBA52, CXCR4, NACA, PFN1, GLTSCR2, FTL, S100A4, ZFP36L2, BTF3, PABPC1, CD3D, LDHB, ACTG1, DNAJB1, YBX1, SERBP1, S100A6, CD247, EEF1B2, SATB1, TRAT1, CNBP, LTB, TMEM66, EEF1D, HSPA8, SLC2A3.</code> | | <code>This cell shows high expression of MALAT1, TMSB4X, B2M, EIF4A2, TPT1, TBCC, EEF1A1, PTMA, FOS, CHTF8, GNB2L1, HLA-C, FAU, NACA, JUNB, TMSB10, LTB, ACTB, EEF1D, DUSP1, HNRNPA1, GLTSCR2, JUN, TSC22D3, BTG1, UBC, HLA-B, CFL1, HLA-A, GAPDH, LDHB, EIF1, UBA52, EEF1B2, COX7C, HNRNPA2B1, PPIA, CD3D, NAP1L1, SRSF5, SERF2, H3F3B, ACTG1, ENO1, SH3BGRL3, MCL1, ZFP36L2, MGAT4A, NBEAL1, ARPC2, suggesting it is a CD4 T cells.</code> | <code>The genes MALAT1, TMSB4X, B2M, EEF1A1, JUNB, TMSB10, HLA-C, FTL, ACTB, UBA52, NPM1, GNB2L1, HLA-A, PTPRCAP, NACA, EIF1, PTMA, FAU, HNRNPA1, TPT1, FOS, ZFP36L2, HLA-E, EEF1D, IL32, IL7R, VIM, ATP5L, LDHB, MYL6, SRSF11, TXNIP, S100A6, BTG2, CXCR4, TUBA4A, DUSP1, HLA-B, C6orf48, SYPL1, TMEM66, FTH1, HSPA8, ARHGDIB, NAP1L1, BTG1, COMMD6, PSME1, CIB1, EIF4A1 are expressed in this cell, which is classified as a CD4 T cells.</code> | <code>Consistent with B cells function, genes like CD74, MALAT1, TMSB4X, B2M, HLA-DRA, SBDS, HLA-DPB1, PTMA, HLA-DRB1, LAPTM5, JUN, HLA-C, EEF1A1, FTH1, FOS, JUNB, HLA-DQA1, CFL1, UBE2I, OAZ1, CD79A, FTL, DUSP1, GNB2L1, HLA-B, ACTB, EEF1D, CIB1, EIF1, ARPC2, HLA-E, SLC25A6, MS4A1, ISCU, UBB, CD37, SH3BGRL3, HLA-A, HLA-DRB5, HLA-DPA1, HSPB1, FAU, NEAT1, PFN1, DDX5, H3F3B, ACTG1, UBA52, CD52, TMSB10 are expressed.</code> |
• —Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }

Training Hyperparameters

Non-Default Hyperparameters

• —eval_strategy: steps
• —per_device_train_batch_size: 32
• —num_train_epochs: 2
• —warmup_steps: 100

All Hyperparameters

<details><summary>Click to expand</summary>

• —overwrite_output_dir: False
• —do_predict: False
• —eval_strategy: steps
• —prediction_loss_only: True
• —per_device_train_batch_size: 32
• —per_device_eval_batch_size: 8
• —per_gpu_train_batch_size: None
• —per_gpu_eval_batch_size: None
• —gradient_accumulation_steps: 1
• —eval_accumulation_steps: None
• —torch_empty_cache_steps: None
• —learning_rate: 5e-05
• —weight_decay: 0.0
• —adam_beta1: 0.9
• —adam_beta2: 0.999
• —adam_epsilon: 1e-08
• —max_grad_norm: 1.0
• —num_train_epochs: 2
• —max_steps: -1
• —lr_scheduler_type: linear
• —lr_scheduler_kwargs: {}
• —warmup_ratio: 0.0
• —warmup_steps: 100
• —log_level: passive
• —log_level_replica: warning
• —log_on_each_node: True
• —logging_nan_inf_filter: True
• —save_safetensors: True
• —save_on_each_node: False
• —save_only_model: False
• —restore_callback_states_from_checkpoint: False
• —no_cuda: False
• —use_cpu: False
• —use_mps_device: False
• —seed: 42
• —data_seed: None
• —jit_mode_eval: False
• —bf16: False
• —fp16: False
• —fp16_opt_level: O1
• —half_precision_backend: auto
• —bf16_full_eval: False
• —fp16_full_eval: False
• —tf32: None
• —local_rank: 0
• —ddp_backend: None
• —tpu_num_cores: None
• —tpu_metrics_debug: False
• —debug: []
• —dataloader_drop_last: False
• —dataloader_num_workers: 0
• —dataloader_prefetch_factor: None
• —past_index: -1
• —disable_tqdm: False
• —remove_unused_columns: True
• —label_names: None
• —load_best_model_at_end: False
• —ignore_data_skip: False
• —fsdp: []
• —fsdp_min_num_params: 0
• —fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
• —fsdp_transformer_layer_cls_to_wrap: None
• —accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
• —parallelism_config: None
• —deepspeed: None
• —label_smoothing_factor: 0.0
• —optim: adamwtorchfused
• —optim_args: None
• —adafactor: False
• —group_by_length: False
• —length_column_name: length
• —project: huggingface
• —trackio_space_id: trackio
• —ddp_find_unused_parameters: None
• —ddp_bucket_cap_mb: None
• —ddp_broadcast_buffers: False
• —dataloader_pin_memory: True
• —dataloader_persistent_workers: False
• —skip_memory_metrics: True
• —use_legacy_prediction_loop: False
• —push_to_hub: False
• —resume_from_checkpoint: None
• —hub_model_id: None
• —hub_strategy: every_save
• —hub_private_repo: None
• —hub_always_push: False
• —hub_revision: None
• —gradient_checkpointing: False
• —gradient_checkpointing_kwargs: None
• —include_inputs_for_metrics: False
• —include_for_metrics: []
• —eval_do_concat_batches: True
• —fp16_backend: auto
• —push_to_hub_model_id: None
• —push_to_hub_organization: None
• —mp_parameters:
• —auto_find_batch_size: False
• —full_determinism: False
• —torchdynamo: None
• —ray_scope: last
• —ddp_timeout: 1800
• —torch_compile: False
• —torch_compile_backend: None
• —torch_compile_mode: None
• —include_tokens_per_second: False
• —include_num_input_tokens_seen: no
• —neftune_noise_alpha: None
• —optim_target_modules: None
• —batch_eval_metrics: False
• —eval_on_start: False
• —use_liger_kernel: False
• —liger_kernel_config: None
• —eval_use_gather_object: False
• —average_tokens_across_devices: True
• —prompts: None
• —batch_sampler: batch_sampler
• —multi_dataset_batch_sampler: proportional
• —router_mapping: {}
• —learning_rate_mapping: {}

</details>

Training Logs

Framework Versions

• —Python: 3.12.12
• —Sentence Transformers: 5.1.2
• —Transformers: 4.57.1
• —PyTorch: 2.9.1
• —Accelerate: 1.11.0
• —Datasets: 4.4.1
• —Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->