ksg-dfci/TrialSpace-1225

SentenceTransformer

This is a sentence-transformers model. It maps sentences & paragraphs to a dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. In the context of MatchMiner-AI, this model is trained to embed patient summaries and clinical trial "space" (target population summaries) to enable rapid retrieval of trial options for patients and vice versa.

Model Details

Model Description

• —Model Type: Sentence Transformer <!-- - Base model: Unknown -->
• —Maximum Sequence Length: 2500 tokens
• —Output Dimensionality: 1024 dimensions
• —Similarity Function: Cosine Similarity
• —Training Datasets:
• —mnri_dataset
• —contrastive_dataset <!-- - Language: English --> <!-- - License: Unknown -->

Model Sources

• —Documentation: Sentence Transformers Documentation
• —Repository: Sentence Transformers on GitHub
• —Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 2500, 'do_lower_case': False, 'architecture': 'Qwen3Model'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    "Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 63  \nSex: Male  \nCancer type: Diffuse large B\u2011cell lymphoma (DLBCL)  \nHistology: Activated B\u2011cell (non\u2011germinal\u2011center) phenotype, CD20\u202fpositive, MUM1\u202fpositive, BCL2\u202fpositive, Ki\u201167\u224895\u202f%  \nCurrent extent: Limited residual disease \u2013 residual metabolic activity in the original right posterior thigh soft\u2011tissue mass (partial metabolic response) and a solitary right iliac lymph node with stable low\u2011volume uptake (stable disease); no other sites of disease evident.  \n\nBiomarkers:  \n- Immunohistochemistry: CD20+, MUM1+, BCL2+, CD10\u2011, BCL6\u2011, Ki\u201167\u224895%  \n- FISH: No MYC, BCL2, or BCL6 rearrangements (triple\u2011hit excluded)  \n- NGS (original thigh lesion): MYD88 L265P (VAF\u202f\u224835\u202f%), CD79B Y196F (VAF\u202f\u224830\u202f%), CARD11 S328F (VAF\u202f\u224828\u202f%), REL copy\u2011number gain (\u22483\u20134 copies) \u2013 low tumor mutational burden (5\u202fmut/Mb), microsatellite stable.  \n- Additional alterations identified in iliac node (September\u00a02013): TP53 p.R248Q (VAF\u202f\u224822\u202f%), CDKN2A homozygous deletion.  \n\nTreatment history:  \n# Early\u202f2011\u2013mid\u202f2011: R\u2011CHOP (rituximab, cyclophosphamide, doxorubicin, vincristine, prednisolone) \u2013 6 cycles, 21\u2011day schedule. Best response: partial metabolic response (Deauville\u202f4).  \n# Mid\u202f2011 (following R\u2011CHOP): High\u2011dose BEAM conditioning with autologous stem\u2011cell transplantation \u2013 achieved only persistent residual disease (no complete remission).  \n# Early\u202f2012 (exact infusion date unspecified, Day\u202f0 of CAR\u2011T): Axicabtagene\u202fciloleucel (axi\u2011cel) CAR\u2011T after fludarabine/cyclophosphamide lymphodepletion. Course complicated by Grade\u202f3 cytokine\u2011release syndrome (tocilizumab, brief vasopressors) and Grade\u202f2 immune effector cell\u2011associated neurotoxicity syndrome (resolved). Day\u202f60 restaging (July\u202f2012) showed partial metabolic response of thigh lesion (\u2193SUV\u202ffrom\u202f\u224813\u202fto\u202f4.2) but emergence of a new right iliac node (SUV\u202f\u22489.3) consistent with progressive disease.  \n# Late\u202f2012 (December\u202f2012): Palliative external\u2011beam radiation to right iliac node, 30\u202fGy in 15 fractions (2\u202fGy per fraction). Resulted in symptom relief and reduction of nodal size/metabolic activity (node size \u2193 to 2.5\u202fcm, SUV\u202f\u22485.8).  \n# 2013\u202fMarch onward: Observation/off\u2011therapy; serial imaging through 2014 demonstrates stable disease (persistent low\u2011grade uptake in thigh lesion, SUV\u202f\u22484.2; iliac node stable, SUV\u202f\u22485.8). No further systemic therapy administered.",
]
documents = [
    'Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: ≥18 years. Sex allowed: both. Cancer type allowed: diffuse large b cell lymphoma. Histology allowed: diffuse large b cell lymphoma–not otherwise specified. Cancer burden allowed: evaluable/measurable disease; absent massive uncontrolled pleural, pericardial, or peritoneal effusions, pulmonary lymphangitis, and liver involvement exceeding 50\u202f%. Prior treatment required: relapsed or refractory disease after prior systemic anti‑lymphoma therapy (including patients ineligible for car t‑cell therapy). Prior treatment excluded: any concurrent antitumor‑directed drug therapy. Biomarkers required: NA. Biomarkers excluded: NA.',
    'Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 18-75 years. Sex allowed: Male and Female. Cancer type allowed: Any malignant solid tumor. Histology allowed: NA. Cancer burden allowed: Stage\u202fIIIB‑IV disease that has progressed after prior standard therapy and for whom immune‐monotherapy is planned, regardless of PD‑L1 result. Prior treatment required: Progression after receipt of standard therapy. Prior treatment excluded: NA. Biomarkers required: Presence of at least one positively expressed tumor neoantigen; PD‑L1 testing permitted but not mandatory. Biomarkers excluded: NA.',
    'Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: >=18. Sex allowed: female. Cancer type allowed: ovarian carcinosarcoma. Histology allowed: mixed carcinoma. Cancer burden allowed: relapsed or refractory disease after at least one prior platinum‑based regimen (platinum‑resistant <183\xa0days) with measurable disease. Prior treatment required: at least one prior line of platinum chemotherapy. Prior treatment excluded: prior MUC1‑directed therapy. Biomarkers required: MUC1 expression positive (>0 IHC) assessed at screening. Biomarkers excluded: NA',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.8415, 0.3722, 0.0634]])

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Datasets

mnri_dataset

• —Dataset: mnri_dataset
• —Size: 756,840 training samples
• —Columns: <code>patientsummarytrunc</code> and <code>thisspacetrunc</code>
• —Approximate statistics based on the first 1000 samples: | | patientsummarytrunc | thisspacetrunc | |:--------|:---------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------| | type | string | string | | details | <ul><li>min: 386 tokens</li><li>mean: 911.75 tokens</li><li>max: 1827 tokens</li></ul> | <ul><li>min: 112 tokens</li><li>mean: 168.64 tokens</li><li>max: 388 tokens</li></ul> |
• —Samples: | patientsummarytrunc | thisspacetrunc | |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 32 <br>Sex: Male <br>Cancer type: B‑cell precursor acute lymphoblastic leukemia (ALL) <br>Histology: B‑cell lineage lymphoblasts (CD19⁺, CD20⁺, CD10⁺, TdT⁺) <br>Current extent: Systemic marrow disease (diffusely involved bone marrow, no detectable extramedullary or CNS disease) – considered advanced/systemic <br><br>Biomarkers: <br>- PAX5 p.Arg31Ser (likely pathogenic missense) – VAF ≈ 48% <br>- NRAS p.Gln61Lys (activating missense) – VAF ≈ 35‑41% <br>- FLT3 internal tandem duplication (30‑bp juxtamembrane) – pathogenic, allelic ratio ≈ 30‑36% <br>- CDKN2A homozygous deletion (exons 1‑2) – pathogenic <br>- Low‑level JAK2 V617F (p.Val617Phe) – subclonal, VAF ≈ 3% <br>- CREBBP p.Arg1664His – variant of uncertain significance, VAF ≈ 7% <br>- No BCR‑ABL1 fusion, no TEL‑AML1, no MLL/KMT2A rearrange...</code> | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 0 through 45 years. Sex allowed: both sexes. Cancer type allowed: acute lymphoblastic leukaemia. Histology allowed: B‑cell precursor acute lymphoblastic leukaemia. Cancer burden allowed: newly diagnosed, never relapsed, de novo disease. Prior treatment required: NA. Prior treatment excluded: systemic corticosteroids ≥10 mg/m²/day prednisone equivalents for longer than one week before diagnosis; any chemotherapeutic agent administered within four weeks before diagnosis. Biomarkers required: surface immunoglobulin negative phenotype; IG::MYC rearrangement accepted only when BCL2 and BCL6 rearrangements are absent. Biomarkers excluded: KMT2A‑rearranged B‑cell precursor ALL in patients younger than 1 year; Philadelphia chromosome‑positive (t[9;22]/BCR‑ABL) ...</code> | | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 66 years (born 1959, most recent entry dated September 2017) <br>Sex: Female <br><br>Cancer type: Pancreatic ductal adenocarcinoma (PDAC) <br>Histology: Poorly differentiated invasive ductal adenocarcinoma <br><br>Current extent: Metastatic (Stage IV) – hepatic dominant disease with two liver deposits (segment VI ≈ 1.4 cm, segment VII ≈ 1.8 cm), solitary osteoblastic metastasis to L3 vertebral body, and persistent peripancreatic soft‑tissue thickening (likely post‑surgical fibrosis vs low‑volume nodal disease). No CNS disease documented. <br><br>Biomarkers (most recent specimen, September 2017 hepatic core biopsy): <br>• KRAS G12D point mutation (VAF 18%) – activating hotspot, different from earlier KRAS G12R clone <br>• KRAS focal amplification (estimated copy ≈ 7) <br>• TP53 R273H missense...</code> | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 18-75 years. Sex allowed: Male and Female. Cancer type allowed: Advanced solid tumor. Histology allowed: Any. Cancer burden allowed: Locally advanced or metastatic disease refractory to standard therapy or lacking effective treatment. Prior treatment required: Progressive disease after standard systemic therapy. Prior treatment excluded: Prior receipt of targeted ROR1 inhibitor therapy. Biomarkers required: ROR1 positive (assessment planned during screening). Biomarkers excluded: NA.</code> | | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 66 <br>Sex: Female <br>Cancer type: Ovarian carcinosarcoma (malignant mixed Müllerian tumor) <br>Histology: Biphasic high‑grade serous carcinoma component (WT1⁺, PAX8⁺, p53 overexpression) with undifferentiated spindle‑cell sarcomatous component (vimentin⁺, desmin‑) <br><br>Current extent: Metastatic disease (peritoneal carcinomatosis with implants up to ≈2 cm, right pleural effusion, bilateral subcentimeter pulmonary nodules) – clinically stage IV, stable on current therapy <br><br>Biomarkers: <br>- TP53 p.Y220C (Likely pathogenic, somatic, VAF 68%) <br>- KRAS p.G12D (Likely pathogenic, somatic, VAF 9%) <br>- BRCA2 splice‑site alteration c.5946+2T>A (Likely pathogenic, somatic, VAF 12%) – HRD score negative, microsatellite stable, TMB 6 mut/Mb (low) <br>- CCNE1 copy‑number gain (~5‑fold am...</code> | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: >=18. Sex allowed: female. Cancer type allowed: ovarian carcinosarcoma. Histology allowed: mixed carcinoma. Cancer burden allowed: relapsed or refractory disease after at least one prior platinum‑based regimen (platinum‑resistant <183 days) with measurable disease. Prior treatment required: at least one prior line of platinum chemotherapy. Prior treatment excluded: prior MUC1‑directed therapy. Biomarkers required: MUC1 expression positive (>0 IHC) assessed at screening. Biomarkers excluded: NA</code> |
• —Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }

contrastive_dataset

• —Dataset: contrastive_dataset
• —Size: 1,530,582 training samples
• —Columns: <code>patientsummarytrunc</code>, <code>thisspacetrunc</code>, and <code>label</code>
• —Approximate statistics based on the first 1000 samples: | | patientsummarytrunc | thisspacetrunc | label | |:--------|:---------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 202 tokens</li><li>mean: 906.99 tokens</li><li>max: 1827 tokens</li></ul> | <ul><li>min: 121 tokens</li><li>mean: 185.89 tokens</li><li>max: 388 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.39</li><li>max: 1.0</li></ul> |
• —Samples: | patientsummarytrunc | thisspacetrunc | label | |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------| | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 32 <br>Sex: Male <br>Cancer type: B‑cell precursor acute lymphoblastic leukemia (ALL) <br>Histology: B‑cell lineage lymphoblasts (CD19⁺, CD20⁺, CD10⁺, TdT⁺) <br>Current extent: Systemic marrow disease (diffusely involved bone marrow, no detectable extramedullary or CNS disease) – considered advanced/systemic <br><br>Biomarkers: <br>- PAX5 p.Arg31Ser (likely pathogenic missense) – VAF ≈ 48% <br>- NRAS p.Gln61Lys (activating missense) – VAF ≈ 35‑41% <br>- FLT3 internal tandem duplication (30‑bp juxtamembrane) – pathogenic, allelic ratio ≈ 30‑36% <br>- CDKN2A homozygous deletion (exons 1‑2) – pathogenic <br>- Low‑level JAK2 V617F (p.Val617Phe) – subclonal, VAF ≈ 3% <br>- CREBBP p.Arg1664His – variant of uncertain significance, VAF ≈ 7% <br>- No BCR‑ABL1 fusion, no TEL‑AML1, no MLL/KMT2A rearrange...</code> | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 0 through 45 years. Sex allowed: both sexes. Cancer type allowed: acute lymphoblastic leukaemia. Histology allowed: B‑cell precursor acute lymphoblastic leukaemia. Cancer burden allowed: newly diagnosed, never relapsed, de novo disease. Prior treatment required: NA. Prior treatment excluded: systemic corticosteroids ≥10 mg/m²/day prednisone equivalents for longer than one week before diagnosis; any chemotherapeutic agent administered within four weeks before diagnosis. Biomarkers required: surface immunoglobulin negative phenotype; IG::MYC rearrangement accepted only when BCL2 and BCL6 rearrangements are absent. Biomarkers excluded: KMT2A‑rearranged B‑cell precursor ALL in patients younger than 1 year; Philadelphia chromosome‑positive (t[9;22]/BCR‑ABL) ...</code> | <code>1.0</code> | | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 66 years (born 1959, most recent entry dated September 2017) <br>Sex: Female <br><br>Cancer type: Pancreatic ductal adenocarcinoma (PDAC) <br>Histology: Poorly differentiated invasive ductal adenocarcinoma <br><br>Current extent: Metastatic (Stage IV) – hepatic dominant disease with two liver deposits (segment VI ≈ 1.4 cm, segment VII ≈ 1.8 cm), solitary osteoblastic metastasis to L3 vertebral body, and persistent peripancreatic soft‑tissue thickening (likely post‑surgical fibrosis vs low‑volume nodal disease). No CNS disease documented. <br><br>Biomarkers (most recent specimen, September 2017 hepatic core biopsy): <br>• KRAS G12D point mutation (VAF 18%) – activating hotspot, different from earlier KRAS G12R clone <br>• KRAS focal amplification (estimated copy ≈ 7) <br>• TP53 R273H missense...</code> | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 18-75 years. Sex allowed: Male and Female. Cancer type allowed: Advanced solid tumor. Histology allowed: Any. Cancer burden allowed: Locally advanced or metastatic disease refractory to standard therapy or lacking effective treatment. Prior treatment required: Progressive disease after standard systemic therapy. Prior treatment excluded: Prior receipt of targeted ROR1 inhibitor therapy. Biomarkers required: ROR1 positive (assessment planned during screening). Biomarkers excluded: NA.</code> | <code>1.0</code> | | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 18 <br>Sex: Female <br><br>Cancer type: B‑myeloid mixed‑phenotype acute leukemia (MPAL) <br>Histology: Biphenotypic blasts expressing B‑lineage (CD19+) and myeloid (CD33+, MPO+) markers; Philadelphia‑positive (BCR‑ABL1 p190) <br><br>Current extent: Relapsed/refractory disease with low‑burden molecular persistence (BCR‑ABL1 transcript ~0.02 % IS); no morphologic blasts in marrow, no extramedullary disease evident on imaging <br><br>Biomarkers: <br>- BCR‑ABL1 p190 fusion (RNA‑based detection, pathogenic) – present throughout disease course <br>- IKZF1 deletion, KRAS Q61H, PTPN11 N308S, MYC copy‑number gain – identified on initial NGS (2018‑12) and persistently reported <br>- MRD quantitative PCR: 0.05 % (day 30 post‑transplant, 2019‑03‑08); 0.02 % (Sept 2019); 0.03 % (Mar 2020); 0.022 % (June ...</code> | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 1 Year to 39 Years. Sex allowed: Male and Female. Cancer type allowed: Mixed phenotype acute leukemia, B‑myeloid. Histology allowed: B‑myeloid mixed phenotype acute leukemia. Cancer burden allowed: Relapsed or refractory disease. Prior treatment required: Full recovery from prior hematopoietic stem cell transplantation or anthracycline exposure. Prior treatment excluded: Current administration of anticancer agents (except intrathecal agents or hydroxyurea). Biomarkers required: NA. Biomarkers excluded: KMT2A rearrangement, Philadelphia chromosome/BCR‑ABL1 fusion.</code> | <code>0.0</code> |
• —Loss: <code>OnlineContrastiveLoss</code>

Training Hyperparameters

Non-Default Hyperparameters

• —per_device_train_batch_size: 18
• —learning_rate: 2e-05
• —warmup_ratio: 0.01
• —bf16: True

All Hyperparameters

<details><summary>Click to expand</summary>

• —overwrite_output_dir: False
• —do_predict: False
• —eval_strategy: no
• —prediction_loss_only: True
• —per_device_train_batch_size: 18
• —per_device_eval_batch_size: 8
• —per_gpu_train_batch_size: None
• —per_gpu_eval_batch_size: None
• —gradient_accumulation_steps: 1
• —eval_accumulation_steps: None
• —torch_empty_cache_steps: None
• —learning_rate: 2e-05
• —weight_decay: 0.0
• —adam_beta1: 0.9
• —adam_beta2: 0.999
• —adam_epsilon: 1e-08
• —max_grad_norm: 1.0
• —num_train_epochs: 3
• —max_steps: -1
• —lr_scheduler_type: linear
• —lr_scheduler_kwargs: {}
• —warmup_ratio: 0.01
• —warmup_steps: 0
• —log_level: passive
• —log_level_replica: warning
• —log_on_each_node: True
• —logging_nan_inf_filter: True
• —save_safetensors: True
• —save_on_each_node: False
• —save_only_model: False
• —restore_callback_states_from_checkpoint: False
• —no_cuda: False
• —use_cpu: False
• —use_mps_device: False
• —seed: 42
• —data_seed: None
• —jit_mode_eval: False
• —bf16: True
• —fp16: False
• —fp16_opt_level: O1
• —half_precision_backend: auto
• —bf16_full_eval: False
• —fp16_full_eval: False
• —tf32: None
• —local_rank: 1
• —ddp_backend: None
• —tpu_num_cores: None
• —tpu_metrics_debug: False
• —debug: []
• —dataloader_drop_last: True
• —dataloader_num_workers: 0
• —dataloader_prefetch_factor: None
• —past_index: -1
• —disable_tqdm: False
• —remove_unused_columns: True
• —label_names: None
• —load_best_model_at_end: False
• —ignore_data_skip: False
• —fsdp: []
• —fsdp_min_num_params: 0
• —fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
• —fsdp_transformer_layer_cls_to_wrap: None
• —accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
• —parallelism_config: None
• —deepspeed: None
• —label_smoothing_factor: 0.0
• —optim: adamwtorchfused
• —optim_args: None
• —adafactor: False
• —group_by_length: False
• —length_column_name: length
• —project: huggingface
• —trackio_space_id: trackio
• —ddp_find_unused_parameters: None
• —ddp_bucket_cap_mb: None
• —ddp_broadcast_buffers: False
• —dataloader_pin_memory: True
• —dataloader_persistent_workers: False
• —skip_memory_metrics: True
• —use_legacy_prediction_loop: False
• —push_to_hub: False
• —resume_from_checkpoint: None
• —hub_model_id: None
• —hub_strategy: every_save
• —hub_private_repo: None
• —hub_always_push: False
• —hub_revision: None
• —gradient_checkpointing: False
• —gradient_checkpointing_kwargs: None
• —include_inputs_for_metrics: False
• —include_for_metrics: []
• —eval_do_concat_batches: True
• —fp16_backend: auto
• —push_to_hub_model_id: None
• —push_to_hub_organization: None
• —mp_parameters:
• —auto_find_batch_size: False
• —full_determinism: False
• —torchdynamo: None
• —ray_scope: last
• —ddp_timeout: 1800
• —torch_compile: False
• —torch_compile_backend: None
• —torch_compile_mode: None
• —include_tokens_per_second: False
• —include_num_input_tokens_seen: no
• —neftune_noise_alpha: None
• —optim_target_modules: None
• —batch_eval_metrics: False
• —eval_on_start: False
• —use_liger_kernel: False
• —liger_kernel_config: None
• —eval_use_gather_object: False
• —average_tokens_across_devices: True
• —prompts: None
• —batch_sampler: batch_sampler
• —multi_dataset_batch_sampler: proportional
• —router_mapping: {}
• —learning_rate_mapping: {}

</details>

Training Logs

<details><summary>Click to expand</summary>

</details>

Framework Versions

• —Python: 3.12.12
• —Sentence Transformers: 5.1.2
• —Transformers: 4.57.1
• —PyTorch: 2.9.0+cu128
• —Accelerate: 1.12.0
• —Datasets: 4.4.1
• —Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->