ksg-dfci/TrialSpace-0426

SentenceTransformer

This is a sentence-transformers model trained on the mnridataset and contrastivedataset datasets. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• —Model Type: Sentence Transformer <!-- - Base model: Unknown -->
• —Maximum Sequence Length: 2500 tokens
• —Output Dimensionality: 1024 dimensions
• —Similarity Function: Cosine Similarity
• —Training Datasets:
• —mnri_dataset
• —contrastive_dataset <!-- - Language: Unknown --> <!-- - License: Unknown -->

Model Sources

• —Documentation: Sentence Transformers Documentation
• —Repository: Sentence Transformers on GitHub
• —Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 2500, 'do_lower_case': False, 'architecture': 'Qwen3Model'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    "Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 57\nSex: Male\nCancer type: Gastric adenocarcinoma (HER2\u2011amplified, intestinal type)\nHistology: Moderately differentiated intestinal\u2011type adenocarcinoma\nCurrent extent: Metastatic disease with hepatic metastases (dominant lesion ~4\u202fcm in segment\u202fVII, multiple smaller lesions) and diffuse peritoneal carcinomatosis; bulky omental implants and moderate ascites; persistent circumferential thickening of the gastric cardia (~2.5\u202fcm). No longer receiving disease\u2011directed therapy; transitioned to best\u2011supportive hospice care (ECOG\u202f\u2248\u202f4).\nBiomarkers: HER2 IHC\u202f3+, HER2 (ERBB2) amplification confirmed by FISH (ratio\u202f5.2); Microsatellite stable (MSS); KRAS G12V (pathogenic); TP53 R273C (pathogenic); CDH1 G274E (likely pathogenic); PIK3CA E542K (activating); MET amplification (Copy number\u202f\u223c6); FGFR2 amplification (Copy number\u202f\u223c5); CDKN2A homozygous deletion; Tumor mutational burden\u202f\u2248\u202f8\u202fMut/Mb; Ki\u201167 \u223c45%; CK7\u207a/CK20\u207b; retained MLH1,PMS2,MSH2,MSH6.\nTreatment history:\n# 1/2018\u2011mid\u20112018: Front\u2011line trastuzumab (loading 8\u202fmg/kg then 6\u202fmg/kg q21\u202fd)\u202f+\u202fcisplatin 80\u202fmg/m\u00b2 iv day\u202f1 q21\u202fd\u202f+\u202fcontinuous infusional 5\u2011fu 1000\u202fmg/m\u00b2 days\u202f1\u20114 q21\u202fd (four cycles). Best response: Partial response (shrinkage of gastric wall thickening and hepatic lesions).\n# Late\u202f2018 (cycles\u202f5\u20116): Continuation of the same triplet regimen to complete six cycles; maintained partial response.\n# 12/2018: Restaging CT confirmed ongoing partial response.\n# Early\u202f2019: Planned radical gastrectomy aborted intra\u2011operatively due to diffuse peritoneal disease; feeding jejunostomy placed.\n# 5/2019: Elective laparoscopic right hemicolectomy for synchronous ascending colon adenocarcinoma (pT1a\u202fN0\u202fM0, R0). No adjuvant therapy required.\n# 6/2019 onward: Second\u2011line ramucirumab 10\u202fmg/kg i.v q2\u2009wks\u202f+\u202fweekly paclitaxel 80\u202fmg/m\u00b2 (initiated after progression on first\u2011line). Delivered two cycles; development of grade\u202f2 peripheral sensory neuropathy led to dose reduction of paclitaxel to 70\u202fmg/m\u00b2 and eventual cessation after disease progression.\n# 10/2019: Further disease progression evidenced by growing hepatic metastases and new peritoneal implants; cardiology consulted after NSTEMI; recommendation to cease cytotoxic and anti\u2011angiogenic agents.\n# 1/2020: Evaluated for experimental HER2/PD\u2011L1 bispecific antibody trial; found ineligible due to therapeutic anticoagulation (apixaban) begun shortly after NSTEMI.\n# 4/2020: Transitioned to hospice/best\u2011supportive care; all systemic anticancer agents (incl. trastuzumab) discontinued. Focus shifted to symptom control (opioids, anti\u2011emetics, nutritional support via jejunostomy) and advance\u2011directive implementation.\nCancer type: Ascending colon adenocarcinoma\nHistology: Well\u2011to\u2011moderately differentiated tubular adenocarcinoma, grade\u202f1\nCurrent extent: Resected, pathologic stage\u202fIA (pT1a\u202fN0\u202fM0), R0 margins; no evidence of disease on surveillance; no further therapy required.\nBiomarkers: KRAS wild\u2011type; BRAF V600E negative; MSI stable; CK20\u207a, CDX2\u207a.\nTreatment history:\n# 5/2019: Laparoscopic right hemicolectomy with regional node sampling (0/15 positive); postoperative course uncomplicated.",
]
documents = [
    'Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 18-75 years. Sex allowed: Male and Female. Cancer type allowed: Any solid malignant tumor. Histology allowed: NA. Cancer burden allowed: Advanced unresectable or metastatic disease where standard therapies have failed, are intolerable, or ineffective. Prior treatment required: Progression after standard anticancer therapy. Prior treatment excluded: Any prior gene or cell therapy product or any prior therapy directly targeting KRAS G12V mutation (e.g., KRAS\u202fG12V‐specific small‑molecule inhibitor or cellular therapy). Biomarkers required: KRAS G12V mutation and HLA‑A*11:01 positivity (both assessed during screening). Biomarkers excluded: NA.',
    'Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: NA. Sex allowed: Male and Female. Cancer type allowed: Non‑small cell lung cancer. Histology allowed: Adenocarcinoma, large\u202fcell, neuroendocrine or any non‑squamous histology (predominantly squamous histology excluded). Cancer burden allowed: Unresectable, locally advanced, or metastatic disease. Prior treatment required: Disease refractory to or progressive after all standard‐of‑care therapies demonstrating clinical benefit, unless no applicable standard therapy exists or the patient chooses to decline. Prior treatment excluded: Receipt of concurrent systemic anticancer therapy other than protocol‑permitted localized palliative radiation or hormone ablative therapy. Biomarkers required: NA. Biomarkers excluded: NA.',
    'Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 18-75 years. Sex allowed: Male and Female. Cancer type allowed: Any solid malignant tumor. Histology allowed: NA. Cancer burden allowed: Locally advanced or metastatic disease. Prior treatment required: NA. Prior treatment excluded: NA. Biomarkers required: NECTIN4 gene amplification positive. Biomarkers excluded: NA.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.4265, 0.1800, 0.3230]])

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Datasets

mnri_dataset

• —Dataset: mnri_dataset
• —Size: 913,338 training samples
• —Columns: <code>patientsummarytrunc</code> and <code>thisspacetrunc</code>
• —Approximate statistics based on the first 1000 samples: | | patientsummarytrunc | thisspacetrunc | |:--------|:---------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------| | type | string | string | | details | <ul><li>min: 264 tokens</li><li>mean: 805.57 tokens</li><li>max: 1600 tokens</li></ul> | <ul><li>min: 112 tokens</li><li>mean: 170.21 tokens</li><li>max: 388 tokens</li></ul> |
• —Samples: | patientsummarytrunc | thisspacetrunc | |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 15<br>Sex: Female<br>Cancer type: B‑cell acute lymphoblastic leukemia (precursor B‑cell ALL)<br>Histology: Precursor B‑lymphoblasts (flow: CD19⁺, CD22⁺, CD10⁺, CD34⁺, TdT⁺; CD20−, MPO−, surface IgM−)<br>Current extent: Systemic disease with diffuse bone‑marrow involvement; now in partial remission after induction (marrow blasts ≈15 %, MRD low‑level positive ∼0.02 %)<br>Biomarkers: TCF3‑PBX1 fusion (t(1;19)), hyperdiploid karyotype (extra chromosomes 4, 10, 17), PAX5 truncating mutation (p.R38), CDKN2A/CDKN2B homozygous deletion, NRAS activating mutation (p.Q61K), low‑frequency CREBBP p.R1746H (subclonal), negative for BCR‑ABL1 and ETV6‑RUNX1; immunophenotype as above<br>Treatment history:<br># 08/2017 – 09/2017 (Day 1‑28 of induction): Multidrug induction per COG AALL1131 – vincristine ...</code> | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 0 through 45 years. Sex allowed: both sexes. Cancer type allowed: acute lymphoblastic leukaemia. Histology allowed: B‑cell precursor acute lymphoblastic leukaemia. Cancer burden allowed: newly diagnosed, never relapsed, de novo disease. Prior treatment required: NA. Prior treatment excluded: systemic corticosteroids ≥10 mg/m²/day prednisone equivalents for longer than one week before diagnosis; any chemotherapeutic agent administered within four weeks before diagnosis. Biomarkers required: surface immunoglobulin negative phenotype; IG::MYC rearrangement accepted only when BCL2 and BCL6 rearrangements are absent. Biomarkers excluded: KMT2A‑rearranged B‑cell precursor ALL in patients younger than 1 year; Philadelphia chromosome‑positive (t[9;22]/BCR‑ABL) ...</code> | | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 53<br>Sex: Male<br>Cancer type: Lung cancer<br>Histology: Poorly differentiated squamous cell carcinoma (left lower lobe)<br>Current extent: Metastatic disease – primary left lower‑lobar lesion ~3.5 cm, new left adrenal metastasis (1.8 cm), right hilar lymph node enlargement; overall progressive disease per RECIST 1.1<br>Biomarkers: PD‑L1 tumor proportion score 10 %; HER2 (ERBB2) amplification (≈9 copies); TP53 missense mutation p.R273C (likely pathogenic); CDKN2A homozygous deletion; FGFR1 amplification (≈7 copies); PIK3CA H1047R activating mutation; NOTCH1 L1575P variant of uncertain significance; KRAS wild‑type; EGFR wild‑type<br>Treatment history:<br># 1/Jan 2017: Diagnostic bronchoscopic forceps biopsies of left lower lobar endobronchial lesion → confirmation of poorly differentiate...</code> | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 18-75 years. Sex allowed: Male and Female. Cancer type allowed: Advanced solid tumor. Histology allowed: Any. Cancer burden allowed: Locally advanced or metastatic disease refractory to standard therapy or lacking effective treatment. Prior treatment required: Progressive disease after standard systemic therapy. Prior treatment excluded: Prior receipt of targeted ROR1 inhibitor therapy. Biomarkers required: ROR1 positive (assessment planned during screening). Biomarkers excluded: NA.</code> | | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 9<br>Sex: Male<br>Cancer type: B‑myeloid Mixed Phenotype Acute Leukemia (MPAL)<br>Histology: Biphasic B‑cell/myeloid acute leukemia<br>Current extent: Post‑second allogeneic HSCT, in remission; flow MRD < 0.05 % (0.02 %), low‑level FLT3‑ITD persistence (allelic ratio ∼0.38)<br>Biomarkers: FLT3‑ITD (persisting, AR ∼ 0.38); WT1 truncating mutation p.Arg430; NRAS p.Gly12Asp; DNMT3A p.Arg882His; CDKN2A/B homozygous deletion (9p21); IDH2 p.Arg140Gln (VAF ≈ 21 %→7 % across samples); NUP98‑NSD1 fusion (detected at diagnosis, subsequently undetectable); Additional routine panels negative for KMT2A rearrangement, BCR‑ABL1, other actionable hits.<br>Treatment history:<br># 1/2017‑5/2017: FLAG‑IDA induction (Fludarabine, Cytarabine, G‑CSF, Idarubicin) + intrathecal Methotrexate; best response comp...</code> | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 1 Year to 39 Years. Sex allowed: Male and Female. Cancer type allowed: Mixed phenotype acute leukemia, B‑myeloid. Histology allowed: B‑myeloid mixed phenotype acute leukemia. Cancer burden allowed: Relapsed or refractory disease. Prior treatment required: Full recovery from prior hematopoietic stem cell transplantation or anthracycline exposure. Prior treatment excluded: Current administration of anticancer agents (except intrathecal agents or hydroxyurea). Biomarkers required: NA. Biomarkers excluded: KMT2A rearrangement, Philadelphia chromosome/BCR‑ABL1 fusion.</code> |
• —Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }

contrastive_dataset

• —Dataset: contrastive_dataset
• —Size: 1,526,022 training samples
• —Columns: <code>patientsummarytrunc</code>, <code>thisspacetrunc</code>, and <code>label</code>
• —Approximate statistics based on the first 1000 samples: | | patientsummarytrunc | thisspacetrunc | label | |:--------|:---------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------|:----------------------------------------------------------------| | type | string | string | float | | details | <ul><li>min: 264 tokens</li><li>mean: 840.28 tokens</li><li>max: 1600 tokens</li></ul> | <ul><li>min: 121 tokens</li><li>mean: 185.75 tokens</li><li>max: 388 tokens</li></ul> | <ul><li>min: -1.0</li><li>mean: -0.4</li><li>max: 1.0</li></ul> |
• —Samples: | patientsummarytrunc | thisspacetrunc | label | |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------| | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 15<br>Sex: Female<br>Cancer type: B‑cell acute lymphoblastic leukemia (precursor B‑cell ALL)<br>Histology: Precursor B‑lymphoblasts (flow: CD19⁺, CD22⁺, CD10⁺, CD34⁺, TdT⁺; CD20−, MPO−, surface IgM−)<br>Current extent: Systemic disease with diffuse bone‑marrow involvement; now in partial remission after induction (marrow blasts ≈15 %, MRD low‑level positive ∼0.02 %)<br>Biomarkers: TCF3‑PBX1 fusion (t(1;19)), hyperdiploid karyotype (extra chromosomes 4, 10, 17), PAX5 truncating mutation (p.R38), CDKN2A/CDKN2B homozygous deletion, NRAS activating mutation (p.Q61K), low‑frequency CREBBP p.R1746H (subclonal), negative for BCR‑ABL1 and ETV6‑RUNX1; immunophenotype as above<br>Treatment history:<br># 08/2017 – 09/2017 (Day 1‑28 of induction): Multidrug induction per COG AALL1131 – vincristine ...</code> | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 0 through 45 years. Sex allowed: both sexes. Cancer type allowed: acute lymphoblastic leukaemia. Histology allowed: B‑cell precursor acute lymphoblastic leukaemia. Cancer burden allowed: newly diagnosed, never relapsed, de novo disease. Prior treatment required: NA. Prior treatment excluded: systemic corticosteroids ≥10 mg/m²/day prednisone equivalents for longer than one week before diagnosis; any chemotherapeutic agent administered within four weeks before diagnosis. Biomarkers required: surface immunoglobulin negative phenotype; IG::MYC rearrangement accepted only when BCL2 and BCL6 rearrangements are absent. Biomarkers excluded: KMT2A‑rearranged B‑cell precursor ALL in patients younger than 1 year; Philadelphia chromosome‑positive (t[9;22]/BCR‑ABL) ...</code> | <code>0.6000000000000001</code> | | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 53<br>Sex: Male<br>Cancer type: Lung cancer<br>Histology: Poorly differentiated squamous cell carcinoma (left lower lobe)<br>Current extent: Metastatic disease – primary left lower‑lobar lesion ~3.5 cm, new left adrenal metastasis (1.8 cm), right hilar lymph node enlargement; overall progressive disease per RECIST 1.1<br>Biomarkers: PD‑L1 tumor proportion score 10 %; HER2 (ERBB2) amplification (≈9 copies); TP53 missense mutation p.R273C (likely pathogenic); CDKN2A homozygous deletion; FGFR1 amplification (≈7 copies); PIK3CA H1047R activating mutation; NOTCH1 L1575P variant of uncertain significance; KRAS wild‑type; EGFR wild‑type<br>Treatment history:<br># 1/Jan 2017: Diagnostic bronchoscopic forceps biopsies of left lower lobar endobronchial lesion → confirmation of poorly differentiate...</code> | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 18-75 years. Sex allowed: Male and Female. Cancer type allowed: Advanced solid tumor. Histology allowed: Any. Cancer burden allowed: Locally advanced or metastatic disease refractory to standard therapy or lacking effective treatment. Prior treatment required: Progressive disease after standard systemic therapy. Prior treatment excluded: Prior receipt of targeted ROR1 inhibitor therapy. Biomarkers required: ROR1 positive (assessment planned during screening). Biomarkers excluded: NA.</code> | <code>0.19999999999999996</code> | | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age: 9<br>Sex: Male<br>Cancer type: B‑myeloid Mixed Phenotype Acute Leukemia (MPAL)<br>Histology: Biphasic B‑cell/myeloid acute leukemia<br>Current extent: Post‑second allogeneic HSCT, in remission; flow MRD < 0.05 % (0.02 %), low‑level FLT3‑ITD persistence (allelic ratio ∼0.38)<br>Biomarkers: FLT3‑ITD (persisting, AR ∼ 0.38); WT1 truncating mutation p.Arg430; NRAS p.Gly12Asp; DNMT3A p.Arg882His; CDKN2A/B homozygous deletion (9p21); IDH2 p.Arg140Gln (VAF ≈ 21 %→7 % across samples); NUP98‑NSD1 fusion (detected at diagnosis, subsequently undetectable); Additional routine panels negative for KMT2A rearrangement, BCR‑ABL1, other actionable hits.<br>Treatment history:<br># 1/2017‑5/2017: FLAG‑IDA induction (Fludarabine, Cytarabine, G‑CSF, Idarubicin) + intrathecal Methotrexate; best response comp...</code> | <code>Instruct: Given a cancer patient summary, retrieve clinical trial options that are reasonable for that patient; or, given a clinical trial option, retrieve cancer patients who are reasonable candidates for that trial. Age range allowed: 1 Year to 39 Years. Sex allowed: Male and Female. Cancer type allowed: Mixed phenotype acute leukemia, B‑myeloid. Histology allowed: B‑myeloid mixed phenotype acute leukemia. Cancer burden allowed: Relapsed or refractory disease. Prior treatment required: Full recovery from prior hematopoietic stem cell transplantation or anthracycline exposure. Prior treatment excluded: Current administration of anticancer agents (except intrathecal agents or hydroxyurea). Biomarkers required: NA. Biomarkers excluded: KMT2A rearrangement, Philadelphia chromosome/BCR‑ABL1 fusion.</code> | <code>0.19999999999999996</code> |
• —Loss: <code>CoSENTLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }

Training Hyperparameters

Non-Default Hyperparameters

• —per_device_train_batch_size: 10
• —learning_rate: 2e-05
• —warmup_ratio: 0.01
• —bf16: True

All Hyperparameters

<details><summary>Click to expand</summary>

• —overwrite_output_dir: False
• —do_predict: False
• —eval_strategy: no
• —prediction_loss_only: True
• —per_device_train_batch_size: 10
• —per_device_eval_batch_size: 8
• —per_gpu_train_batch_size: None
• —per_gpu_eval_batch_size: None
• —gradient_accumulation_steps: 1
• —eval_accumulation_steps: None
• —torch_empty_cache_steps: None
• —learning_rate: 2e-05
• —weight_decay: 0.0
• —adam_beta1: 0.9
• —adam_beta2: 0.999
• —adam_epsilon: 1e-08
• —max_grad_norm: 1.0
• —num_train_epochs: 3
• —max_steps: -1
• —lr_scheduler_type: linear
• —lr_scheduler_kwargs: None
• —warmup_ratio: 0.01
• —warmup_steps: 0
• —log_level: passive
• —log_level_replica: warning
• —log_on_each_node: True
• —logging_nan_inf_filter: True
• —save_safetensors: True
• —save_on_each_node: False
• —save_only_model: False
• —restore_callback_states_from_checkpoint: False
• —no_cuda: False
• —use_cpu: False
• —use_mps_device: False
• —seed: 42
• —data_seed: None
• —jit_mode_eval: False
• —bf16: True
• —fp16: False
• —fp16_opt_level: O1
• —half_precision_backend: auto
• —bf16_full_eval: False
• —fp16_full_eval: False
• —tf32: None
• —local_rank: 2
• —ddp_backend: None
• —tpu_num_cores: None
• —tpu_metrics_debug: False
• —debug: []
• —dataloader_drop_last: True
• —dataloader_num_workers: 0
• —dataloader_prefetch_factor: None
• —past_index: -1
• —disable_tqdm: False
• —remove_unused_columns: True
• —label_names: None
• —load_best_model_at_end: False
• —ignore_data_skip: False
• —fsdp: []
• —fsdp_min_num_params: 0
• —fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
• —fsdp_transformer_layer_cls_to_wrap: None
• —accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
• —parallelism_config: None
• —deepspeed: None
• —label_smoothing_factor: 0.0
• —optim: adamwtorchfused
• —optim_args: None
• —adafactor: False
• —group_by_length: False
• —length_column_name: length
• —project: huggingface
• —trackio_space_id: trackio
• —ddp_find_unused_parameters: None
• —ddp_bucket_cap_mb: None
• —ddp_broadcast_buffers: False
• —dataloader_pin_memory: True
• —dataloader_persistent_workers: False
• —skip_memory_metrics: True
• —use_legacy_prediction_loop: False
• —push_to_hub: False
• —resume_from_checkpoint: None
• —hub_model_id: None
• —hub_strategy: every_save
• —hub_private_repo: None
• —hub_always_push: False
• —hub_revision: None
• —gradient_checkpointing: False
• —gradient_checkpointing_kwargs: None
• —include_inputs_for_metrics: False
• —include_for_metrics: []
• —eval_do_concat_batches: True
• —fp16_backend: auto
• —push_to_hub_model_id: None
• —push_to_hub_organization: None
• —mp_parameters:
• —auto_find_batch_size: False
• —full_determinism: False
• —torchdynamo: None
• —ray_scope: last
• —ddp_timeout: 1800
• —torch_compile: False
• —torch_compile_backend: None
• —torch_compile_mode: None
• —include_tokens_per_second: False
• —include_num_input_tokens_seen: no
• —neftune_noise_alpha: None
• —optim_target_modules: None
• —batch_eval_metrics: False
• —eval_on_start: False
• —use_liger_kernel: False
• —liger_kernel_config: None
• —eval_use_gather_object: False
• —average_tokens_across_devices: True
• —prompts: None
• —batch_sampler: batch_sampler
• —multi_dataset_batch_sampler: proportional
• —router_mapping: {}
• —learning_rate_mapping: {}

</details>

Training Logs

<details><summary>Click to expand</summary>

</details>

Framework Versions

• —Python: 3.13.12
• —Sentence Transformers: 5.2.3
• —Transformers: 4.57.6
• —PyTorch: 2.9.1+cu128
• —Accelerate: 1.13.0
• —Datasets: 4.6.1
• —Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

CoSENTLoss

@article{10531646,
    author={Huang, Xiang and Peng, Hao and Zou, Dongcheng and Liu, Zhiwei and Li, Jianxin and Liu, Kay and Wu, Jia and Su, Jianlin and Yu, Philip S.},
    journal={IEEE/ACM Transactions on Audio, Speech, and Language Processing},
    title={CoSENT: Consistent Sentence Embedding via Similarity Ranking},
    year={2024},
    doi={10.1109/TASLP.2024.3402087}
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->