ubco-mds-2025-capstone-fujitsu-1/bge-large-en-v1.5-cve-finetuned

SentenceTransformer based on BAAI/bge-large-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-large-en-v1.5. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• —Model Type: Sentence Transformer
• —Base model: BAAI/bge-large-en-v1.5 <!-- at revision d4aa6901d3a41ba39fb536a557fa166f842b0e09 -->
• —Maximum Sequence Length: 256 tokens
• —Output Dimensionality: 1024 dimensions
• —Similarity Function: Cosine Similarity
• —Supported Modality: Text <!-- - Training Dataset: Unknown --> <!-- - Language: Unknown --> <!-- - License: Unknown -->

Model Sources

• —Documentation: Sentence Transformers Documentation
• —Repository: Sentence Transformers on GitHub
• —Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 1024, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Represent this sentence for searching relevant passages: What issues exist with Intel Killer Control Center software?',
    'Improper access control for the Intel(R) Killer(TM) Control Center software before version 2.4.3337.0 may allow an authorized user to potentially enable escalation of privilege via local access.',
    'A content spoofing vulnerability in the following components allows to render html pages containing arbitrary plain text content, which might fool an end user: UI add-on for SAP NetWeaver (UI_Infra, 1.0), SAP UI Implementation for Decoupled Innovations (UI_700, 2.0): SAP NetWeaver 7.00 Implementation, SAP User Interface Technology (SAP_UI 7.4, 7.5, 7.51, 7.52). There is little impact as it is not possible to embed active contents such as JavaScript or hyperlinks.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.7885, -0.0462],
#         [ 0.7885,  1.0000, -0.1053],
#         [-0.0462, -0.1053,  1.0000]])

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

Unnamed Dataset

• —Size: 861,607 training samples
• —Columns: <code>anchor</code> and <code>positive</code>
• —Approximate statistics based on the first 100 samples: | | anchor | positive | |:---------|:-----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | | modality | text | text | | details | <ul><li>min: 15 tokens</li><li>mean: 19.66 tokens</li><li>max: 31 tokens</li></ul> | <ul><li>min: 24 tokens</li><li>mean: 84.49 tokens</li><li>max: 256 tokens</li></ul> |
• —Samples: | anchor | positive | |:-----------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>Represent this sentence for searching relevant passages: fuse filesystem sync issues</code> | <code>In the Linux kernel, the following vulnerability has been resolved:<br><br>writeback: don't block sync for filesystems with no data integrity guarantees<br><br>Add a SBINODATAINTEGRITY superblock flag for filesystems that cannot<br>guarantee data persistence on sync (eg fuse). For superblocks with this<br>flag set, sync kicks off writeback of dirty inodes but does not wait<br>for the flusher threads to complete the writeback.<br><br>This replaces the per-inode ASNODATAINTEGRITY mapping flag added in<br>commit f9a49aa302a0 ("fs/writeback: skip ASNODATAINTEGRITY mappings<br>in waitsbinodes()"). The flag belongs at the superblock level because<br>data integrity is a filesystem-wide property, not a per-inode one.<br>Having this flag at the superblock level also allows us to skip having<br>to iterate every dirty inode in waitsbinodes() only to skip each inode<br>individually.<br><br>Prior to this commit, mappings with no data integrity guarantees skipped<br>waiting on writeback completion but still waited on the flusher threads<br>t...</code> | | <code>Represent this sentence for searching relevant passages: EMC Documentum WebTop CSRF issue</code> | <code>Cross-site request forgery (CSRF) vulnerability in EMC Documentum WebTop before 6.8P01, Documentum Administrator through 7.2, Documentum Digital Assets Manager through 6.5SP6, Documentum Web Publishers through 6.5SP7, and Documentum Task Space through 6.7SP2 allows remote attackers to hijack the authentication of arbitrary users. NOTE: this vulnerability exists because of an incomplete fix for CVE-2014-2518.</code> | | <code>Represent this sentence for searching relevant passages: sensitive data exposed on Windows 10</code> | <code>An information disclosure vulnerability exists when the Windows kernel improperly handles objects in memory, aka "Windows Kernel Information Disclosure Vulnerability." This affects Windows 7, Windows Server 2012 R2, Windows RT 8.1, Windows Server 2008, Windows Server 2019, Windows Server 2012, Windows 8.1, Windows Server 2016, Windows Server 2008 R2, Windows 10, Windows 10 Servers.</code> |
• —Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }

Evaluation Dataset

Unnamed Dataset

• —Size: 8,000 evaluation samples
• —Columns: <code>anchor</code> and <code>positive</code>
• —Approximate statistics based on the first 100 samples: | | anchor | positive | |:---------|:-----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | | modality | text | text | | details | <ul><li>min: 14 tokens</li><li>mean: 20.67 tokens</li><li>max: 34 tokens</li></ul> | <ul><li>min: 22 tokens</li><li>mean: 82.34 tokens</li><li>max: 256 tokens</li></ul> |
• —Samples: | anchor | positive | |:-------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>Represent this sentence for searching relevant passages: Why are image titles causing issues in My Album Gallery?</code> | <code>The My Album Gallery plugin for WordPress is vulnerable to Stored Cross-Site Scripting via image titles in all versions up to, and including, 1.0.4. This is due to insufficient input sanitization and output escaping on the 'attachment->title' attribute. This makes it possible for authenticated attackers, with Author-level access and above, to inject arbitrary web scripts in pages that will execute whenever a user accesses an injected page.</code> | | <code>Represent this sentence for searching relevant passages: jenkins bitbucket server integration csrf bypass</code> | <code>Jenkins Bitbucket Server Integration Plugin 2.1.0 through 4.1.3 (both inclusive) allows attackers to craft URLs that would bypass the CSRF protection of any target URL in Jenkins.</code> | | <code>Represent this sentence for searching relevant passages: Android keystore integer overflow issue</code> | <code>Multiple integer overflows in the Blob class in keystore/keystore.cpp in Keystore in Android before 5.1.1 LMY48M allow attackers to execute arbitrary code and read arbitrary Keystore keys via an application that uses a crafted blob in an insert operation, aka internal bug 22802399.</code> |
• —Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }

Training Hyperparameters

Non-Default Hyperparameters

• —per_device_train_batch_size: 16
• —num_train_epochs: 1.0
• —learning_rate: 2e-05
• —warmup_steps: 0.1
• —gradient_accumulation_steps: 2
• —fp16: True
• —per_device_eval_batch_size: 16
• —load_best_model_at_end: True
• —dataloader_drop_last: True

All Hyperparameters

<details><summary>Click to expand</summary>

• —per_device_train_batch_size: 16
• —num_train_epochs: 1.0
• —max_steps: -1
• —learning_rate: 2e-05
• —lr_scheduler_type: linear
• —lr_scheduler_kwargs: None
• —warmup_steps: 0.1
• —optim: adamwtorchfused
• —optim_args: None
• —weight_decay: 0.0
• —adam_beta1: 0.9
• —adam_beta2: 0.999
• —adam_epsilon: 1e-08
• —optim_target_modules: None
• —gradient_accumulation_steps: 2
• —average_tokens_across_devices: True
• —max_grad_norm: 1.0
• —label_smoothing_factor: 0.0
• —bf16: False
• —fp16: True
• —bf16_full_eval: False
• —fp16_full_eval: False
• —tf32: None
• —gradient_checkpointing: False
• —gradient_checkpointing_kwargs: None
• —torch_compile: False
• —torch_compile_backend: None
• —torch_compile_mode: None
• —use_liger_kernel: False
• —liger_kernel_config: None
• —use_cache: False
• —neftune_noise_alpha: None
• —torch_empty_cache_steps: None
• —auto_find_batch_size: False
• —log_on_each_node: True
• —logging_nan_inf_filter: True
• —include_num_input_tokens_seen: no
• —log_level: passive
• —log_level_replica: warning
• —disable_tqdm: False
• —project: huggingface
• —trackio_space_id: None
• —trackio_bucket_id: None
• —trackio_static_space_id: None
• —per_device_eval_batch_size: 16
• —prediction_loss_only: True
• —eval_on_start: False
• —eval_do_concat_batches: True
• —eval_use_gather_object: False
• —eval_accumulation_steps: None
• —include_for_metrics: []
• —batch_eval_metrics: False
• —save_only_model: False
• —save_on_each_node: False
• —enable_jit_checkpoint: False
• —push_to_hub: False
• —hub_private_repo: None
• —hub_model_id: None
• —hub_strategy: every_save
• —hub_always_push: False
• —hub_revision: None
• —load_best_model_at_end: True
• —ignore_data_skip: False
• —restore_callback_states_from_checkpoint: False
• —full_determinism: False
• —seed: 42
• —data_seed: None
• —use_cpu: False
• —accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
• —parallelism_config: None
• —dataloader_drop_last: True
• —dataloader_num_workers: 0
• —dataloader_pin_memory: True
• —dataloader_persistent_workers: False
• —dataloader_prefetch_factor: None
• —remove_unused_columns: True
• —label_names: None
• —train_sampling_strategy: random
• —length_column_name: length
• —ddp_find_unused_parameters: None
• —ddp_bucket_cap_mb: None
• —ddp_broadcast_buffers: False
• —ddp_static_graph: None
• —ddp_backend: None
• —ddp_timeout: 1800
• —fsdp: None
• —fsdp_config: None
• —deepspeed: None
• —debug: []
• —skip_memory_metrics: True
• —do_predict: False
• —resume_from_checkpoint: None
• —warmup_ratio: None
• —local_rank: -1
• —prompts: None
• —batch_sampler: batch_sampler
• —multi_dataset_batch_sampler: proportional
• —router_mapping: {}
• —learning_rate_mapping: {}

</details>

Training Logs

<details><summary>Click to expand</summary>

• —The bold row denotes the saved checkpoint. </details>

Training Time

• —Training: 4.5 days
• —Evaluation: 30.9 minutes
• —Total: 4.5 days

Framework Versions

• —Python: 3.11.9
• —Sentence Transformers: 5.5.1
• —Transformers: 5.10.2
• —PyTorch: 2.12.0+cu126
• —Accelerate: 1.13.0
• —Datasets: 5.0.0
• —Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->