Modelpublic

Fatin757/ssf-retriever-modernbert-v7

sourceHugging Faceupdated 1y agoView on Hugging Face

0likes17downloads

Model Card

SentenceTransformer based on nomic-ai/modernbert-embed-base

This is a sentence-transformers model finetuned from nomic-ai/modernbert-embed-base on the ssf-train-valid_v7 dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

—Model Type: Sentence Transformer
—Base model: nomic-ai/modernbert-embed-base
—Maximum Sequence Length: 8192 tokens
—Output Dimensionality: 768 dimensions
—Similarity Function: Cosine Similarity
—Training Dataset:
—ssf-train-valid_v7

Model Sources

—Documentation: Sentence Transformers Documentation
—Repository: Sentence Transformers on GitHub
—Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

bash

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Fatin757/ssf-retriever-modernbert-v7")
# Run inference
sentences = [
    "The Managing Director establishes the business strategies for the organisation and develops plans to enable execution of the business strategies. He/She is responsible for tracking market development and trends to inform strategic decision making and ensure the organisation remains current with the changing face of the sector. He leads the organisation's business development efforts to get more projects and grow the business. He also drives the adoption of innovation and new technology to continuously improve the productivity and efficiency of the workforce. The work involves strategic goal setting, business development and business leadership. A significant part of his time goes into external meetings with potential clients for the purpose of business development. He also spends his time developing strategies and plans, and reviewing business and operational performance. He is a strategic thinker and business planner. He is an able leader who guides the organisation and the management in the execution of business plans. He should also be an effective communicator in order to influence external stakeholders.",
    'business strategy, market analysis, strategic planning, business development, leadership, innovation adoption, technology integration, client relationship management, performance review, communication skills',
    'culinary arts, fashion design, gardening, animal care, music theory, painting, carpentry, automotive repair',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.6303, -0.0021],
#         [ 0.6303,  1.0000,  0.0045],
#         [-0.0021,  0.0045,  1.0000]])

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

ssf-train-valid_v7

—Dataset: ssf-train-valid_v7 at 0ec0099
—Size: 7,540 training samples
—Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
—Approximate statistics based on the first 1000 samples: | | anchor | positive | negative | |:--------|:-------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------| | type | string | string | string | | details | <ul><li>min: 57 tokens</li><li>mean: 168.08 tokens</li><li>max: 380 tokens</li></ul> | <ul><li>min: 8 tokens</li><li>mean: 74.36 tokens</li><li>max: 248 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 79.92 tokens</li><li>max: 372 tokens</li></ul> |
—Samples: | anchor | positive | negative | |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>Prop Designers are responsible for identifying and designing appropriate props for a production. They typically work closely with Stage Managers and Set Designers to design and create props that match the style and period of the production. They understand and utilise different tools, methods and materials to create props that look authentic and can produce the desired effects. They are responsible for estimating cost of props and ensuring any purchases and/or rentals fall within the budget. They also manage the prop team's schedule.</code> | <code>The Prop Designer is tasked with selecting and crafting suitable props for theatrical productions. Collaborating closely with Stage Managers and Set Designers, they ensure the props align with the production’s style and era. They apply various tools, techniques, and materials to produce authentic-looking props that achieve the intended visual effects. Additionally, they estimate prop costs and manage procurement or rentals within budget constraints, while overseeing the scheduling of the prop team.</code> | <code>The Retail Store Manager oversees daily retail operations, manages inventory levels, and trains staff to deliver excellent customer service. They ensure the store meets sales targets and maintain a clean, organized shopping environment. The Software Developer designs, codes, and tests software applications. They collaborate with cross-functional teams to develop new features and fix bugs, ensuring the software performs efficiently and meets user requirements. The Human Resources Coordinator assists with recruitment, employee onboarding, and maintaining personnel records. They support HR initiatives and help facilitate employee engagement programs.</code> | | <code>The Area Manager/District Manager oversees the operations of a group of stores in a given area/district. He/she is responsible for developing business opportunities, managing the areas operational and service excellence plans. In addition, he oversees the order fulfilment processes for customers to ensure seamless customer experience across all channels. He is also responsible for driving the organisations innovation and productivity aspirations across the group of stores. He operates in a fast-paced environment where he is required to attend to operational and service excellence issues across a group of stores with varied characteristics. He promotes a positive working culture across stores and drives the achievement of sales results. He is energetic, adaptable, highly-driven and sales-oriented. He also possesses strong people management skills and is able to engage with management and key stakeholders.</code> | <code>The Area Manager/District Manager is responsible for managing multiple store locations within a specified region. This role involves identifying new business opportunities, overseeing operational and customer service standards, and ensuring efficient order fulfillment to provide a consistent customer experience across all sales channels. The manager leads efforts to enhance innovation and productivity throughout the stores, working in a dynamic environment that requires quick resolution of operational challenges. They foster a positive work environment, motivate teams to achieve sales targets, and demonstrate strong leadership and stakeholder engagement abilities.</code> | <code>The Software Developer designs, codes, and tests software applications to meet user requirements. They collaborate with cross-functional teams to develop scalable solutions and maintain existing systems. This role requires proficiency in programming languages, problem-solving skills, and the ability to work in an agile environment. The Graphic Designer creates visual concepts to communicate ideas that inspire, inform, or captivate consumers. They develop layouts for advertisements, brochures, and digital media, using design software and collaborating with marketing teams. The Human Resources Coordinator supports recruitment processes, manages employee records, and assists with training and development programs. They ensure compliance with company policies and foster a positive workplace culture.</code> | | <code>The Cluster Manager oversees the daily operations in the deployment of the team across Centres and ensures the team operates in compliance with all policies. He/she also manages manpower resources, including onboarding and staff development. He possesses strong leadership skills and is able to build and leverage effective relationships with stakeholders. He also drives the overall initiatives for cross-Centre programmes, curricula and quality of learning.</code> | <code>Team management, operational compliance, manpower planning, staff onboarding, leadership skills, stakeholder engagement, cross-Centre program coordination, curriculum development, quality assurance in learning</code> | <code>Graphic design, culinary arts, automotive repair, fashion merchandising, wildlife conservation, dance choreography, marine biology, event photography</code> |
—Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }

Evaluation Dataset

ssf-train-valid_v7

—Dataset: ssf-train-valid_v7 at 0ec0099
—Size: 1,885 evaluation samples
—Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
—Approximate statistics based on the first 1000 samples: | | anchor | positive | negative | |:--------|:------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------| | type | string | string | string | | details | <ul><li>min: 57 tokens</li><li>mean: 168.0 tokens</li><li>max: 403 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 72.26 tokens</li><li>max: 243 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 80.24 tokens</li><li>max: 376 tokens</li></ul> |
—Samples: | anchor | positive | negative | |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>The Assistant Technical Superintendent monitors ship operations and evaluates technical aspects of vessels for maintenance needs. He/She collaborates with vessel operators to develop the proper technical repair plans to address identified maintenance needs, and supervises maintenance procedures to ensure compliance with port rules and regulations, as well as international codes and regulations, including the International Maritime Organisation (IMO) code, International Labour Organisation (ILO) regulations, the International Safety Management (ISM) code, International Ship and Port Facility Security (ISPS) code, Maritime Labour Convention (MLC) regulations, and relevant ISO standards. He is also in-charge of crew-level administration matters. He is flexible and possesses strong initiative and good communication skills</code> | <code>The Assistant Technical Superintendent oversees vessel operations and assesses the technical condition of ships to determine maintenance requirements. He/She works closely with vessel operators to formulate appropriate technical repair plans and supervises maintenance activities to ensure adherence to port regulations and international standards, including IMO, ILO, ISM, ISPS, MLC codes, and applicable ISO standards. Additionally, he manages crew administration tasks and demonstrates flexibility, strong initiative, and effective communication skills.</code> | <code>The Senior Technical Superintendent directs ship operations and leads the technical management of multiple vessels, including strategic planning for fleet maintenance and compliance with international maritime conventions such as SOLAS and MARPOL, while overseeing a team of junior superintendents and engineers. The Assistant Marine Engineer is responsible for monitoring engine performance and mechanical systems on board, coordinating routine machinery maintenance, and ensuring compliance with technical safety standards, including ISO certifications and environmental regulations, but does not handle crew administration. The Port Operations Coordinator manages day-to-day port logistics and vessel scheduling, liaising with shipping agents and port authorities to facilitate cargo handling and berth assignments, focusing on operational efficiency rather than technical ship maintenance or maritime regulatory compliance.</code> | | <code>The Business Intelligence Manager identifies and translates market opportunities into actionable recommendations for the organisation. He/She supervises professionals in gathering and analysing business intelligence (BI) data to help make informed business decisions. He manages the timely reporting of data analysis outcomes and effectively communicates findings, insights and recommendations to business leaders. He develops data and/or information quality metrics and researches new technology and develops business cases to support enterprise wide business intelligence solutions. He is responsible for developing guidelines on data insight reporting for the team. He is also responsible for managing BI-related projects from end to end. He manages a team and is proficient in the analytics tools and techniques required by the organisation. He is also familiar with the relevant software platforms on which the solution is deployed on. The BI Manager has a deep passion for analysing and resolvi...</code> | <code>Business intelligence, data analysis, market opportunity identification, reporting, data quality metrics, analytics tools, BI software platforms, project management, stakeholder engagement, problem-solving, business case development, team management</code> | <code>Culinary arts, fashion design, landscape gardening, automotive repair, creative writing, performing arts, veterinary care, carpentry, event planning, childcare</code> | | <code>The Head of IT Audit develops the organisation's IT audit framework to manage regulatory and operational risks to safeguard IT assets. He/She defines key objectives and guiding principles for the formulation of IT risk management programs, as well as procedures for documenting and updating policies, standards, guidelines relating to the management of IT assets. He advices on the development of IT audit plans and ensures that audit plans comply with regulatory, operational, security risks and relevant internal auditing standards. He oversees the conduct of audits, respective investigations into non-compliance and risks identified from audits. He overlooks new IT policies, systems and processes necessary for enhancing IT controls and mitigate risks. He consults with and advises senior leaders regarding internal controls and security procedures, prepares activity and progress reports relating to the IT audit function. He also guide team members on procedures, technical problems, prioritie...</code> | <code>IT audit framework, regulatory risk management, operational risk management, IT asset safeguarding, IT risk management programs, IT policies and standards, audit planning, compliance with internal auditing standards, IT controls, risk mitigation, internal controls advisory, security procedures, audit investigations, audit reporting, leadership in IT audit, technology risk management, stakeholder influence</code> | <code>Retail sales strategies, customer relationship management, visual merchandising, inventory stocktaking, cashier operations, food service management, hospitality guest services, event planning logistics</code> |
—Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }

Training Hyperparameters

Non-Default Hyperparameters

—eval_strategy: epoch
—per_device_train_batch_size: 32
—per_device_eval_batch_size: 16
—gradient_accumulation_steps: 16
—learning_rate: 2e-05
—num_train_epochs: 5
—lr_scheduler_type: cosine
—warmup_ratio: 0.1
—bf16: True
—tf32: False
—load_best_model_at_end: True
—batch_sampler: no_duplicates

All Hyperparameters

<details><summary>Click to expand</summary>

—overwrite_output_dir: False
—do_predict: False
—eval_strategy: epoch
—prediction_loss_only: True
—per_device_train_batch_size: 32
—per_device_eval_batch_size: 16
—per_gpu_train_batch_size: None
—per_gpu_eval_batch_size: None
—gradient_accumulation_steps: 16
—eval_accumulation_steps: None
—torch_empty_cache_steps: None
—learning_rate: 2e-05
—weight_decay: 0.0
—adam_beta1: 0.9
—adam_beta2: 0.999
—adam_epsilon: 1e-08
—max_grad_norm: 1.0
—num_train_epochs: 5
—max_steps: -1
—lr_scheduler_type: cosine
—lr_scheduler_kwargs: {}
—warmup_ratio: 0.1
—warmup_steps: 0
—log_level: passive
—log_level_replica: warning
—log_on_each_node: True
—logging_nan_inf_filter: True
—save_safetensors: True
—save_on_each_node: False
—save_only_model: False
—restore_callback_states_from_checkpoint: False
—no_cuda: False
—use_cpu: False
—use_mps_device: False
—seed: 42
—data_seed: None
—jit_mode_eval: False
—use_ipex: False
—bf16: True
—fp16: False
—fp16_opt_level: O1
—half_precision_backend: auto
—bf16_full_eval: False
—fp16_full_eval: False
—tf32: False
—local_rank: 0
—ddp_backend: None
—tpu_num_cores: None
—tpu_metrics_debug: False
—debug: []
—dataloader_drop_last: False
—dataloader_num_workers: 0
—dataloader_prefetch_factor: None
—past_index: -1
—disable_tqdm: False
—remove_unused_columns: True
—label_names: None
—load_best_model_at_end: True
—ignore_data_skip: False
—fsdp: []
—fsdp_min_num_params: 0
—fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
—fsdp_transformer_layer_cls_to_wrap: None
—accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
—parallelism_config: None
—deepspeed: None
—label_smoothing_factor: 0.0
—optim: adamwtorchfused
—optim_args: None
—adafactor: False
—group_by_length: False
—length_column_name: length
—ddp_find_unused_parameters: None
—ddp_bucket_cap_mb: None
—ddp_broadcast_buffers: False
—dataloader_pin_memory: True
—dataloader_persistent_workers: False
—skip_memory_metrics: True
—use_legacy_prediction_loop: False
—push_to_hub: False
—resume_from_checkpoint: None
—hub_model_id: None
—hub_strategy: every_save
—hub_private_repo: None
—hub_always_push: False
—hub_revision: None
—gradient_checkpointing: False
—gradient_checkpointing_kwargs: None
—include_inputs_for_metrics: False
—include_for_metrics: []
—eval_do_concat_batches: True
—fp16_backend: auto
—push_to_hub_model_id: None
—push_to_hub_organization: None
—mp_parameters:
—auto_find_batch_size: False
—full_determinism: False
—torchdynamo: None
—ray_scope: last
—ddp_timeout: 1800
—torch_compile: False
—torch_compile_backend: None
—torch_compile_mode: None
—include_tokens_per_second: False
—include_num_input_tokens_seen: False
—neftune_noise_alpha: None
—optim_target_modules: None
—batch_eval_metrics: False
—eval_on_start: False
—use_liger_kernel: False
—liger_kernel_config: None
—eval_use_gather_object: False
—average_tokens_across_devices: False
—prompts: None
—batch_sampler: no_duplicates
—multi_dataset_batch_sampler: proportional
—router_mapping: {}
—learning_rate_mapping: {}

</details>

Training Logs

Epoch	Step	Training Loss	Validation Loss
1.0	15	0.3405	0.0202
2.0	30	0.0262	0.0092
3.0	45	0.0161	0.0071
4.0	60	0.0117	0.0061
5.0	75	0.0116	0.006

—The bold row denotes the saved checkpoint.

Framework Versions

—Python: 3.12.11
—Sentence Transformers: 5.1.1
—Transformers: 4.56.2
—PyTorch: 2.8.0+cu128
—Accelerate: 1.10.0
—Datasets: 4.0.0
—Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

bibtex

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->