dhammanana/harrier-tipitaka-v1

SentenceTransformer based on microsoft/harrier-oss-v1-270m

This is a sentence-transformers model finetuned from microsoft/harrier-oss-v1-270m. It maps sentences & paragraphs to a 640-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• —Model Type: Sentence Transformer
• —Base model: microsoft/harrier-oss-v1-270m <!-- at revision 31de22b673913c7d658c0f03f792d77c2dcf8ebd -->
• —Maximum Sequence Length: 128 tokens
• —Output Dimensionality: 640 dimensions
• —Similarity Function: Cosine Similarity
• —Supported Modality: Text <!-- - Training Dataset: Unknown --> <!-- - Language: Unknown --> <!-- - License: Unknown -->

Model Sources

• —Documentation: Sentence Transformers Documentation
• —Repository: Sentence Transformers on GitHub
• —Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'embedding_dimension': 640, 'pooling_mode': 'lasttoken', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Instruct: Represent this Buddhist passage for semantic retrieval.\nQuery: Bhagavā taṃ paṭiggahetvā ānandattheraṃ āṇāpesi – ‘‘ imaṃ phalaṃ madditvā pānaṃ dehī ’’ ti. Thero tathā akāsi. Bhagavā ambarasaṃ pivitvā ambaṭṭhiṃ uyyānapālassa datvā ‘‘ imaṃ ropehī ’’ ti āha. So vālukaṃ viyūhitvā taṃ ropesi, ānandatthero kuṇḍikāya udakaṃ āsiñci.',
    'The Blessed One accepted it and instructed Venerable Ānanda: “Crush this fruit and give it as a drink.” The Elder did so. The Blessed One, having drunk the mango juice, gave the mango seed to the gardener and said, “Plant this.” He dug up the sand and planted it, and Venerable Ānanda poured water from a vessel.',
    '“The foremost of my disciples, monks, who are very learned. Who are mindful. Who are clear-headed. Who are resolute. Who are attendants, is Ānanda” –',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 640]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.8314, 0.1940],
#         [0.8314, 1.0000, 0.2902],
#         [0.1940, 0.2902, 1.0000]])

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

Unnamed Dataset

• —Size: 744,489 training samples
• —Columns: <code>anchor</code> and <code>positive</code>
• —Approximate statistics based on the first 1000 samples: | | anchor | positive | |:--------|:------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------| | type | string | string | | details | <ul><li>min: 22 tokens</li><li>mean: 90.93 tokens</li><li>max: 128 tokens</li></ul> | <ul><li>min: 9 tokens</li><li>mean: 55.09 tokens</li><li>max: 128 tokens</li></ul> |
• —Samples: | anchor | positive | |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>Instruct: Represent this Buddhist passage for semantic retrieval.<br>Query: Namo tassa bhagavato arahato sammāsambuddhassa</code> | <code>Homage to the Blessed One, the Perfected One, the Fully Self-Awakened Buddha</code> | | <code>Instruct: Represent this Buddhist passage for semantic retrieval.<br>Query: 1. Evaṃ me sutaṃ – ekaṃ samayaṃ bhagavā sāvatthiyaṃ viharati jetavane anāthapiṇḍikassa ārāme. Tatra kho bhagavā bhikkhū āmantesi – ‘‘ bhikkhavo ’’ ti. ‘‘ Bhadante ’’ ti te bhikkhū bhagavato paccassosuṃ. Bhagavā etadavoca –</code> | <code>Thus have I heard — on one occasion the Blessed One was dwelling at Sāvatthī in Jeta's Grove, in Anāthapiṇḍika’s Park. There the Blessed One addressed the monks: “Monks!” “Venerable sir,” those monks replied to the Blessed One. The Blessed One said:</code> | | <code>Instruct: Represent this Buddhist passage for semantic retrieval.<br>Query: 2.‘‘ Nāhaṃ, bhikkhave, aññaṃ ekasaddampi samanupassāmi yaṃ evaṃ purisassa cittaṃ pariyādāya tiṭṭhati yathayidaṃ, bhikkhave, itthisaddo. Itthisaddo, bhikkhave, purisassa cittaṃ pariyādāya tiṭṭhatī ’’ ti. Dutiyaṃ.</code> | <code>“Monks, I do not see a single sound that invades his mind and remains in a person as much as the sound of a woman. The sound of a woman, monks, overpowers a man’s mind.” The second.</code> |
• —Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }

Evaluation Dataset

Unnamed Dataset

• —Size: 15,180 evaluation samples
• —Columns: <code>anchor</code> and <code>positive</code>
• —Approximate statistics based on the first 1000 samples: | | anchor | positive | |:--------|:------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------| | type | string | string | | details | <ul><li>min: 21 tokens</li><li>mean: 82.69 tokens</li><li>max: 128 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 58.92 tokens</li><li>max: 128 tokens</li></ul> |
• —Samples: | anchor | positive | |:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>Instruct: Represent this Buddhist passage for semantic retrieval.<br>Query: ‘‘ Nāhaṃ, bhikkhave, aññaṃ ekarūpampi samanupassāmi yaṃ evaṃ purisassa cittaṃ pariyādāya tiṭṭhati yathayidaṃ, bhikkhave, itthirūpaṃ. Itthirūpaṃ, bhikkhave, purisassa cittaṃ pariyādāya tiṭṭhatī ’’ ti. Paṭhamaṃ.</code> | <code>“Monks, I do not see a single sight that invades his mind and remains in a person as much as the sight of a woman. The sight of a woman, monks, overpowers a man’s mind.” The first.</code> | | <code>Instruct: Represent this Buddhist passage for semantic retrieval.<br>Query: Paṇihitaacchavaggo pañcamo.</code> | <code>The Chapter on the Directed and Clear is the fifth.</code> | | <code>Instruct: Represent this Buddhist passage for semantic retrieval.<br>Query: 8. Kalyāṇamittādivaggo</code> | <code>8. Chapter on Good Friendship</code> |
• —Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }

Training Hyperparameters

Non-Default Hyperparameters

• —per_device_train_batch_size: 16
• —gradient_accumulation_steps: 2
• —learning_rate: 2e-05
• —num_train_epochs: 1
• —lr_scheduler_type: cosine
• —warmup_steps: 0.05
• —fp16: True
• —remove_unused_columns: False
• —load_best_model_at_end: True
• —batch_sampler: no_duplicates

All Hyperparameters

<details><summary>Click to expand</summary>

• —do_predict: False
• —prediction_loss_only: True
• —per_device_train_batch_size: 16
• —per_device_eval_batch_size: 8
• —gradient_accumulation_steps: 2
• —eval_accumulation_steps: None
• —torch_empty_cache_steps: None
• —learning_rate: 2e-05
• —weight_decay: 0.0
• —adam_beta1: 0.9
• —adam_beta2: 0.999
• —adam_epsilon: 1e-08
• —max_grad_norm: 1.0
• —num_train_epochs: 1
• —max_steps: -1
• —lr_scheduler_type: cosine
• —lr_scheduler_kwargs: None
• —warmup_ratio: None
• —warmup_steps: 0.05
• —log_level: passive
• —log_level_replica: warning
• —log_on_each_node: True
• —logging_nan_inf_filter: True
• —enable_jit_checkpoint: False
• —save_on_each_node: False
• —save_only_model: False
• —restore_callback_states_from_checkpoint: False
• —use_cpu: False
• —seed: 42
• —data_seed: None
• —bf16: False
• —fp16: True
• —bf16_full_eval: False
• —fp16_full_eval: False
• —tf32: None
• —local_rank: -1
• —ddp_backend: None
• —debug: []
• —dataloader_drop_last: False
• —dataloader_num_workers: 0
• —dataloader_prefetch_factor: None
• —disable_tqdm: False
• —remove_unused_columns: False
• —label_names: None
• —load_best_model_at_end: True
• —ignore_data_skip: False
• —fsdp: []
• —fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
• —accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
• —parallelism_config: None
• —deepspeed: None
• —label_smoothing_factor: 0.0
• —optim: adamwtorchfused
• —optim_args: None
• —group_by_length: False
• —length_column_name: length
• —project: huggingface
• —trackio_space_id: trackio
• —ddp_find_unused_parameters: None
• —ddp_bucket_cap_mb: None
• —ddp_broadcast_buffers: False
• —dataloader_pin_memory: True
• —dataloader_persistent_workers: False
• —skip_memory_metrics: True
• —push_to_hub: False
• —resume_from_checkpoint: None
• —hub_model_id: None
• —hub_strategy: every_save
• —hub_private_repo: None
• —hub_always_push: False
• —hub_revision: None
• —gradient_checkpointing: False
• —gradient_checkpointing_kwargs: None
• —include_for_metrics: []
• —eval_do_concat_batches: True
• —auto_find_batch_size: False
• —full_determinism: False
• —ddp_timeout: 1800
• —torch_compile: False
• —torch_compile_backend: None
• —torch_compile_mode: None
• —include_num_input_tokens_seen: no
• —neftune_noise_alpha: None
• —optim_target_modules: None
• —batch_eval_metrics: False
• —eval_on_start: False
• —use_liger_kernel: False
• —liger_kernel_config: None
• —eval_use_gather_object: False
• —average_tokens_across_devices: True
• —use_cache: False
• —prompts: None
• —batch_sampler: no_duplicates
• —multi_dataset_batch_sampler: proportional
• —router_mapping: {}
• —learning_rate_mapping: {}

</details>

Training Logs

<details><summary>Click to expand</summary>

</details>

Training Time

• —Training: 6.1 hours

Framework Versions

• —Python: 3.12.13
• —Sentence Transformers: 5.4.1
• —Transformers: 5.0.0
• —PyTorch: 2.10.0+cu128
• —Accelerate: 1.13.0
• —Datasets: 5.0.0
• —Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->