Modelpublic

auukjkjk/poem_embedding_model

sourceHugging Faceupdated 3mo agoView on Hugging Face

1likes7downloads

Model Card

SentenceTransformer based on sentence-transformers/all-mpnet-base-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-mpnet-base-v2. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

—Model Type: Sentence Transformer
—Base model: sentence-transformers/all-mpnet-base-v2
—Maximum Sequence Length: 384 tokens
—Output Dimensionality: 768 dimensions
—Similarity Function: Cosine Similarity
—Supported Modality: Text
—Training Dataset: Poemclassification - traindata.csv & Poemclassification - testdata.csv from kaggle
—Loss : BatchAllTripletLoss
—Fine tuning type : supervised metric learning

Model Sources

—Documentation: Sentence Transformers Documentation
—Repository: Sentence Transformers on GitHub
—Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'MPNetModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

bash

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("auukjkjk/poem_embedding_model")
# Run inference
sentences = [
    'Six months ago, the measuring of whiskey left in the jug, urine on the mattress, couch cushions, the crotch of pants in wear. You watch how breath lifts a chest, how a person breathes— sick hobbies of when we must. You watch how you become illiterate at counting. Six or seven broken breathalyzers; a joke formulates in',
    'Music',
    'Affection',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.2384, -0.0032],
#         [ 0.2384,  1.0000,  0.1950],
#         [-0.0032,  0.1950,  1.0000]])

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

Unnamed Dataset

—Size: 991 training samples
—Columns: <code>sentence</code> and <code>label</code>
—Approximate statistics based on the first 100 samples: | | sentence | label | |:---------|:-----------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------| | type | string | int | | modality | text | | | details | <ul><li>min: 11 tokens</li><li>mean: 65.32 tokens</li><li>max: 97 tokens</li></ul> | <ul><li>0: ~27.88%</li><li>1: ~26.92%</li><li>2: ~25.00%</li><li>3: ~20.19%</li></ul> |
—Samples: | sentence | label | |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------| | <code>What is it you feel I asked Kurt when you listen toRavel’s String Quartet in F-major, his face was so lit upand I wondered, “the music is unlike the world I liveor think in, it’s from somewhere else, unfamiliar and unknown,not because it is relevant to the familiar and comfortable,but</code> | <code>3</code> | | <code>I have been a spendthrift Dropping from lazy fingers Quiet coloured hours, Fluttering away from me Like oak and beech leaves in October.I have lived keenly and wastefully, Like a bush or a sun insect— Lived sensually and thoughtfully, Loving the flesh and the beauty of this world— Green ivy about ruined towers, The out-pouring of the grey sea, And</code> | <code>1</code> | | <code>makes me think plurality. Maybe I can love you with many selves. Or. I love all the Porgys. Even as a colloquialism: a queering of love as singular. English is a strange language because I loves and He loves are not both grammarly. I loves you, Porgy. Better to ask what man is not, Porgy. The beauty of Nina’s</code> | <code>0</code> |
—Loss: <code>BatchAllTripletLoss</code> with these parameters:

json

  {
      "distance_metric": "euclidean_distance",
      "margin": 5
  }

Training Hyperparameters

Non-Default Hyperparameters

—per_device_train_batch_size: 32
—num_train_epochs: 5
—learning_rate: 2e-05
—warmup_steps: 0.1

All Hyperparameters

<details><summary>Click to expand</summary>

—per_device_train_batch_size: 32
—num_train_epochs: 5
—max_steps: -1
—learning_rate: 2e-05
—lr_scheduler_type: linear
—lr_scheduler_kwargs: None
—warmup_steps: 0.1
—optim: adamwtorchfused
—optim_args: None
—weight_decay: 0.0
—adam_beta1: 0.9
—adam_beta2: 0.999
—adam_epsilon: 1e-08
—optim_target_modules: None
—gradient_accumulation_steps: 1
—average_tokens_across_devices: True
—max_grad_norm: 1.0
—label_smoothing_factor: 0.0
—bf16: False
—fp16: False
—bf16_full_eval: False
—fp16_full_eval: False
—tf32: None
—gradient_checkpointing: False
—gradient_checkpointing_kwargs: None
—torch_compile: False
—torch_compile_backend: None
—torch_compile_mode: None
—use_liger_kernel: False
—liger_kernel_config: None
—use_cache: False
—neftune_noise_alpha: None
—torch_empty_cache_steps: None
—auto_find_batch_size: False
—log_on_each_node: True
—logging_nan_inf_filter: True
—include_num_input_tokens_seen: no
—log_level: passive
—log_level_replica: warning
—disable_tqdm: False
—project: huggingface
—trackio_space_id: None
—trackio_bucket_id: None
—trackio_static_space_id: None
—per_device_eval_batch_size: 8
—prediction_loss_only: True
—eval_on_start: False
—eval_do_concat_batches: True
—eval_use_gather_object: False
—eval_accumulation_steps: None
—include_for_metrics: []
—batch_eval_metrics: False
—save_only_model: False
—save_on_each_node: False
—enable_jit_checkpoint: False
—push_to_hub: False
—hub_private_repo: None
—hub_model_id: None
—hub_strategy: every_save
—hub_always_push: False
—hub_revision: None
—load_best_model_at_end: False
—ignore_data_skip: False
—restore_callback_states_from_checkpoint: False
—full_determinism: False
—seed: 42
—data_seed: None
—use_cpu: False
—accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
—parallelism_config: None
—dataloader_drop_last: False
—dataloader_num_workers: 0
—dataloader_pin_memory: True
—dataloader_persistent_workers: False
—dataloader_prefetch_factor: None
—remove_unused_columns: True
—label_names: None
—train_sampling_strategy: random
—length_column_name: length
—ddp_find_unused_parameters: None
—ddp_bucket_cap_mb: None
—ddp_broadcast_buffers: False
—ddp_static_graph: None
—ddp_backend: None
—ddp_timeout: 1800
—fsdp: None
—fsdp_config: None
—deepspeed: None
—debug: []
—skip_memory_metrics: True
—do_predict: False
—resume_from_checkpoint: None
—warmup_ratio: None
—local_rank: -1
—prompts: None
—batch_sampler: batch_sampler
—multi_dataset_batch_sampler: proportional
—router_mapping: {}
—learning_rate_mapping: {}

</details>

Training Logs

Epoch	Step	Training Loss
0.3226	10	4.9797
0.6452	20	4.9687
0.9677	30	4.9419
1.2903	40	4.8351
1.6129	50	4.8372
1.9355	60	4.7127
2.2581	70	4.6246
2.5806	80	4.7205
2.9032	90	4.6340
3.2258	100	4.4178
3.5484	110	4.5711
3.8710	120	4.5545
4.1935	130	4.4915
4.5161	140	4.4780
4.8387	150	4.4197

Training Time

—Training: 2.4 days

Framework Versions

—Python: 3.12.13
—Sentence Transformers: 5.6.0
—Transformers: 5.12.1
—PyTorch: 2.11.0+cpu
—Accelerate: 1.14.0
—Datasets: 5.0.0
—Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

BatchAllTripletLoss

bibtex

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->