Modelpublic

shuaih777/music-crs-biencoder-personalized

sourceHugging Faceupdated 3mo agoView on Hugging Face

0likes73downloads

Model Card

SentenceTransformer based on BAAI/bge-large-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-large-en-v1.5. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

—Model Type: Sentence Transformer
—Base model: BAAI/bge-large-en-v1.5
—Maximum Sequence Length: 512 tokens
—Output Dimensionality: 1024 dimensions
—Similarity Function: Cosine Similarity

Model Sources

—Documentation: Sentence Transformers Documentation
—Repository: Sentence Transformers on GitHub
—Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

bash

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    '[20s | female | Germany]\nUser: Suggest some classic pop or dance tracks that bring back good memories.\nAssistant: Awesome! Glad you enjoyed "Thriller"! If you\'re looking to keep that energetic vibe going, how about\nUser: Oh, the Spice Girls! That definitely brings back memories, classic 90s pop. I like the energetic vibe. Can you suggest something similar, maybe still 90s but with a bit more of a dance beat?\nAssistant: Awesome, so glad you loved the Spice Girls! Sticking with that fun, upbeat 90s vibe, how about "MMMB\nUser: MMMBop is super nostalgic and fun, I remember that one! But it\'s not quite hitting that dance beat I was looking for. Could we try something with a stronger rhythm, still from the 90s or early 2000s, that\'s more geared for dancing?',
    'The Nights by Avicii from The Days / Nights, Full Circle: The Live Anthology [Dance, 2014 releases, 2014 single, Pop, pop dance, house, playlista rmf fm, epic, 2014, 2010s, dance, hino, chillout, robbie williams, avicii fig] (2014)',
    'Ziggy Stardust - 2012 Remastered Version by David Bowie from The Rise And Fall Of Ziggy Stardust And The Spiders From Mars, The Rise And Fall Of Ziggy Stardust And The Spiders From Mars (2012 Remastered Version) [Pop, classic rock] (1972)',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

Unnamed Dataset

—Size: 121,592 training samples
—Columns: <code>sentence0</code> and <code>sentence1</code>
—Approximate statistics based on the first 1000 samples: | | sentence0 | sentence1 | |:--------|:-------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | | details | <ul><li>min: 19 tokens</li><li>mean: 319.55 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 15 tokens</li><li>mean: 61.09 tokens</li><li>max: 112 tokens</li></ul> |
—Samples: | sentence0 | sentence1 | |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>[30s | male | Australia] User: Can you help me find popular Christian music from around 2000-2010 that fans of the genre were really into, not just mainstream hits? Assistant: Awesome, glad you liked "Who We Are"! Since you're enjoying the uplifting vibe, how about "Miracles" User: That's a good one, "Miracles" by Newsboys definitely fits the Christian rock vibe and timeframe I'm looking for. Can you give me some more popular Christian rock and also some Christian hip-hop from that 2000-2010 era, like something really well-known by TobyMac or Lecrae? Assistant: Glad you enjoyed "Miracles"! If you liked that beat and uplifting vibe, you've got to check out "Jes User: These Christian rock suggestions like 'Miracles' and 'Jesus Freak' are exactly what I was looking for in terms of community favorites from that early 2000s-2010s era! Thank you for that. Now, could you give me some strong Christian hip-hop recommendations from that same period, perhaps by TobyMac or Lecrae, or similar artis...</code> | <code>Ignition by tobyMac from Portable Sounds With Bonus Remixes [christian hard rock, hard rock, cross rhythms, pump it up, lurv eet, very favorite, alternative rock, elimination chamber, christian rapcore, tobymac, cross2life, awesome, wwe ppv theme, epic, rock] (2008)</code> | | <code>[20s | male | Brazil] User: I'm trying to remember a really popular Latin song from the 2000s, it had a unique sound and was everywhere. Assistant: Awesome, glad you liked it! Let's keep the Legião Urbana train rolling. How about "Andrea Doria" nex User: I love Legião Urbana, thanks for that! But I'm actually looking for something different. I'm trying to remember a Latin pop-rock song, definitely from the 2000s, with a very recognizable female vocalist. Assistant: Awesome! Glad to hear you're liking the Legião Urbana picks. Let's keep it going with "Os Barcos," a User: Okay, I really do love Legião Urbana, but I need to clarify – I'm looking for a different kind of Latin music, not Brazilian rock right now. Think more broadly Latin America, and definitely a pop-rock sound from the 2000s, with a prominent female singer.</code> | <code>Antes Das Seis by Legião Urbana from Mais Do Mesmo, Uma Outra Estação [pop rock nacional, rock alternativo, lyrics i love, uma outra estacao, rock brasilero, rockpop, 90s, brazilian rock, boo hoo, seis, balada, rock, para se matar de triste, canto junto, 10s collections] (1997)</code> | | <code>[60+ | female | Poland] User: What are some popular alternative rock songs I might know from the 2000s or 2010s? Assistant: Absolutely! Speaking of popular alternative rock from the 2010s, you might know "Sweater Weather" by User: Yes, that's a good one, very recognizable! Are there any other tracks that were big hits for alternative bands around that time, perhaps something with a strong, recognizable chorus?</code> | <code>Sex on Fire by Kings of Leon from Balance Le Son, Only By The Night [usa, energetic, fucking awesome, alternative rock, 00s, obsessive repeated listening, amazing, completely awesome, tunes i have played for hours, catchy, favourites, gossip girl, american, british, awesome] (2008)</code> |
—Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

json

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }

Training Hyperparameters

Non-Default Hyperparameters

—per_device_train_batch_size: 64
—per_device_eval_batch_size: 64
—multi_dataset_batch_sampler: round_robin

All Hyperparameters

<details><summary>Click to expand</summary>

—overwrite_output_dir: False
—do_predict: False
—eval_strategy: no
—prediction_loss_only: True
—per_device_train_batch_size: 64
—per_device_eval_batch_size: 64
—per_gpu_train_batch_size: None
—per_gpu_eval_batch_size: None
—gradient_accumulation_steps: 1
—eval_accumulation_steps: None
—torch_empty_cache_steps: None
—learning_rate: 5e-05
—weight_decay: 0.0
—adam_beta1: 0.9
—adam_beta2: 0.999
—adam_epsilon: 1e-08
—max_grad_norm: 1
—num_train_epochs: 3
—max_steps: -1
—lr_scheduler_type: linear
—lr_scheduler_kwargs: {}
—warmup_ratio: 0.0
—warmup_steps: 0
—log_level: passive
—log_level_replica: warning
—log_on_each_node: True
—logging_nan_inf_filter: True
—save_safetensors: True
—save_on_each_node: False
—save_only_model: False
—restore_callback_states_from_checkpoint: False
—no_cuda: False
—use_cpu: False
—use_mps_device: False
—seed: 42
—data_seed: None
—jit_mode_eval: False
—use_ipex: False
—bf16: False
—fp16: False
—fp16_opt_level: O1
—half_precision_backend: auto
—bf16_full_eval: False
—fp16_full_eval: False
—tf32: None
—local_rank: 0
—ddp_backend: None
—tpu_num_cores: None
—tpu_metrics_debug: False
—debug: []
—dataloader_drop_last: False
—dataloader_num_workers: 0
—dataloader_prefetch_factor: None
—past_index: -1
—disable_tqdm: False
—remove_unused_columns: True
—label_names: None
—load_best_model_at_end: False
—ignore_data_skip: False
—fsdp: []
—fsdp_min_num_params: 0
—fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
—tp_size: 0
—fsdp_transformer_layer_cls_to_wrap: None
—accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
—deepspeed: None
—label_smoothing_factor: 0.0
—optim: adamw_torch
—optim_args: None
—adafactor: False
—group_by_length: False
—length_column_name: length
—ddp_find_unused_parameters: None
—ddp_bucket_cap_mb: None
—ddp_broadcast_buffers: False
—dataloader_pin_memory: True
—dataloader_persistent_workers: False
—skip_memory_metrics: True
—use_legacy_prediction_loop: False
—push_to_hub: False
—resume_from_checkpoint: None
—hub_model_id: None
—hub_strategy: every_save
—hub_private_repo: None
—hub_always_push: False
—gradient_checkpointing: False
—gradient_checkpointing_kwargs: None
—include_inputs_for_metrics: False
—include_for_metrics: []
—eval_do_concat_batches: True
—fp16_backend: auto
—push_to_hub_model_id: None
—push_to_hub_organization: None
—mp_parameters:
—auto_find_batch_size: False
—full_determinism: False
—torchdynamo: None
—ray_scope: last
—ddp_timeout: 1800
—torch_compile: False
—torch_compile_backend: None
—torch_compile_mode: None
—dispatch_batches: None
—split_batches: None
—include_tokens_per_second: False
—include_num_input_tokens_seen: False
—neftune_noise_alpha: None
—optim_target_modules: None
—batch_eval_metrics: False
—eval_on_start: False
—use_liger_kernel: False
—eval_use_gather_object: False
—average_tokens_across_devices: False
—prompts: None
—batch_sampler: batch_sampler
—multi_dataset_batch_sampler: round_robin

</details>

Training Logs

Epoch	Step	Training Loss
0.2632	500	2.0753
0.5263	1000	1.4308
0.7895	1500	1.3659
1.0526	2000	1.3324
1.3158	2500	1.3118
1.5789	3000	1.2938
1.8421	3500	1.3031
2.1053	4000	1.2948
2.3684	4500	1.2999
2.6316	5000	1.2905
2.8947	5500	1.262

Framework Versions

—Python: 3.11.10
—Sentence Transformers: 3.3.1
—Transformers: 4.50.3
—PyTorch: 2.4.1+cu124
—Accelerate: 1.14.0
—Datasets: 5.0.0
—Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

bibtex

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->