AneetaXavier/reformer-pilates-embed-ft-49fc1835-9968-433d-9c45-1538ea91dcc9

SentenceTransformer based on Snowflake/snowflake-arctic-embed-l

This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-l. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• —Model Type: Sentence Transformer
• —Base model: Snowflake/snowflake-arctic-embed-l <!-- at revision d8fb21ca8d905d2832ee8b96c894d3298964346b -->
• —Maximum Sequence Length: 512 tokens
• —Output Dimensionality: 1024 dimensions
• —Similarity Function: Cosine Similarity <!-- - Training Dataset: Unknown --> <!-- - Language: Unknown --> <!-- - License: Unknown -->

Model Sources

• —Documentation: Sentence Transformers Documentation
• —Repository: Sentence Transformers on GitHub
• —Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("AneetaXavier/reformer-pilates-embed-ft-49fc1835-9968-433d-9c45-1538ea91dcc9")
# Run inference
sentences = [
    'What modifications are suggested if the exercise feels too intense on the arms or wrists?',
    "spine relax the shoulders lift the head\nand again\nhead goes down Pike it up inhale\nand exhale roll through\n[Applause]\ngood keep going here if this is too\nintense on the arms or wrists especially\nyou're going to do the same thing here\nPike it up\non your knees and roll through my knees\nare just kind of facing over to the left\nside Pike it up\ninhale and exhale roll\ngood two more you guys you're doing so\ngood it's intense I know\nroll through\nand lift\nlast one\nand finishing that Pike good you guys\ntake those feet\nonto the carriage catch your breath if\nyou want lean it back if you can lift\nyour foot bar to find that click to kind\nof lean back stretch through your\nshoulders kind of depending on your\nreformer if yours is able to pull back",
    "towards the spine but keep the spine in\na neutral position fully straighten the\nlegs when you straighten them and now\ninto VMO knock-knees okay so your toes\nare exactly where they are you push out\nkeeping the knees together go all the\nway back into the stopper and then\nwithin that range you're going to do 20\nof these so the knees are together\nthroughout the whole of the exercise the\ntoes are on the bar as they were in the\nV position but then the heels are out\nwider so it's like a knocked knee this\nreally gets into the muscles on the\ninside of the knees and the inside of\nthe legs in through the nose out through\nthe mouth\nexpanding the ribs and then contracting\nthe abdominals keeping the muscles in\nthe legs engaged throughout prehensile",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

Evaluation

Metrics

Information Retrieval

• —Evaluated with <code>InformationRetrievalEvaluator</code>

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

Unnamed Dataset

• —Size: 120 training samples
• —Columns: <code>sentence0</code> and <code>sentence1</code>
• —Approximate statistics based on the first 120 samples: | | sentence0 | sentence1 | |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------| | type | string | string | | details | <ul><li>min: 12 tokens</li><li>mean: 18.46 tokens</li><li>max: 29 tokens</li></ul> | <ul><li>min: 85 tokens</li><li>mean: 158.07 tokens</li><li>max: 173 tokens</li></ul> |
• —Samples: | sentence0 | sentence1 | |:-----------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>What equipment and spring settings does Dez recommend for starting the Pilates reformer workout?</code> | <code>[Music]<br>foreign<br>[Music]<br>hey guys welcome back to my channel I'm<br>Dez and today I'm taking you through<br>another full body Pilates reformer<br>workout this workout includes some fun<br>and challenging series and will give you<br>a full class experience you won't need<br>any additional props today just you and<br>your reformer so let's get started<br>okay you guys we're going to start today<br>on two heavy Springs with hip rolls so<br>if you need additional assistance for<br>your low back add on also a light to<br>medium tension spring I'm going to be<br>going to two heavy Springs or two Reds<br>on this machine<br>and we're going to light on on our box<br>head rest will be down flat<br>to protect the neck<br>good we're going to place our heels on<br>the bar<br>find your neutral spine</code> | | <code>How does Dez suggest protecting the neck during the hip rolls exercise?</code> | <code>[Music]<br>foreign<br>[Music]<br>hey guys welcome back to my channel I'm<br>Dez and today I'm taking you through<br>another full body Pilates reformer<br>workout this workout includes some fun<br>and challenging series and will give you<br>a full class experience you won't need<br>any additional props today just you and<br>your reformer so let's get started<br>okay you guys we're going to start today<br>on two heavy Springs with hip rolls so<br>if you need additional assistance for<br>your low back add on also a light to<br>medium tension spring I'm going to be<br>going to two heavy Springs or two Reds<br>on this machine<br>and we're going to light on on our box<br>head rest will be down flat<br>to protect the neck<br>good we're going to place our heels on<br>the bar<br>find your neutral spine</code> | | <code>What is the correct breathing technique to use while rocking between imprint and neutral spine positions?</code> | <code>heels on the bar hip distance there we<br>go inhale<br>exhale we're just going to tuck into our<br>imprint<br>pressing that low back down activating<br>the core and inhale Rock back<br>and exhale press that low back down<br>going into your imprinted spine and then<br>rocking back to your neutral<br>good keep that breathing going we're<br>thinking just ribs towards your hips as<br>you rock into that imprint and then Rock<br>back<br>one more time<br>and rock it back this time we're going<br>to roll all the way up press that low<br>back down and then scoop the hips use<br>the hamstrings and glutes to roll up we<br>want to keep the carriage into the<br>stopper that's the challenging part<br>inhale and then exhale soften from the<br>ribs and roll back down one vertebrae at</code> |
• —Loss: <code>MatryoshkaLoss</code> with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }

Training Hyperparameters

Non-Default Hyperparameters

• —eval_strategy: steps
• —per_device_train_batch_size: 10
• —per_device_eval_batch_size: 10
• —num_train_epochs: 30
• —multi_dataset_batch_sampler: round_robin

All Hyperparameters

<details><summary>Click to expand</summary>

• —overwrite_output_dir: False
• —do_predict: False
• —eval_strategy: steps
• —prediction_loss_only: True
• —per_device_train_batch_size: 10
• —per_device_eval_batch_size: 10
• —per_gpu_train_batch_size: None
• —per_gpu_eval_batch_size: None
• —gradient_accumulation_steps: 1
• —eval_accumulation_steps: None
• —torch_empty_cache_steps: None
• —learning_rate: 5e-05
• —weight_decay: 0.0
• —adam_beta1: 0.9
• —adam_beta2: 0.999
• —adam_epsilon: 1e-08
• —max_grad_norm: 1
• —num_train_epochs: 30
• —max_steps: -1
• —lr_scheduler_type: linear
• —lr_scheduler_kwargs: {}
• —warmup_ratio: 0.0
• —warmup_steps: 0
• —log_level: passive
• —log_level_replica: warning
• —log_on_each_node: True
• —logging_nan_inf_filter: True
• —save_safetensors: True
• —save_on_each_node: False
• —save_only_model: False
• —restore_callback_states_from_checkpoint: False
• —no_cuda: False
• —use_cpu: False
• —use_mps_device: False
• —seed: 42
• —data_seed: None
• —jit_mode_eval: False
• —use_ipex: False
• —bf16: False
• —fp16: False
• —fp16_opt_level: O1
• —half_precision_backend: auto
• —bf16_full_eval: False
• —fp16_full_eval: False
• —tf32: None
• —local_rank: 0
• —ddp_backend: None
• —tpu_num_cores: None
• —tpu_metrics_debug: False
• —debug: []
• —dataloader_drop_last: False
• —dataloader_num_workers: 0
• —dataloader_prefetch_factor: None
• —past_index: -1
• —disable_tqdm: False
• —remove_unused_columns: True
• —label_names: None
• —load_best_model_at_end: False
• —ignore_data_skip: False
• —fsdp: []
• —fsdp_min_num_params: 0
• —fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
• —tp_size: 0
• —fsdp_transformer_layer_cls_to_wrap: None
• —accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
• —deepspeed: None
• —label_smoothing_factor: 0.0
• —optim: adamw_torch
• —optim_args: None
• —adafactor: False
• —group_by_length: False
• —length_column_name: length
• —ddp_find_unused_parameters: None
• —ddp_bucket_cap_mb: None
• —ddp_broadcast_buffers: False
• —dataloader_pin_memory: True
• —dataloader_persistent_workers: False
• —skip_memory_metrics: True
• —use_legacy_prediction_loop: False
• —push_to_hub: False
• —resume_from_checkpoint: None
• —hub_model_id: None
• —hub_strategy: every_save
• —hub_private_repo: None
• —hub_always_push: False
• —gradient_checkpointing: False
• —gradient_checkpointing_kwargs: None
• —include_inputs_for_metrics: False
• —include_for_metrics: []
• —eval_do_concat_batches: True
• —fp16_backend: auto
• —push_to_hub_model_id: None
• —push_to_hub_organization: None
• —mp_parameters:
• —auto_find_batch_size: False
• —full_determinism: False
• —torchdynamo: None
• —ray_scope: last
• —ddp_timeout: 1800
• —torch_compile: False
• —torch_compile_backend: None
• —torch_compile_mode: None
• —include_tokens_per_second: False
• —include_num_input_tokens_seen: False
• —neftune_noise_alpha: None
• —optim_target_modules: None
• —batch_eval_metrics: False
• —eval_on_start: False
• —use_liger_kernel: False
• —eval_use_gather_object: False
• —average_tokens_across_devices: False
• —prompts: None
• —batch_sampler: batch_sampler
• —multi_dataset_batch_sampler: round_robin

</details>

Training Logs

Framework Versions

• —Python: 3.11.12
• —Sentence Transformers: 4.1.0
• —Transformers: 4.51.3
• —PyTorch: 2.6.0+cu124
• —Accelerate: 1.6.0
• —Datasets: 2.14.4
• —Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->