philipp-zettl/gte-micro-v4-mtg

SentenceTransformer based on Mihaiii/gte-micro-v4

This is a sentence-transformers model finetuned from Mihaiii/gte-micro-v4 on the mtg_cards-2025-04-04 dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• —Model Type: Sentence Transformer
• —Base model: Mihaiii/gte-micro-v4 <!-- at revision 78e1a4b348f8524c3ab2e3e3475788f5adb8c98f -->
• —Maximum Sequence Length: 512 tokens
• —Output Dimensionality: 384 dimensions
• —Similarity Function: Cosine Similarity
• —Training Dataset:
• —mtg_cards-2025-04-04
• —Language: en <!-- - License: Unknown -->

Model Sources

• —Documentation: Sentence Transformers Documentation
• —Repository: Sentence Transformers on GitHub
• —Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("philipp-zettl/gte-micro-v4-mtg")
# Run inference
sentences = [
    '141a031d-f899-497b-adf7-4af142078085_0367fac8-6990-4544-ac7d-ed363b55a9cf',
    "Title: Quirion Explorer\nCost: {1}{G}\nColors: ['G']\nType: Creature — Elf Druid Scout\nDesc: {T}: Add one mana of any color that a land an opponent controls could produce.",
    "Title: Savage Hunger\nCost: {2}{G}\nColors: ['G']\nType: Enchantment — Aura\nDesc: Enchant creature\nEnchanted creature gets +1/+0 and has trample.\nCycling {2} ({2}, Discard this card: Draw a card.)",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

Evaluation

Metrics

Semantic Similarity

• —Datasets: sts-dev and sts-test
• —Evaluated with <code>EmbeddingSimilarityEvaluator</code>

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

mtg_cards-2025-04-04

• —Dataset: mtg_cards-2025-04-04 at a35ccc4
• —Size: 2,839,738 training samples
• —Columns: <code>uuid</code>, <code>sentence1</code>, <code>sentence2</code>, <code>image1</code>, <code>image2</code>, and <code>score</code>
• —Approximate statistics based on the first 1000 samples: | | uuid | sentence1 | sentence2 | image1 | image2 | score | |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------| | type | string | string | string | string | string | float | | details | <ul><li>min: 49 tokens</li><li>mean: 56.99 tokens</li><li>max: 65 tokens</li></ul> | <ul><li>min: 17 tokens</li><li>mean: 69.4 tokens</li><li>max: 180 tokens</li></ul> | <ul><li>min: 15 tokens</li><li>mean: 68.59 tokens</li><li>max: 166 tokens</li></ul> | <ul><li>min: 53 tokens</li><li>mean: 58.17 tokens</li><li>max: 64 tokens</li></ul> | <ul><li>min: 52 tokens</li><li>mean: 58.28 tokens</li><li>max: 64 tokens</li></ul> | <ul><li>min: -1.0</li><li>mean: -0.43</li><li>max: 0.5</li></ul> |
• —Samples: | uuid | sentence1 | sentence2 | image1 | image2 | score | |:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------|:------------------| | <code>08f9b863-10b7-46d6-badd-97381e6c7c5e4330efa7-a11b-4776-9fb0-1cae8aed67b1</code> | <code>Title: Blast Zone<br>Type: Land<br>Desc: This land enters with a charge counter on it.<br>{T}: Add {C}.<br>{X}{X}, {T}: Put X charge counters on this land.<br>{3}, {T}, Sacrifice this land: Destroy each nonland permanent with mana value equal to the number of charge counters on this land.</code> | <code>Title: Tom van de Logt Bio (2000)<br>Type: Card<br>Desc: Quarterfinalist Tom van de Logt posted a perfect 6—0 record during the Standard portion of this year's World Championships. The 19-year-old Groesbeek, Holland native was playing a deck that had a big impact on the metagame this year, "Replenish." This deck used cards like Attunement and Frantic Search to put powerful enchantments, such as Parallax Wave and Opalescence, into the graveyard and then used Replenish to put them all back into play at once.</code> | <code>https://cards.scryfall.io/normal/front/0/8/08f9b863-10b7-46d6-badd-97381e6c7c5e.jpg?1674423042</code> | <code>https://cards.scryfall.io/normal/front/4/3/4330efa7-a11b-4776-9fb0-1cae8aed67b1.jpg?1562767017</code> | <code>0.25</code> | | <code>abe9cf1e-d398-41e0-8b11-afe1015e4fd940cb67f7-b4e1-423b-8f55-d44ed383e778</code> | <code>Title: Coral Net<br>Cost: {U}<br>Colors: ['U']<br>Type: Enchantment — Aura<br>Desc: Enchant green or white creature<br>Enchanted creature has "At the beginning of your upkeep, sacrifice this creature unless you discard a card."</code> | <code>Title: Silumgar Butcher<br>Cost: {4}{B}<br>Colors: ['B']<br>Type: Creature — Zombie Djinn<br>Desc: Exploit (When this creature enters, you may sacrifice a creature.)<br>When this creature exploits a creature, target creature gets -3/-3 until end of turn.</code> | <code>https://cards.scryfall.io/normal/front/a/b/abe9cf1e-d398-41e0-8b11-afe1015e4fd9.jpg?1562631469</code> | <code>https://cards.scryfall.io/normal/front/4/0/40cb67f7-b4e1-423b-8f55-d44ed383e778.jpg?1562785294</code> | <code>-1.0</code> | | <code>3dd13408-b4db-42e7-bf3c-d46716538a7c_05a6dc90-3997-4911-8bd6-854c85eca35b</code> | <code>Title: Rishadan Brigand<br>Cost: {4}{U}<br>Colors: ['U']<br>Type: Creature — Human Pirate<br>Desc: Flying<br>When this creature enters, each opponent sacrifices a permanent of their choice unless they pay {3}.<br>This creature can block only creatures with flying.</code> | <code>Title: Banishing Stroke<br>Cost: {5}{W}<br>Colors: ['W']<br>Type: Instant<br>Desc: Put target artifact, creature, or enchantment on the bottom of its owner's library.<br>Miracle {W} (You may cast this card for its miracle cost when you draw it if it's the first card you drew this turn.)</code> | <code>https://cards.scryfall.io/normal/front/3/d/3dd13408-b4db-42e7-bf3c-d46716538a7c.jpg?1632145390</code> | <code>https://cards.scryfall.io/normal/front/0/5/05a6dc90-3997-4911-8bd6-854c85eca35b.jpg?1723433851</code> | <code>-1.0</code> |
• —Loss: <code>CosineSimilarityLoss</code> with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss"
  }

Evaluation Dataset

mtg_cards-2025-04-04

• —Dataset: mtg_cards-2025-04-04 at a35ccc4
• —Size: 74,730 evaluation samples
• —Columns: <code>uuid</code>, <code>sentence1</code>, <code>sentence2</code>, <code>image1</code>, <code>image2</code>, and <code>score</code>
• —Approximate statistics based on the first 1000 samples: | | uuid | sentence1 | sentence2 | image1 | image2 | score | |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:------------------------------------------------------------------| | type | string | string | string | string | string | float | | details | <ul><li>min: 50 tokens</li><li>mean: 56.9 tokens</li><li>max: 65 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 68.44 tokens</li><li>max: 181 tokens</li></ul> | <ul><li>min: 15 tokens</li><li>mean: 69.49 tokens</li><li>max: 179 tokens</li></ul> | <ul><li>min: 52 tokens</li><li>mean: 58.22 tokens</li><li>max: 64 tokens</li></ul> | <ul><li>min: 52 tokens</li><li>mean: 58.21 tokens</li><li>max: 64 tokens</li></ul> | <ul><li>min: -1.0</li><li>mean: -0.44</li><li>max: 0.75</li></ul> |
• —Samples: | uuid | sentence1 | sentence2 | image1 | image2 | score | |:---------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------|:------------------| | <code>6bdd8645-aee9-44cb-acaa-2674f55cdf2fb34bb149-2e50-462e-8b83-5c8339bb3aff</code> | <code>Title: Syr Cadian, Knight Owl<br>Cost: {3}{W}{W}<br>Colors: ['W']<br>Type: Legendary Creature — Bird Knight<br>Desc: Knightlifelink (Damage dealt by Knights you control also causes you to gain that much life.)<br>{W}: Syr Cadian gains vigilance until end of turn. Activate only from sunrise to sunset.<br>{B}: Syr Cadian gains flying until end of turn. Activate only from sunset to sunrise.</code> | <code>Title: Non-Human Cannonball<br>Cost: {2}{R}<br>Colors: ['R']<br>Type: Artifact Creature — Clown Robot<br>Desc: When this creature dies, roll a six-sided die. If the result is 4 or less, this creature deals that much damage to you.</code> | <code>https://cards.scryfall.io/normal/front/6/b/6bdd8645-aee9-44cb-acaa-2674f55cdf2f.jpg?1664317187</code> | <code>https://cards.scryfall.io/normal/front/b/3/b34bb149-2e50-462e-8b83-5c8339bb3aff.jpg?1673917877</code> | <code>0.25</code> | | <code>860f4304-38f1-4c2f-a122-2590619522fd08d6db9b-b2da-4148-aa49-8c2fecac6e32</code> | <code>Title: Hindering Light<br>Cost: {W}{U}<br>Colors: ['U', 'W']<br>Type: Instant<br>Desc: Counter target spell that targets you or a permanent you control.<br>Draw a card.</code> | <code>Title: Gleam of Resistance<br>Cost: {4}{W}<br>Colors: ['W']<br>Type: Instant<br>Desc: Creatures you control get +1/+2 until end of turn. Untap those creatures.<br>Basic landcycling {1}{W} ({1}{W}, Discard this card: Search your library for a basic land card, reveal it, put it into your hand, then shuffle.)</code> | <code>https://cards.scryfall.io/normal/front/8/6/860f4304-38f1-4c2f-a122-2590619522fd.jpg?1712353583</code> | <code>https://cards.scryfall.io/normal/front/0/8/08d6db9b-b2da-4148-aa49-8c2fecac6e32.jpg?1573505575</code> | <code>0.25</code> | | <code>91b448f4-aa0c-42c7-a771-e8dd20e0520c_46f810c2-310e-42f5-ab1f-d56396cf5124</code> | <code>Title: Practiced Tactics<br>Cost: {W}<br>Colors: ['W']<br>Type: Instant<br>Desc: Choose target attacking or blocking creature. Practiced Tactics deals damage to that creature equal to twice the number of creatures in your party. (Your party consists of up to one each of Cleric, Rogue, Warrior, and Wizard.)</code> | <code>Title: Anointer Priest<br>Cost: {1}{W}<br>Colors: ['W']<br>Type: Creature — Human Cleric<br>Desc: Whenever a creature token you control enters, you gain 1 life.<br>Embalm {3}{W} ({3}{W}, Exile this card from your graveyard: Create a token that's a copy of it, except it's a white Zombie Human Cleric with no mana cost. Embalm only as a sorcery.)</code> | <code>https://cards.scryfall.io/normal/front/9/1/91b448f4-aa0c-42c7-a771-e8dd20e0520c.jpg?1604192922</code> | <code>https://cards.scryfall.io/normal/front/4/6/46f810c2-310e-42f5-ab1f-d56396cf5124.jpg?1599769231</code> | <code>0.25</code> |
• —Loss: <code>CosineSimilarityLoss</code> with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss"
  }

Training Hyperparameters

Non-Default Hyperparameters

• —eval_strategy: steps
• —per_device_train_batch_size: 64
• —per_device_eval_batch_size: 64
• —learning_rate: 2e-05
• —num_train_epochs: 1
• —warmup_ratio: 0.1
• —log_level_replica: passive
• —log_on_each_node: False
• —logging_nan_inf_filter: False
• —push_to_hub: True
• —resume_from_checkpoint: ./models/gte-micro-v4-mtg/
• —hub_model_id: philipp-zettl/gte-micro-v4-mtg
• —hub_always_push: True
• —batch_sampler: no_duplicates

All Hyperparameters

<details><summary>Click to expand</summary>

• —overwrite_output_dir: False
• —do_predict: False
• —eval_strategy: steps
• —prediction_loss_only: True
• —per_device_train_batch_size: 64
• —per_device_eval_batch_size: 64
• —per_gpu_train_batch_size: None
• —per_gpu_eval_batch_size: None
• —gradient_accumulation_steps: 1
• —eval_accumulation_steps: None
• —torch_empty_cache_steps: None
• —learning_rate: 2e-05
• —weight_decay: 0.0
• —adam_beta1: 0.9
• —adam_beta2: 0.999
• —adam_epsilon: 1e-08
• —max_grad_norm: 1.0
• —num_train_epochs: 1
• —max_steps: -1
• —lr_scheduler_type: linear
• —lr_scheduler_kwargs: {}
• —warmup_ratio: 0.1
• —warmup_steps: 0
• —log_level: passive
• —log_level_replica: passive
• —log_on_each_node: False
• —logging_nan_inf_filter: False
• —save_safetensors: True
• —save_on_each_node: False
• —save_only_model: False
• —restore_callback_states_from_checkpoint: False
• —no_cuda: False
• —use_cpu: False
• —use_mps_device: False
• —seed: 42
• —data_seed: None
• —jit_mode_eval: False
• —use_ipex: False
• —bf16: False
• —fp16: False
• —fp16_opt_level: O1
• —half_precision_backend: auto
• —bf16_full_eval: False
• —fp16_full_eval: False
• —tf32: None
• —local_rank: 0
• —ddp_backend: None
• —tpu_num_cores: None
• —tpu_metrics_debug: False
• —debug: []
• —dataloader_drop_last: False
• —dataloader_num_workers: 0
• —dataloader_prefetch_factor: None
• —past_index: -1
• —disable_tqdm: False
• —remove_unused_columns: True
• —label_names: None
• —load_best_model_at_end: False
• —ignore_data_skip: False
• —fsdp: []
• —fsdp_min_num_params: 0
• —fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
• —tp_size: 0
• —fsdp_transformer_layer_cls_to_wrap: None
• —accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
• —deepspeed: None
• —label_smoothing_factor: 0.0
• —optim: adamw_torch
• —optim_args: None
• —adafactor: False
• —group_by_length: False
• —length_column_name: length
• —ddp_find_unused_parameters: None
• —ddp_bucket_cap_mb: None
• —ddp_broadcast_buffers: False
• —dataloader_pin_memory: True
• —dataloader_persistent_workers: False
• —skip_memory_metrics: True
• —use_legacy_prediction_loop: False
• —push_to_hub: True
• —resume_from_checkpoint: ./models/gte-micro-v4-mtg/
• —hub_model_id: philipp-zettl/gte-micro-v4-mtg
• —hub_strategy: every_save
• —hub_private_repo: None
• —hub_always_push: True
• —gradient_checkpointing: False
• —gradient_checkpointing_kwargs: None
• —include_inputs_for_metrics: False
• —include_for_metrics: []
• —eval_do_concat_batches: True
• —fp16_backend: auto
• —push_to_hub_model_id: None
• —push_to_hub_organization: None
• —mp_parameters:
• —auto_find_batch_size: False
• —full_determinism: False
• —torchdynamo: None
• —ray_scope: last
• —ddp_timeout: 1800
• —torch_compile: False
• —torch_compile_backend: None
• —torch_compile_mode: None
• —dispatch_batches: None
• —split_batches: None
• —include_tokens_per_second: False
• —include_num_input_tokens_seen: False
• —neftune_noise_alpha: None
• —optim_target_modules: None
• —batch_eval_metrics: False
• —eval_on_start: False
• —use_liger_kernel: False
• —eval_use_gather_object: False
• —average_tokens_across_devices: False
• —prompts: None
• —batch_sampler: no_duplicates
• —multi_dataset_batch_sampler: proportional

</details>

Training Logs

Framework Versions

• —Python: 3.10.14
• —Sentence Transformers: 4.0.2
• —Transformers: 4.50.3
• —PyTorch: 2.6.0+cu124
• —Accelerate: 1.6.0
• —Datasets: 3.5.0
• —Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->