InstalilyAI/synonym-transformer-LoRA

Automotive Parts Synonym Model

A fine-tuned SentenceTransformer model specialized for finding synonyms and related terms in automotive parts and service descriptions.

Model Description

This model is fine-tuned from sentence-transformers/all-MiniLM-L6-v2 specifically for automotive parts synonym detection. It can identify when different part names refer to the same or similar components.

Base Model: sentence-transformers/all-MiniLM-L6-v2

Training Details

Training Strategy: 3-phase approach

1. 1.Contextual Training (30 epochs): Full phrases with synonyms/antonyms
2. 2.Foundation Training (15 epochs): Word-to-word synonyms/antonyms
3. 3.Real-world Fine-tuning (4 epochs): Search phrases and repair descriptions

Loss Function: OnlineContrastiveLoss with varying margins (0.6 → 0.4 → 0.4) Training Data: Automotive parts synonym/antonym pairs with contextual repair descriptions LoRA Integration: To significantly improve general semantic performance, LoRA adapters were attached to the attention and feed-forward layers of the MiniLM encoder.

• —Config: r=64, alpha=256, dropout=0.05
• —Only adapter parameters were trained, while the majority of the base model remained frozen.
• —Adapters were enabled during training phases, disabled for baseline evaluations, and saved separately at checkpoints.

Performance

Evaluated on 15-state synonyms phrases, deepseek general queries, STS-B and MTB datasets:

• —Top-200 Synonym Recall 15192/15330 (99.1%)
• —MRR@200 0.6183
• —Top-200 Deepseek Recall 140844/142325 (99.0%)
• —MRR@200 0.4436
• —STS-B Spearman: 0.867
• —MTB Spearman: 0.724

Limitations

• —Optimized specifically for automotive parts and repair terminology
• —May not perform well on general-domain text