Modelpublic

axondendriteplus/Legal-Embed-bge-base-en-v1.5

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

3likes1.2kdownloads

Model Card

Legal-Embed-bge-base-en-v1.5

This repository hosts a fine-tuned version of BAAI/bge-base-en-v1.5 optimized for legal document (text) retrieval and Retrieval-Augmented Generation (RAG) tasks.

Model Details

—Base model: BAAI/bge-base-en-v1.5
—Dataset: axondendriteplus/legal-rag-embedding-dataset
—Task: Dense embedding learning for legal Q&A retrieval
—Framework: SentenceTransformers + HuggingFace Trainer
—Loss: MatryoshkaLossFunction (multi-resolution contrastive)

Evaluation (NDCG@10)

Dimension	Baseline	Fine-tuned	Improvement (%)
768	0.6105	0.6412	5.03
512	0.6037	0.6379	5.67
256	0.5853	0.6268	7.08
128	0.5276	0.5652	7.13
64	0.4469	0.5187	16.07

Metrics include cosine accuracy, MRR, MAP and NDCG.

Training Configuration

—Epochs: 4
—Batch size: 32
—Learning rate: 2e-5
—Data: 1,456 train / 162 test samples
—Hardware: CUDA GPU with FlashAttention

Findings

—Maximum improvement: 16.07%
—Fine-tuned 64D vs Baseline 768D: -15.03%
—Fine-tuned 128D vs Baseline 768D: -7.41%
—Storage reduction with 128D: 6× smaller
—Storage reduction with 64D: 12× smaller

—Baseline best score: 0.6105
—Fine-tuned best score: 0.6412

Usage

python

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("axondendriteplus/Legal-Embed-bge-base-en-v1.5")
embeddings = model.encode(["your legal text"])

Credits

Fine-tuning guide: https://www.philschmid.de/fine-tune-embedding-model-for-rag