Modelpublic

yjoonjang/all-MiniLM-L6-v2-multi-qa-ties-merge

sourceHugging Faceupdated 1mo agoView on Hugging Face

0likes22downloads

Model Card

all-MiniLM-L6-v2-multi-qa-ties-merge

A demonstration model produced by the native model-merging feature (SentenceTransformer.merge) added to Sentence Transformers. It merges the top-70% task deltas with sign election (ties), relative to sentence-transformers/all-MiniLM-L6-v2 as the base.

Merged checkpoints:

—`sentence-transformers/all-MiniLM-L6-v2`
—`sentence-transformers/multi-qa-MiniLM-L6-cos-v1`
—Base model (delta reference): `sentence-transformers/all-MiniLM-L6-v2`

How it was created

python

from sentence_transformers import SentenceTransformer

merged = SentenceTransformer.merge(
    models=["sentence-transformers/all-MiniLM-L6-v2",
            "sentence-transformers/multi-qa-MiniLM-L6-cos-v1"],
    weights=[0.5, 0.5],
    densities=[0.7, 0.7],
    method="ties",
    base_model="sentence-transformers/all-MiniLM-L6-v2",
    output_path="all-MiniLM-L6-v2-multi-qa-ties-merge",
    dtype="float32",
)

Usage

python

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("yjoonjang/all-MiniLM-L6-v2-multi-qa-ties-merge")
emb = model.encode(["Model merging combines fine-tuned checkpoints.",
                    "It often beats each individual model."])
print(model.similarity(emb, emb))

License

Derivative of the two base models above; their licenses apply. See each base model's card.