CoolFace
Modelpublic

BM-K/KoSimCSE-roberta-multitask

sourceHugging Faceupdated 4y agoView on Hugging Face
30likes25kdownloads
Model Card

https://github.com/BM-K/Sentence-Embedding-is-all-you-need

Korean-Sentence-Embedding

๐Ÿญ Korean sentence embedding repository. You can download the pre-trained models and inference right away, also it provides environments where individuals can train models.

Quick tour

python
import torch
from transformers import AutoModel, AutoTokenizer

def cal_score(a, b):
    if len(a.shape) == 1: a = a.unsqueeze(0)
    if len(b.shape) == 1: b = b.unsqueeze(0)

    a_norm = a / a.norm(dim=1)[:, None]
    b_norm = b / b.norm(dim=1)[:, None]
    return torch.mm(a_norm, b_norm.transpose(0, 1)) * 100

model = AutoModel.from_pretrained('BM-K/KoSimCSE-roberta-multitask') 
AutoTokenizer.from_pretrained('BM-K/KoSimCSE-roberta-multitask')

sentences = ['์น˜ํƒ€๊ฐ€ ๋“คํŒ์„ ๊ฐ€๋กœ ์งˆ๋Ÿฌ ๋จน์ด๋ฅผ ์ซ“๋Š”๋‹ค.',
             '์น˜ํƒ€ ํ•œ ๋งˆ๋ฆฌ๊ฐ€ ๋จน์ด ๋’ค์—์„œ ๋‹ฌ๋ฆฌ๊ณ  ์žˆ๋‹ค.',
             '์›์ˆญ์ด ํ•œ ๋งˆ๋ฆฌ๊ฐ€ ๋“œ๋Ÿผ์„ ์—ฐ์ฃผํ•œ๋‹ค.']

inputs = tokenizer(sentences, padding=True, truncation=True, return_tensors="pt")
embeddings, _ = model(**inputs, return_dict=False)

score01 = cal_score(embeddings[0][0], embeddings[1][0])
score02 = cal_score(embeddings[0][0], embeddings[2][0])

Performance

  • โ€”Semantic Textual Similarity test set results <br>
ModelAVGCosine PearsonCosine SpearmanEuclidean PearsonEuclidean SpearmanManhattan PearsonManhattan SpearmanDot PearsonDot Spearman
KoSBERT<sup>โ€ </sup><sub>SKT</sub>77.4078.8178.4777.6877.7877.7177.8375.7575.22
KoSBERT80.3982.1382.2580.6780.7580.6980.7877.9677.90
KoSRoBERTa81.6481.2082.2081.7982.3481.5982.2080.6281.25
KoSentenceBART77.1479.7178.7478.4278.0278.4078.0074.2472.15
KoSentenceT577.8380.8779.7480.2479.3680.1979.2772.8170.17
KoSimCSE-BERT<sup>โ€ </sup><sub>SKT</sub>81.3282.1282.5681.8481.6381.9981.7479.5579.19
KoSimCSE-BERT83.3783.2283.5883.2483.6083.1583.5483.1383.49
KoSimCSE-RoBERTa83.6583.6083.7783.5483.7683.5583.7783.5583.64
KoSimCSE-BERT-multitask85.7185.2986.0285.6386.0185.5785.9785.2685.93
KoSimCSE-RoBERTa-multitask85.7785.0886.1285.8486.1285.8386.1285.0385.99