sts
Datasets
All datasets matching “sts”sts12-sts
STS12
An MTEB dataset
Massive Text Embedding Benchmark
SemEval-2012 Task 6.
Task category
t2t
Domains
Encyclopaedic, News, Written
Reference
https://www.aclweb.org/anthology/S12-1051.pdf
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["STS12"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)
evaluator.run(model)
To learn more about how to… See the full description on the dataset page: https://huggingface.co/datasets/mteb/sts12-sts.sts22-crosslingual-sts
STS22.v2
An MTEB dataset
Massive Text Embedding Benchmark
SemEval 2022 Task 8: Multilingual News Article Similarity. Version 2 filters updated on STS22 by removing pairs where one of entries contain empty sentences.
Task category
t2t
Domains
News, Written
Reference
https://competitions.codalab.org/competitions/33835
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/sts22-crosslingual-sts.sickr-sts
SICK-R
An MTEB dataset
Massive Text Embedding Benchmark
Semantic Textual Similarity SICK-R dataset
Task category
t2t
Domains
Web, Written
Reference
https://aclanthology.org/L14-1314/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["SICK-R"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)
evaluator.run(model)
To learn more about how to run models… See the full description on the dataset page: https://huggingface.co/datasets/mteb/sickr-sts.sts13-sts
STS13
An MTEB dataset
Massive Text Embedding Benchmark
SemEval STS 2013 dataset.
Task category
t2t
Domains
Web, News, Non-fiction, Written
Reference
https://www.aclweb.org/anthology/S13-1004/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["STS13"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)
evaluator.run(model)
To learn more about how… See the full description on the dataset page: https://huggingface.co/datasets/mteb/sts13-sts.sts14-sts
STS14
An MTEB dataset
Massive Text Embedding Benchmark
SemEval STS 2014 dataset. Currently only the English dataset
Task category
t2t
Domains
Blog, Web, Spoken
Reference
https://www.aclweb.org/anthology/S14-1002
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["STS14"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)
evaluator.run(model)
To… See the full description on the dataset page: https://huggingface.co/datasets/mteb/sts14-sts.biosses-sts
BIOSSES
An MTEB dataset
Massive Text Embedding Benchmark
Biomedical Semantic Similarity Estimation.
Task category
t2t
Domains
Medical
Reference
https://tabilab.cmpe.boun.edu.tr/BIOSSES/DataSet.html
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["BIOSSES"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)
evaluator.run(model)
To learn more… See the full description on the dataset page: https://huggingface.co/datasets/mteb/biosses-sts.



