datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
stsbenchmark-sts
STSBenchmark
An MTEB dataset
Massive Text Embedding Benchmark
Semantic Textual Similarity Benchmark (STSbenchmark) dataset.
Task category
t2t
Domains
Blog, News, Written
Reference
https://github.com/PhilipMay/stsb-multi-mt/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["STSBenchmark"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)… See the full description on the dataset page: https://huggingface.co/datasets/mteb/stsbenchmark-sts.stsb
Glue STS-B
This dataset is a port of the official sts-b dataset on the Hub.
This is not a classification task, so the label_text column is only included for consistency
Note that the sentence1 and sentence2 columns have been renamed to text1 and text2 respectively.
Also, the test split is not labeled; the label column values are always -1.
ru-stsbenchmark-stsgerman-STSbenchmark
German STS Benchmark
This data is orinally from https://github.com/t-systems-on-site-services-gmbh/german-STSbenchmark
The license information can be found under:
https://github.com/t-systems-on-site-services-gmbh/german-STSbenchmark/blob/master/LICENSE
stsbenchmark-sts_CS-MTEB
STSBenchmark CS-MTEB
An MTEB dataset
Massive Text Embedding Benchmark
Code-switching version of mteb/stsbenchmark-sts, with test set sentence pairs rewritten in Chinese-English, Japanese-English, German-English, Spanish-English, Korean-English, French-English, Italian-English, Portuguese-English, Dutch-English code-switching styles.
Dataset Structure
The dataset contains the following configurations:
From original dataset (unchanged):
default: Original train, validation… See the full description on the dataset page: https://huggingface.co/datasets/UTokyo-Yokoya-Lab/stsbenchmark-sts_CS-MTEB.stsb_tr
Dataset Description
Repository: https://github.com/verimsu/STSb-TR
Dataset Summary
STSb-TR dataset is the machine translated version of English STS benchmark dataset using Google Cloud Translation API.
Citation
@inproceedings{beken-fikri-etal-2021-semantic,
title = "Semantic Similarity Based Evaluation for Abstractive News Summarization",
author = "Beken Fikri, Figen and Oflazer, Kemal and Yanikoglu, Berrin",
booktitle = "Proceedings of the 1st… See the full description on the dataset page: https://huggingface.co/datasets/figenfikri/stsb_tr.STSBench
STSBench: A Spatio-temporal Scenario Benchmark for Multi-modal Large Language Models in Autonomous Driving
Abstract
We introduce STSBench, a scenario-based framework to benchmark the holistic understanding of vision-language models (VLMs) for autonomous driving. The framework automatically mines pre-defined traffic scenarios from any dataset using ground-truth annotations, provides an intuitive user interface for efficient human verification, and generates… See the full description on the dataset page: https://huggingface.co/datasets/ivc-lrp/STSBench.stsbenchmark-teststsb-skRetrieving the 50th example from the train set:
> print(dataset['train']['sentence1'][0][50])
Muž hrá na gitare.
> print(dataset['train']['sentence2'][0][50])
Chlapec hrá na gitare.
> print(dataset['train']['similarity_score'][0][50])
3.200000047683716
For score explanation see stsb_multi_mt.
kyrgyz-stsb
Kyrgyz STS-B
Task
Semantic Textual Similarity (Regression)
Description
STS Benchmark dataset translated from English to Kyrgyz. Each entry contains two sentences with Kyrgyz translations and a continuous similarity score (0.0 - 5.0).
Score range: 0.0 (unrelated) to 5.0 (equivalent meaning)
Format: JSONL with fields: sentence1, sentence2, similarity_score, sentence1_ky, sentence2_ky
Dataset Size
Split
Entries
Train
5,749
Validation… See the full description on the dataset page: https://huggingface.co/datasets/metinovadilet/kyrgyz-stsb.Turkish-STSBenchmark
Turkish STS Benchmark
This data is orinally from https://github.com/verimsu/STSb-TR/tree/main
BibTeX Citation
If you use this dataset, please cite following paper:
@inproceedings{beken-fikri-etal-2021-semantic,
title = "Semantic Similarity Based Evaluation for Abstractive News Summarization",
author = "Beken Fikri, Figen and Oflazer, Kemal and Yanikoglu, Berrin",
booktitle = "Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and… See the full description on the dataset page: https://huggingface.co/datasets/asparius/Turkish-STSBenchmark.stsbenchmark-sts
STSBenchmark
An MTEB dataset
Massive Text Embedding Benchmark
Semantic Textual Similarity Benchmark (STSbenchmark) dataset.
Task category
t2t
Domains
Blog, News, Written
Reference
https://github.com/PhilipMay/stsb-multi-mt/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["STSBenchmark"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)… See the full description on the dataset page: https://huggingface.co/datasets/keerat879/stsbenchmark-sts.stsbenchmark-sts-vietnamese
High-quality translation of STS Benchmark Dataset into Vietnamese with synonym
test_stsb_stsbenchmark_vi