datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
farsick-sts
Dataset Summary
FarSick STS is a Persian (Farsi) dataset designed for the Semantic Textual Similarity (STS) task. It is a part of the FaMTEB (Farsi Massive Text Embedding Benchmark). The dataset was developed by translating and adapting the English SICK (Sentences Involving Compositional Knowledge) dataset, and it features Persian sentence pairs annotated for their degree of semantic relatedness.
Language(s): Persian (Farsi)
Task(s): Semantic Textual Similarity (STS)
Source:… See the full description on the dataset page: https://huggingface.co/datasets/MCINext/farsick-sts.stssts 2012-2016 datasets
mteb_ASSIN2_STSIRIS_sts
Work developed as part of Project IRIS.
Thesis: A Semantic Search System for Supremo Tribunal de Justiça
Portuguese Legal Sentences
Collection of Legal Sentences pairs from the Portuguese Supreme Court of Justice
The goal of this dataset was to be used for Semantic Textual Similarity
Values from 0-1: random sentences across documents
Values from 2-4: sentences from the same summary (implying some level of entailment)
Values from 4-5: sentences pairs generated through OpenAi'… See the full description on the dataset page: https://huggingface.co/datasets/stjiris/IRIS_sts.agentlans__multilingual-sentences__paired_10_stsSentences from agentlans/multilingual-sentences in Spanish, and processed with Sentence Similarity Cosine Scores with model hiiamsid/sentence_similarity_spanish_es
Each sentence in original dataset was randomly assigned 10 rows (sentences) within a batch of 1000, calculate the sentence similarity, and then deleted duplicate pairs
The code for processing can be found here
Useful for data distillation, training or benchmarking.
Its recommended resampling the dataset to undersample to get a… See the full description on the dataset page: https://huggingface.co/datasets/erickfmm/agentlans__multilingual-sentences__paired_10_sts.sts-companionhttps://ixa2.si.ehu.eus/stswiki/index.php/STSbenchmark
The companion datasets to the STS Benchmark comprise the rest of the English datasets used in the STS tasks organized by us in the context of SemEval between 2012 and 2017.
Authors collated two datasets, one with pairs of sentences related to machine translation evaluation. Another one with the rest of datasets, which can be used for domain adaptation studies.
@inproceedings{cer-etal-2017-semeval,
title = "{S}em{E}val-2017 Task 1:… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/sts-companion.Eurostat_Tourism_STS_Dataset_Turnover_in_Services
Eurostat Tourism STS Dataset – Turnover in Services (Monthly)
This repository contains data extracted from the Eurostat Short-Term Statistics (STS) domain, with a focus on:
tour_sts – Tourism industries short-term indicators
sts_setu_m – Turnover in services (monthly data)
These datasets measure monthly turnover and sales volume indices across tourism-related industries following the NACE Rev.2 classification.
Source: https://ec.europa.eu/eurostat/web/tourism/databaseLicense: CC… See the full description on the dataset page: https://huggingface.co/datasets/VillanovaAI/Eurostat_Tourism_STS_Dataset_Turnover_in_Services.stsbenchmark-sts-vi
STS Benchmark Dataset for Vietnamese
sts-arabic-translated-modifiedsts_mteb_fsstsb-binary-paraphrase-labelled
Paraphrase Detection Dataset (Derived from SetFit/stsb)
Description:
This dataset originates from the SetFit/stsb dataset, which was initially created for semantic textual similarity (STS) tasks with a label range of 0 to 5. It has been adapted for binary paraphrase detection by leveraging the high-accuracy paraphrase classification model viswadarshan06/pd-robert.
Each sentence pair in the original dataset has been re-labeled according to the following binary scheme:
1 →… See the full description on the dataset page: https://huggingface.co/datasets/viswadarshan06/stsb-binary-paraphrase-labelled.STSThis is a synthetic Faroese Semantic Textual Similarity dataset. Labels range from 0 (no similarity) to 5 (the two sentences are completely equivalent).
The dataset was generated by:
Translating sentences from the Basic Faroese Language Resource Kit (BLARK) corpus to English by leveraging a Nordic LLM, GPT-Sw3.
Sentences were compared to each other in terms of semantic similarity by Sentence BERT (SBERT, )
Pairs of sentences were then sampled uniformly in terms of similarity score, to compile… See the full description on the dataset page: https://huggingface.co/datasets/barbaroo/STS.sts-original-2sts_task! pip install python-Levenshtein
! pip install fuzzywuzzy
import pandas as pd
from datasets import load_dataset
from fuzzywuzzy import fuzz
dataset_name_list = [
"mteb/sts12-sts",
"mteb/sts13-sts",
"mteb/sts14-sts",
"mteb/sts15-sts",
"mteb/sts16-sts",
"mteb/stsbenchmark-sts",
"mteb/sickr-sts",
]
dataset_dict = { _[5:-4]:load_dataset(_) for _ in dataset_name_list}
df_list = []
for dataset_name, datasetDict in dataset_dict.items():
for split_name, dataset in… See the full description on the dataset page: https://huggingface.co/datasets/bcai001/sts_task.IRIS_sts
Work developed as part of Project IRIS.
Thesis: A Semantic Search System for Supremo Tribunal de Justiça
Portuguese Legal Sentences
Collection of Legal Sentences pairs from the Portuguese Supreme Court of Justice
The goal of this dataset was to be used for Semantic Textual Similarity
Values from 0-1: random sentences across documents
Values from 2-4: sentences from the same summary (implying some level of entailment)
Values from 4-5: sentences pairs generated through OpenAi'… See the full description on the dataset page: https://huggingface.co/datasets/Aronwinner/IRIS_sts.sts_mteb_ours