datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
twentynewsgroups-clustering
TwentyNewsgroupsClustering.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of the 20 Newsgroups dataset (subject only).
Task category
t2c
Domains
News, Written
Reference
https://scikit-learn.org/0.19/datasets/twenty_newsgroups.html
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["TwentyNewsgroupsClustering.v2"])
evaluator = mteb.MTEB(task)… See the full description on the dataset page: https://huggingface.co/datasets/mteb/twentynewsgroups-clustering.stackexchange-clustering
StackExchangeClustering.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of titles from 121 stackexchanges. Clustering of 25 sets, each with 10-50 classes, and each class with 100 - 1000 sentences.
Task category
t2c
Domains
Web, Written
Reference
https://arxiv.org/abs/2104.07081
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/stackexchange-clustering.arxiv-clustering-s2s
ArXivHierarchicalClusteringS2S
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of titles from arxiv. Clustering of 30 sets, either on the main or secondary category
Task category
t2c
Domains
Academic, Written
Reference
https://www.kaggle.com/Cornell-University/arxiv
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["ArXivHierarchicalClusteringS2S"])… See the full description on the dataset page: https://huggingface.co/datasets/mteb/arxiv-clustering-s2s.biorxiv-clustering-p2p
BiorxivClusteringP2P.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of titles+abstract from biorxiv across 26 categories.
Task category
t2c
Domains
Academic, Written
Reference
https://api.biorxiv.org/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["BiorxivClusteringP2P.v2"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)… See the full description on the dataset page: https://huggingface.co/datasets/mteb/biorxiv-clustering-p2p.medrxiv-clustering-s2s
MedrxivClusteringS2S.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of titles from medrxiv across 51 categories.
Task category
t2c
Domains
Academic, Medical, Written
Reference
https://api.medrxiv.org/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["MedrxivClusteringS2S.v2"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)… See the full description on the dataset page: https://huggingface.co/datasets/mteb/medrxiv-clustering-s2s.arxiv-clustering-p2p
ArXivHierarchicalClusteringP2P
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of titles+abstract from arxiv. Clustering of 30 sets, either on the main or secondary category
Task category
t2c
Domains
Academic, Written
Reference
https://www.kaggle.com/Cornell-University/arxiv
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/arxiv-clustering-p2p.medrxiv-clustering-p2p
MedrxivClusteringP2P.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of titles+abstract from medrxiv across 51 categories.
Task category
t2c
Domains
Academic, Medical, Written
Reference
https://api.medrxiv.org/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["MedrxivClusteringP2P.v2"])
evaluator = mteb.MTEB(task)
model =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/medrxiv-clustering-p2p.stackexchange-clustering-p2p
StackExchangeClusteringP2P.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of title+body from stackexchange. Clustering of 5 sets of 10k paragraphs and 5 sets of 5k paragraphs.
Task category
t2c
Domains
Web, Written
Reference
https://arxiv.org/abs/2104.07081
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["StackExchangeClusteringP2P.v2"])… See the full description on the dataset page: https://huggingface.co/datasets/mteb/stackexchange-clustering-p2p.reddit-clustering
RedditClustering.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of titles from 199 subreddits. Clustering of 25 sets, each with 10-50 classes, and each class with 100 - 1000 sentences.
Task category
t2c
Domains
Web, Social, Written
Reference
https://arxiv.org/abs/2104.07081
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["RedditClustering.v2"])… See the full description on the dataset page: https://huggingface.co/datasets/mteb/reddit-clustering.biorxiv-clustering-s2s
BiorxivClusteringS2S.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of titles from biorxiv across 26 categories.
Task category
t2c
Domains
Academic, Written
Reference
https://api.biorxiv.org/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["BiorxivClusteringS2S.v2"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)… See the full description on the dataset page: https://huggingface.co/datasets/mteb/biorxiv-clustering-s2s.stackoverflow-clustering
StackoverflowPtClustering
Cluster native Brazilian-Portuguese technical question titles from the Portuguese Stack Overflow (pt.stackoverflow.com) into 10 technology tags (python, java, php, javascript, android, mysql, c#, html, css, c). Programming domain.
Part of MTEB-BR — the native Brazilian-Portuguese MTEB sub-benchmark. Task type: Clustering · Language: Brazilian Portuguese (mined from real-world sources) · Domains: Programming, Web, Written.
Dataset structure… See the full description on the dataset page: https://huggingface.co/datasets/MTEB-BR/stackoverflow-clustering.juristcu-clustering
JurisTCUClusteringP2P
Cluster Brazilian Federal Court of Accounts (TCU) jurisprudence excerpts into 10 legal areas (Pessoal, Licitação, Responsabilidade, Direito Processual, Contrato Administrativo, Convênio, Competência do TCU, Finanças Públicas, Gestão Administrativa, Desestatização). Documents are the EXCERTO field of the JurisTCU corpus, labelled by TCU's own AREA taxonomy.
Part of MTEB-BR — the native Brazilian-Portuguese MTEB sub-benchmark. Task type: Clustering ·… See the full description on the dataset page: https://huggingface.co/datasets/MTEB-BR/juristcu-clustering.scielo-clustering
SciELOClusteringP2P
Cluster Brazilian Portuguese scientific abstracts from the SciELO Brazil open-access library into 8 broad research areas (Health Sciences, Social Sciences, Agricultural Sciences, Biological/Life Sciences, Humanities & Arts, Engineering & Technology, Physical Sciences & Chemistry, Mathematics & Computer Science). Areas are consolidated from the Web-of-Science subject categories of each article; only pure CC-BY-4.0 articles are included.
Part of MTEB-BR — the… See the full description on the dataset page: https://huggingface.co/datasets/MTEB-BR/scielo-clustering.BuiltBench-clustering-s2s
Data sources
Industry Foundation Classes (IFC) published by buildingSmart International: https://ifc43-docs.standards.buildingsmart.org/
Uniclass product tables published by NBS: https://www.thenbs.com/our-tools/uniclass
License
cc-by-nc-nd-4.0: https://creativecommons.org/licenses/by-nc-nd/4.0/deed.en
How to cite
Research paper on the dataset development and validations: https://arxiv.org/abs/2411.12056
@article{shahinmoghadam2024benchmarking… See the full description on the dataset page: https://huggingface.co/datasets/mehrzad-shahin/BuiltBench-clustering-s2s.BuiltBench-clustering-p2p
Data sources
Industry Foundation Classes (IFC) published by buildingSmart International: https://ifc43-docs.standards.buildingsmart.org/
Uniclass product tables published by NBS: https://www.thenbs.com/our-tools/uniclass
License
cc-by-nc-nd-4.0: https://creativecommons.org/licenses/by-nc-nd/4.0/deed.en
How to cite
Research paper on the dataset development and validations: https://arxiv.org/abs/2411.12056
@article{shahinmoghadam2024benchmarking… See the full description on the dataset page: https://huggingface.co/datasets/mehrzad-shahin/BuiltBench-clustering-p2p.mteb-human-reddit-clustering
Reddit Clustering subset
Gold labels from official test.
mteb-human-wikicities-clustering
WikiCities Clustering subset
Gold labels from official test.
mteb-human-arxiv-clustering
Arxiv Clustering subset
Gold labels from official test.
camara-proposicoes-clustering
CamaraProposicoesClustering
Cluster the summaries (ementas) of bills from the Brazilian Chamber of Deputies into legislative themes from the Chamber's official taxonomy (Economia, Educação, Saúde, Meio Ambiente, Direitos Humanos, Administração Pública, etc.). Native PT-BR legislative text; public-domain government open data.
Part of MTEB-BR — the native Brazilian-Portuguese MTEB sub-benchmark. Task type: Clustering · Language: Brazilian Portuguese (mined from real-world sources)… See the full description on the dataset page: https://huggingface.co/datasets/MTEB-BR/camara-proposicoes-clustering.multi_clusteringreddit-clustering-p2p
RedditClusteringP2P.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of title+posts from reddit. Clustering of 10 sets of 50k paragraphs and 40 sets of 10k paragraphs.
Task category
t2c
Domains
Web, Social, Written
Reference
https://arxiv.org/abs/2104.07081
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["RedditClusteringP2P.v2"])
evaluator =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/reddit-clustering-p2p.ecthr_clusteringmopb_clusteringweather-clustering-data
Weather & Climate Big Data Analytics — 100-City ERA5 Historical Dataset (2016–2025)
Dataset Summary
This dataset contains 365,300 daily weather observations across 100 geographically diverse global cities spanning 80 countries over a 10-year continuous timeframe (January 1, 2016 – December 31, 2025).
The raw data was ingested from the Open-Meteo Historical Weather API (ERA5 Reanalysis Model) across 1,305 validated work units without missing values, then processed… See the full description on the dataset page: https://huggingface.co/datasets/tharinduperera/weather-clustering-data.stackexchange-clustering-exploded-test-vntenkgnad-clustering-s2sThis dataset can be used as a benchmark for clustering word embeddings for German.
The datasets contains news article titles and is based on the dataset of the One Million Posts Corpus and 10kGNAD. It contains 10'267 unique samples, 10 splits with 1'436 to 9'962 samples and 9 unique classes. Splits are built similarly to MTEB's TwentyNewsgroupsClustering.
Have a look at German Text Embedding Clustering Benchmark (Github, Paper) for more infos, datasets and evaluation results.
If you use this… See the full description on the dataset page: https://huggingface.co/datasets/slvnwhrl/tenkgnad-clustering-s2s.georeview-clustering-p2ptenkgnad-clustering-p2pThis dataset can be used as a benchmark for clustering word embeddings for German.
The datasets contains news article titles and is based on the dataset of the One Million Posts Corpus and 10kGNAD. It contains 10'275 unique samples, 10 splits with 1'436 to 9'962 samples and 9 unique classes. Splits are built similarly to MTEB's TwentyNewsgroupsClustering.
Have a look at German Text Embedding Clustering Benchmark (Github, Paper) for more infos, datasets and evaluation results.
If you use this… See the full description on the dataset page: https://huggingface.co/datasets/slvnwhrl/tenkgnad-clustering-p2p.blurbs-clustering-p2pThis dataset can be used as a benchmark for clustering word embeddings for German.
The datasets contains book titles and is based on the dataset from the GermEval 2019 Shared Task on Hierarchical Classification of Blurbs. It contains 18'084 unqiue samples, 28 splits with 177 to 16'425 samples and 4 to 93 unique classes. Splits are built similarly to MTEB's ArxivClusteringP2P.
Have a look at German Text Embedding Clustering Benchmark (Github, Paper) for more infos, datasets and evaluation… See the full description on the dataset page: https://huggingface.co/datasets/slvnwhrl/blurbs-clustering-p2p.blurbs-clustering-s2sThis dataset can be used as a benchmark for clustering word embeddings for German.
The datasets contains book titles and is based on the dataset from the GermEval 2019 Shared Task on Hierarchical Classification of Blurbs. It contains 17'726 unqiue samples, 28 splits with 177 to 16'425 samples and 4 to 93 unique classes. Splits are built similarly to MTEB's ArxivClusteringS2S.
Have a look at German Text Embedding Clustering Benchmark (Github, Paper) for more infos, datasets and evaluation… See the full description on the dataset page: https://huggingface.co/datasets/slvnwhrl/blurbs-clustering-s2s.