datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
twentynewsgroups-clustering
TwentyNewsgroupsClustering.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of the 20 Newsgroups dataset (subject only).
Task category
t2c
Domains
News, Written
Reference
https://scikit-learn.org/0.19/datasets/twenty_newsgroups.html
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["TwentyNewsgroupsClustering.v2"])
evaluator = mteb.MTEB(task)… See the full description on the dataset page: https://huggingface.co/datasets/mteb/twentynewsgroups-clustering.stackexchange-clustering
StackExchangeClustering.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of titles from 121 stackexchanges. Clustering of 25 sets, each with 10-50 classes, and each class with 100 - 1000 sentences.
Task category
t2c
Domains
Web, Written
Reference
https://arxiv.org/abs/2104.07081
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/stackexchange-clustering.arxiv-clustering-s2s
ArXivHierarchicalClusteringS2S
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of titles from arxiv. Clustering of 30 sets, either on the main or secondary category
Task category
t2c
Domains
Academic, Written
Reference
https://www.kaggle.com/Cornell-University/arxiv
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["ArXivHierarchicalClusteringS2S"])… See the full description on the dataset page: https://huggingface.co/datasets/mteb/arxiv-clustering-s2s.biorxiv-clustering-p2p
BiorxivClusteringP2P.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of titles+abstract from biorxiv across 26 categories.
Task category
t2c
Domains
Academic, Written
Reference
https://api.biorxiv.org/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["BiorxivClusteringP2P.v2"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)… See the full description on the dataset page: https://huggingface.co/datasets/mteb/biorxiv-clustering-p2p.medrxiv-clustering-s2s
MedrxivClusteringS2S.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of titles from medrxiv across 51 categories.
Task category
t2c
Domains
Academic, Medical, Written
Reference
https://api.medrxiv.org/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["MedrxivClusteringS2S.v2"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)… See the full description on the dataset page: https://huggingface.co/datasets/mteb/medrxiv-clustering-s2s.arxiv-clustering-p2p
ArXivHierarchicalClusteringP2P
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of titles+abstract from arxiv. Clustering of 30 sets, either on the main or secondary category
Task category
t2c
Domains
Academic, Written
Reference
https://www.kaggle.com/Cornell-University/arxiv
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/arxiv-clustering-p2p.medrxiv-clustering-p2p
MedrxivClusteringP2P.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of titles+abstract from medrxiv across 51 categories.
Task category
t2c
Domains
Academic, Medical, Written
Reference
https://api.medrxiv.org/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["MedrxivClusteringP2P.v2"])
evaluator = mteb.MTEB(task)
model =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/medrxiv-clustering-p2p.stackexchange-clustering-p2p
StackExchangeClusteringP2P.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of title+body from stackexchange. Clustering of 5 sets of 10k paragraphs and 5 sets of 5k paragraphs.
Task category
t2c
Domains
Web, Written
Reference
https://arxiv.org/abs/2104.07081
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["StackExchangeClusteringP2P.v2"])… See the full description on the dataset page: https://huggingface.co/datasets/mteb/stackexchange-clustering-p2p.reddit-clustering
RedditClustering.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of titles from 199 subreddits. Clustering of 25 sets, each with 10-50 classes, and each class with 100 - 1000 sentences.
Task category
t2c
Domains
Web, Social, Written
Reference
https://arxiv.org/abs/2104.07081
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["RedditClustering.v2"])… See the full description on the dataset page: https://huggingface.co/datasets/mteb/reddit-clustering.101_billion_arabic_words_dataset
101 Billion Arabic Words Dataset
Updates
Maintenance Status: Actively Maintained
Update Frequency: Weekly updates to refine data quality and expand coverage.
Upcoming Version
More Cleaned Version: A more cleaned version of the dataset is in processing, which includes the addition of a UUID column for better data traceability and management.
Dataset Details
The 101 Billion Arabic Words Dataset is curated by the Clusterlab team and consists of 101… See the full description on the dataset page: https://huggingface.co/datasets/ClusterlabAi/101_billion_arabic_words_dataset.biorxiv-clustering-s2s
BiorxivClusteringS2S.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of titles from biorxiv across 26 categories.
Task category
t2c
Domains
Academic, Written
Reference
https://api.biorxiv.org/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["BiorxivClusteringS2S.v2"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)… See the full description on the dataset page: https://huggingface.co/datasets/mteb/biorxiv-clustering-s2s.ClusterWise
Data Overview
ClusterWise is a comprehensive dataset (240GB) containing real-world operational data from the OLCF Summit HPC cluster, one of the world's most powerful supercomputers. The dataset provides a rich collection of system telemetry, including:
Job scheduler logs
GPU failure event logs
High-resolution temperature measurements from CPUs and GPUs
Detailed power consumption metrics among components
Physical layout information of the nodes
The complete dataset related… See the full description on the dataset page: https://huggingface.co/datasets/MachaParfait/ClusterWise.stackoverflow-clustering
StackoverflowPtClustering
Cluster native Brazilian-Portuguese technical question titles from the Portuguese Stack Overflow (pt.stackoverflow.com) into 10 technology tags (python, java, php, javascript, android, mysql, c#, html, css, c). Programming domain.
Part of MTEB-BR — the native Brazilian-Portuguese MTEB sub-benchmark. Task type: Clustering · Language: Brazilian Portuguese (mined from real-world sources) · Domains: Programming, Web, Written.
Dataset structure… See the full description on the dataset page: https://huggingface.co/datasets/MTEB-BR/stackoverflow-clustering.juristcu-clustering
JurisTCUClusteringP2P
Cluster Brazilian Federal Court of Accounts (TCU) jurisprudence excerpts into 10 legal areas (Pessoal, Licitação, Responsabilidade, Direito Processual, Contrato Administrativo, Convênio, Competência do TCU, Finanças Públicas, Gestão Administrativa, Desestatização). Documents are the EXCERTO field of the JurisTCU corpus, labelled by TCU's own AREA taxonomy.
Part of MTEB-BR — the native Brazilian-Portuguese MTEB sub-benchmark. Task type: Clustering ·… See the full description on the dataset page: https://huggingface.co/datasets/MTEB-BR/juristcu-clustering.RefWave-Cluster-Runsscielo-clustering
SciELOClusteringP2P
Cluster Brazilian Portuguese scientific abstracts from the SciELO Brazil open-access library into 8 broad research areas (Health Sciences, Social Sciences, Agricultural Sciences, Biological/Life Sciences, Humanities & Arts, Engineering & Technology, Physical Sciences & Chemistry, Mathematics & Computer Science). Areas are consolidated from the Web-of-Science subject categories of each article; only pure CC-BY-4.0 articles are included.
Part of MTEB-BR — the… See the full description on the dataset page: https://huggingface.co/datasets/MTEB-BR/scielo-clustering.BuiltBench-clustering-s2s
Data sources
Industry Foundation Classes (IFC) published by buildingSmart International: https://ifc43-docs.standards.buildingsmart.org/
Uniclass product tables published by NBS: https://www.thenbs.com/our-tools/uniclass
License
cc-by-nc-nd-4.0: https://creativecommons.org/licenses/by-nc-nd/4.0/deed.en
How to cite
Research paper on the dataset development and validations: https://arxiv.org/abs/2411.12056
@article{shahinmoghadam2024benchmarking… See the full description on the dataset page: https://huggingface.co/datasets/mehrzad-shahin/BuiltBench-clustering-s2s.BuiltBench-clustering-p2p
Data sources
Industry Foundation Classes (IFC) published by buildingSmart International: https://ifc43-docs.standards.buildingsmart.org/
Uniclass product tables published by NBS: https://www.thenbs.com/our-tools/uniclass
License
cc-by-nc-nd-4.0: https://creativecommons.org/licenses/by-nc-nd/4.0/deed.en
How to cite
Research paper on the dataset development and validations: https://arxiv.org/abs/2411.12056
@article{shahinmoghadam2024benchmarking… See the full description on the dataset page: https://huggingface.co/datasets/mehrzad-shahin/BuiltBench-clustering-p2p.mteb-human-reddit-clustering
Reddit Clustering subset
Gold labels from official test.
mteb-human-wikicities-clustering
WikiCities Clustering subset
Gold labels from official test.
mteb-human-arxiv-clustering
Arxiv Clustering subset
Gold labels from official test.
camara-proposicoes-clustering
CamaraProposicoesClustering
Cluster the summaries (ementas) of bills from the Brazilian Chamber of Deputies into legislative themes from the Chamber's official taxonomy (Economia, Educação, Saúde, Meio Ambiente, Direitos Humanos, Administração Pública, etc.). Native PT-BR legislative text; public-domain government open data.
Part of MTEB-BR — the native Brazilian-Portuguese MTEB sub-benchmark. Task type: Clustering · Language: Brazilian Portuguese (mined from real-world sources)… See the full description on the dataset page: https://huggingface.co/datasets/MTEB-BR/camara-proposicoes-clustering.cluster-rerunc4-clusters
Dataset Card for "c4-clusters"
More Information needed
libero-subtaskid-clustereddatacomp-small-with-embeddings-and-cluster-labels
Dataset Card for "datacomp-small-with-embeddings-and-cluster-labels"
More Information needed
motif-cluster-dbmulti_clusteringreddit-clustering-p2p
RedditClusteringP2P.v2
An MTEB dataset
Massive Text Embedding Benchmark
Clustering of title+posts from reddit. Clustering of 10 sets of 50k paragraphs and 40 sets of 10k paragraphs.
Task category
t2c
Domains
Web, Social, Written
Reference
https://arxiv.org/abs/2104.07081
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["RedditClusteringP2P.v2"])
evaluator =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/reddit-clustering-p2p.strawberry_picking_dataset_scara_clustered
Strawberry Picking Dataset — SCARA Clustered
This LeRobot v3.0 dataset contains human-teleoperated demonstrations of a
4-DoF SCARA robot performing clustered strawberry picking in a controlled
physical mock-up. It accompanies the work
Learning to Pick: A Visuomotor Policy for Clustered Strawberry Picking,
which studies imitation-learned visuomotor control for reaching and picking a
target strawberry amid leaves, stems, and neighboring fruit.
The scene uses artificial strawberry… See the full description on the dataset page: https://huggingface.co/datasets/zfff/strawberry_picking_dataset_scara_clustered.
