datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Athar-EmbeddingsFinQAFinQA is a large-scale dataset with 2.8k financial reports for 8k Q&A pairs to study numerical reasoning with structured and unstructured evidence.
Usage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/FinQA", "queries")
documents = datasets.load_dataset("embedding-benchmark/FinQA", "corpus")
pair_labels = datasets.load_dataset("embedding-benchmark/FinQA", "default")
FinanceBenchThe FinanceBench dataset is derived from the PatronusAI/financebench-test dataset, containing only the PASS examples processed into a clean format for question-answering tasks in the financial domain. FinanceBench-rtl has been repurposed for retrieval.
Usage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/FinanceBench", "queries")
documents = datasets.load_dataset("embedding-benchmark/FinanceBench", "corpus")
pair_labels =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/FinanceBench.LMEB
🚀 LMEB is now officially supported by MTEB.You can evaluate embedding models on LMEB through the standard MTEB evaluation framework. After running the benchmark, results can be submitted to the MTEB results repository. Once merged, the LMEB leaderboard will be updated on the MTEB Leaderboard.
🔥 Why LMEB?
LMEB fills a crucial gap in current embedding benchmarks, offering a standardized and reproducible… See the full description on the dataset page: https://huggingface.co/datasets/KaLM-Embedding/LMEB.HC3FinanceThe HC3 dataset comprises tens of thousands of comparison responses from both human experts and ChatGPT across various domains, including open-domain, financial, medical, legal, and psychological areas. The data collection process involved sourcing publicly available question-answering datasets and wiki texts, ensuring that the human answers were either expert-provided or high-quality user responses, thereby minimizing mislabeling and enhancing the dataset's reliability.
Usage
import datasets… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/HC3Finance.asr-ser-quechua-collao-embeddings
ASR-SER embeddings for Quechua Collao
This repository contains embeddings only. It does not contain raw audio.
These embeddings were extracted for ASR-to-SER transfer experiments on the Quechua Collao emotional speech corpus. The release is intended for reproducible feature-extraction experiments and downstream analysis.
Dataset contents
One PyTorch tensor per utterance stored as an embedding file under embeddings/
A sanitized metadata table describing utterance… See the full description on the dataset page: https://huggingface.co/datasets/QuechuaBase/asr-ser-quechua-collao-embeddings.ChatDoctor_HealthCareMagicThe ChatDoctor-HealthCareMagic-100k dataset comprises 112,000 real-world medical question-and-answer pairs, providing a substantial and diverse collection of authentic medical dialogues. There is a slight risk to this dataset since there are grammatical inconsistencies in many of the questions and answers, but this can potentially help separate strong healthcare retrieval models from weak ones.
Usage
import datasets
# Download the dataset
queries =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/ChatDoctor_HealthCareMagic.embedding_datastackexchange_title_body_jsonljsonl.gz format from https://huggingface.co/datasets/flax-sentence-embeddings/stackexchange_xml
Each line contains a dict in the format: {"text": ["title", "body"], "tags": ["tag1", "tag2"]}
The following parameters have been used for filtering: min_title_len = 20 min_body_len = 20 max_body_len = 4096 min_score = 0
If a stackexchange contained less than 10k questions (after filtering), it is written to the small_stackexchanges.jsonl.gz file.
This is a dump of the files from… See the full description on the dataset page: https://huggingface.co/datasets/flax-sentence-embeddings/stackexchange_title_body_jsonl.FreshStack_mtebUsage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/MBPP", "queries")
documents = datasets.load_dataset("embedding-benchmark/MBPP", "corpus")
pair_labels = datasets.load_dataset("embedding-benchmark/MBPP", "default")
kilt-qwen-embeddingThe corpus refers to https://huggingface.co/datasets/corag/kilt-corpus
MBPPThe MBPP dataset consists of around 1,000 crowd-sourced Python programming problems, designed to be solvable by entry level programmers, covering programming fundamentals, standard library functionality, and so on. Each problem consists of a task description, code solution and 3 automated test cases. As described in the paper, a subset of the data has been hand-verified by the dataset authors to ensure quality.
Usage
import datasets
# Download the dataset
queries =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/MBPP.HumanEvalThe HumanEval dataset released by OpenAI includes 164 programming problems with a handwritten function signature, docstring, body, and several unit tests for each problem. The dataset was handcrafted by engineers and researchers at OpenAI.
Usage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/HumanEval", "queries")
documents = datasets.load_dataset("embedding-benchmark/HumanEval", "corpus")
pair_labels =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/HumanEval.CPRet-Embeddings
CPRet-Embeddings
This repository provides the problem descriptions and their corresponding precomputed embeddings used by the CPRet retrieval server.
You can explore the retrieval server via the online demo at https://cpret.online/.
📦 Files
probs_2609.jsonlA JSONL file containing natural language descriptions of competitive programming problems.Each line is a JSON object with metadata such as problem title, platform/source OJ, URL, and full description.… See the full description on the dataset page: https://huggingface.co/datasets/coldchair16/CPRet-Embeddings.WikiSQL_mtebWikiSQL is a dataset comprising 80,654 hand-annotated examples of natural language questions and corresponding SQL queries across 24,241 tables from Wikipedia.
Usage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/MBPP", "queries")
documents = datasets.load_dataset("embedding-benchmark/MBPP", "corpus")
pair_labels = datasets.load_dataset("embedding-benchmark/MBPP", "default")
DS1000DS-1000 is a code generation benchmark with a thousand data science problems spanning seven Python libraries, such as NumPy and Pandas. It employs multi-criteria evaluation metrics, including functional correctness and surface-form constraints, resulting in a high-quality dataset with only 1.8% incorrect solutions among accepted Codex-002 predictions.
Usage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/DS1000", "queries")
documents =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/DS1000.airbnb_embeddings
Overview
This dataset consists of AirBnB listings with property descriptions, reviews, and other metadata.
It also contains text embeddings of the property descriptions as well as image embeddings of the listing image. The text embeddings were created using OpenAI's text-embedding-3-small model and the image embeddings using OpenAI's clip-vit-base-patch32 model available on Hugging Face.
The text embeddings have 1536 dimensions, while the image embeddings have 512 dimensions.… See the full description on the dataset page: https://huggingface.co/datasets/MongoDB/airbnb_embeddings.klein-embedding-dataCPRet-Embeddings
CPRet-Embeddings
This repository provides the problem descriptions and their corresponding precomputed embeddings used by the CPRet retrieval server.
You can explore the retrieval server via the online demo at https://cpret.online/.
📦 Files
probs_2606.jsonlA JSONL file containing natural language descriptions of competitive programming problems.Each line is a JSON object with metadata such as problem title, platform/source OJ, URL, and full description.… See the full description on the dataset page: https://huggingface.co/datasets/upctanker/CPRet-Embeddings.Embedding-model-fine-tuning-datasetETP-Eval26-embeddings
ETP-Eval26 Embeddings
Vorberechnete Vektoren zum Datensatz
ETP-Eval26.
Inhalt
Eine .npz-Datei je Modell, dazu meta.json mit den Zeilenbeschriftungen.
Datei
Modell
Dimension
bge-m3.npz
BAAI/bge-m3
1024
mE5-large.npz
intfloat/multilingual-e5-large
1024
labse.npz
sentence-transformers/LaBSE
768
sphilberta.npz
bowphs/SPhilBerta
768
qwen3-emb-0.6b.npz
Qwen/Qwen3-Embedding-0.6B
1024
qwen3-emb-4b.npz
Qwen/Qwen3-Embedding-4B
2560
xlmr.npz… See the full description on the dataset page: https://huggingface.co/datasets/mschonhardt/ETP-Eval26-embeddings.gemma-2b-dictionary-embeddings-all-layers
Gemma-2B Dictionary Embeddings - All Layers
This dataset contains pre-computed embeddings for 77,477 English words from WordNet using the Gemma-2B model across all 27 layers.
Dataset Structure
metadata.json: Contains dataset metadata (model info, dimensions, word count)
embeddings_layer_X.pkl: Pickle files containing embeddings for layer X (0-26)
Usage
import pickle
from huggingface_hub import hf_hub_download
# Download a specific layer
layer_0_path =… See the full description on the dataset page: https://huggingface.co/datasets/LeeHarrold/gemma-2b-dictionary-embeddings-all-layers.canva-visual-search-embeddings
Visual Search Embedding Benchmark: Extending Canva's DINOv2 Evaluation
Executive Summary
This benchmark extends Canva's January 2025 engineering evaluation
which chose DINOv2 for production image replacement. We test three newer models released since then
against DINOv2 on 500 design-domain images (advertising posters from the CGL-Dataset).
Key Findings
Metric
Winner
Score
vs DINOv2
Recall@1
facebook/dinov2-base
1.0000
—
Recall@5… See the full description on the dataset page: https://huggingface.co/datasets/prometheus04/canva-visual-search-embeddings.klein-embedding-v2coco_captions_quintets
Dataset Card for "coco_captions"
Dataset Summary
COCO is a large-scale object detection, segmentation, and captioning dataset. This repo contains five captions per image; useful for sentence similarity tasks.
Disclaimer: The team releasing COCO did not upload the dataset to the Hub and did not write a dataset card.
These steps were done by the Hugging Face team.
Supported Tasks
Sentence Transformers training; useful for semantic search and sentence… See the full description on the dataset page: https://huggingface.co/datasets/embedding-data/coco_captions_quintets.APPSAPPS is a benchmark for code generation with 10000 problems. It can be used to evaluate the ability of language models to generate code from natural language specifications. To create the APPS dataset, the authors manually curated problems from open-access sites where programmers share problems with each other, including Codewars, AtCoder, Kattis, and Codeforces.
Usage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/APPS", "queries")
documents =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/APPS.WikiAnswers
Dataset Card for "WikiAnswers"
Dataset Summary
The WikiAnswers corpus contains clusters of questions tagged by WikiAnswers users as paraphrases.
Each cluster optionally contains an answer provided by WikiAnswers users. There are 30,370,994 clusters containing an average of 25 questions per cluster. 3,386,256 (11%) of the clusters have an answer.
Supported Tasks
Sentence Transformers training; useful for semantic search and sentence similarity.… See the full description on the dataset page: https://huggingface.co/datasets/embedding-data/WikiAnswers.QQP_triplets
Dataset Card for "QQP_triplets"
Dataset Summary
This dataset will give anyone the opportunity to train and test models of semantic equivalence, based on actual Quora data. The data is organized as triplets (anchor, positive, negative).
Disclaimer: The team releasing Quora data did not upload the dataset to the Hub and did not write a dataset card.
These steps were done by the Hugging Face team.
Supported Tasks
Sentence Transformers training; useful for… See the full description on the dataset page: https://huggingface.co/datasets/embedding-data/QQP_triplets.primevul-codebert-embeddings
PrimeVul Embeddings for PU Learning
Pre-extracted [CLS] token embeddings from two code models for all functions in the PrimeVul v0.1 vulnerability detection dataset, plus the raw PrimeVul v0.1 JSONL source files.
CodeBERT Embeddings (root .npz files)
Each .npz file contains frozen CodeBERT embeddings (768-dimensional vectors) for C/C++ functions, along with their labels and CWE type annotations. These were extracted once using a frozen CodeBERT model and are used for… See the full description on the dataset page: https://huggingface.co/datasets/db-d2/primevul-codebert-embeddings.PAQ_pairs
Dataset Card for "PAQ_pairs"
Dataset Summary
Pairs questions and answers obtained from Wikipedia.
Disclaimer: The team releasing PAQ QA pairs did not upload the dataset to the Hub and did not write a dataset card.
These steps were done by the Hugging Face team.
Supported Tasks
Sentence Transformers training; useful for semantic search and sentence similarity.
Languages
English.
Dataset Structure
Each example in the dataset contains… See the full description on the dataset page: https://huggingface.co/datasets/embedding-data/PAQ_pairs.