datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
mmlu-prox-eval-predictions
MMLU-ProX Multilingual Model Predictions
Raw per-sample model predictions on MMLU-ProX
across 29 languages and 25 open-weight LLMs, produced with
lm-evaluation-harness.
This dataset releases the full prediction logs (not just aggregate scores) so that
item-level responses can be re-analysed — e.g. for Item Response Theory (IRT) modelling
of multilingual benchmarks, error analysis, or per-item difficulty estimation.
Repository structure
mmlu_prox_<lang>/
└──… See the full description on the dataset page: https://huggingface.co/datasets/gililior/mmlu-prox-eval-predictions.fineweb-edu
FineWeb-Edu (Lance Format)
A Lance-formatted version of FineWeb-Edu — over 1.5 billion educational web passages with cleaned text, source metadata, language detection signals, and 384-dim text embeddings — available directly from the Hub at hf://datasets/lance-format/fineweb-edu/data/train.lance.
Key features
Cleaned passage text in the text column with the source url and title carried alongside.
Language detection signals (language, language_probability) for filtered… See the full description on the dataset page: https://huggingface.co/datasets/lance-format/fineweb-edu.docmath-eval-failures-200
DocMath-Eval Failures 200: Agent Benchmark & Leaderboard
A curated benchmark of 200 challenging financial math questions that leading AI models
failed to answer correctly, with comprehensive evaluation results from multiple AI agents.
Leaderboard
Evaluated on 2026-02-21 using LLM-as-Judge (Qwen QwQ-32B) for soft scoring.
Rank
Agent
Model
Exact Match
Judge: Exact
Judge: Approx
Judge: Total
Wrong
Avg Duration
Avg Tool Calls
1
TRAE Agent
Opus 4.5
98/200 (49.0%)
96… See the full description on the dataset page: https://huggingface.co/datasets/Ayushnangia/docmath-eval-failures-200.agent-llm-traces-v2
Exgentic Agent LLM Traces v2 — Agent Chat Only
OpenTelemetry-shaped execution traces for 10,057 agent runs across 6 benchmarks (AppWorld, SWE-bench, BrowseCompPlus, τ²-bench Airline/Retail/Telecom), filtered to the agent under test's chat-only LLM calls. This is the dataset for replay testing, behavioral analysis, or any task where you care about what the benchmarked model actually did — not the eval scaffolding around it.
This v2 release expands upon Exgentic/agent-llm-traces… See the full description on the dataset page: https://huggingface.co/datasets/Exgentic/agent-llm-traces-v2.thai_exam
Dataset Card for Thai_Exam
ThaiExam is a Thai knowledge benchmarking dataset, consisting of multiple-choice questions from examinations in Thailand. The dataset was originally developed for evaluating Typhoon (Thai LLM). This dataset contains 5 splits corresponding to 5 examinations as follows:
ONET: The Ordinary National Educational Test (ONET) is an examination for students in Thailand. This dataset is based on the grade-12 ONET exam, comprising 4 subjects and each question has 5… See the full description on the dataset page: https://huggingface.co/datasets/typhoon-ai/thai_exam.dr-saeid-ghezelbaash-entity-data
Dr. Saeed Ghezelbash Public Knowledge Graph
A public, physician-authored knowledge graph and multilingual retrieval dataset by Dr. Saeed Ghezelbash, a physician in Kermanshah, Iran. It connects physician identity, aesthetic medicine services, published question-answer content and cited evidence for entity resolution and evidence-grounded AI retrieval.
The canonical source is the official website and Dataset graph. This Hugging Face repository is its AI distribution. The… See the full description on the dataset page: https://huggingface.co/datasets/doctor-ghezelbaash/dr-saeid-ghezelbaash-entity-data.casimedicos-exp
Antidote CasiMedicos Dataset - Possible Answers Explanations in Resident Medical Exams
We present a new multilingual parallel medical dataset of commented medical exams which includes not only explanatory arguments
for the correct answer but also arguments to explain why the remaining possible answers are incorrect.
This dataset can be used for various NLP tasks including: Medical Question Answering, Explanatory Argument Extraction or Explanation Generation.
The… See the full description on the dataset page: https://huggingface.co/datasets/HiTZ/casimedicos-exp.BaisBenchThis is the dataset container for the Biological AI Scientist Benchmark (BAISBench). It's a benchmark designed to assess AI scientists' ability to generate biological discoveries through data analysis and reasoning with external knowledge.
This benchmark contains two tasks:
Data Process and cell Type Annotation task (BAIS-DPTA): This task includes 15 single-cell datasets to assess AI scientists' ability to annotate cell types, a fundamental challenge in single-cell analysis. To enable… See the full description on the dataset page: https://huggingface.co/datasets/EperLuo/BaisBench.ExploreToM
Data sample for ExploreToM: Program-guided adversarial data generation for theory of mind reasoning
ExploreToM is the first framework to allow large-scale generation of diverse and challenging theory of mind data for robust training and evaluation.
Our approach leverages an A* search over a custom domain-specific language to produce complex story structures and novel, diverse, yet plausible scenarios to stress test the limits of LLMs.
Our A* search procedure aims to find… See the full description on the dataset page: https://huggingface.co/datasets/facebook/ExploreToM.discrim-eval
Dataset Card for Discrim-Eval
Dataset Summary
The data contains a diverse set of prompts covering 70 hypothetical decision scenarios, ranging from approving a loan to providing press credentials.
Each prompt instructs the model to make a binary decision (yes/no)
about a particular person described in the prompt.
Each person is described in terms of three demographic attributes:
age (ranging from 20 to 100 in increments of 10), gender (male, female, non-binary)
, and race… See the full description on the dataset page: https://huggingface.co/datasets/Anthropic/discrim-eval.Audio-Video-Engineering-Agentic-Tasks-1M
Audio/Video Engineering Agentic Tasks (1M)
Abstract
A highly specialized dataset comprising 1,029,459 in-context troubleshooting prompts and execution commands built for the deepest levels of media production. Unlike standard datasets that simulate clean, theoretical instructions, this matrix captures the chaotic, highly-detailed, and conversational reality of professional audio engineers, composers, and video editors mid-session. It is engineered to train multimodal AI… See the full description on the dataset page: https://huggingface.co/datasets/yatin-superintelligence/Audio-Video-Engineering-Agentic-Tasks-1M.BioMed-R1-Eval
Disentangling Reasoning and Knowledge in Medical Large Language Models
This is the evaluation dataset accompanying our paper, comprising 11 publicly available biomedical benchmarks. We disentangle each benchmark question into either medical reasoning or medical knowledge categories.
Additionally, we provide a set of adversarial reasoning traces designed to evaluate the robustness of medical reasoning models.
For more details, please refer to our GitHub.
If you find this work useful… See the full description on the dataset page: https://huggingface.co/datasets/zou-lab/BioMed-R1-Eval.chess-evaluations
Chess Evaluations Dataset
This dataset contains chess positions represented in FEN (Forsyth-Edwards Notation) along with their evaluations and next moves for tactical evals. The dataset is divided into three configurations:
tactics: Includes chess positions, their evaluations, and the best move in the position.
randoms: Contains random chess positions and their evaluations.
chess_data: General chess positions with evaluations.
This is an in progress dataset which contains millions… See the full description on the dataset page: https://huggingface.co/datasets/ssingh22/chess-evaluations.sp500-earnings-transcripts
S&P 500 Earnings Call Transcripts
Dataset Description
This dataset provides earnings call transcripts for S&P 500 companies, primarily covering 2014-2024, along with quarterly financial metrics and company fundamentals.
📄 Paper: This dataset was prepared for and used in Ca'Zorzi, Manu, Lopardo. Verba Volant, Transcripta Manent: What Corporate Earnings Calls Reveal About the AI Stock Rally. No. 3093. European Central Bank, 2025.
Coverage Statistics
Time… See the full description on the dataset page: https://huggingface.co/datasets/glopardo/sp500-earnings-transcripts.EDGAR_FILINGS_DATASET
SFD: SEC Filings Dataset (v1)
SFD-v1 is an open, layout-faithful reconstruction of U.S. Securities and Exchange Commission (SEC) EDGAR filings into token-efficient MultiMarkdown (MMD), targeted at long-context language modeling, financial reasoning, document understanding, and evaluation.
This release covers filings from January 2022 through June 2025 (~3.4M filings), produced by the SFD parser described in:
The SEC Filings Dataset: Reconstructing U.S. Corporate and Financial… See the full description on the dataset page: https://huggingface.co/datasets/anonymous-md/EDGAR_FILINGS_DATASET.epstein-data
Epstein DOJ Document Archive v2
1.42 million OCR'd documents from the Department of Justice Jeffrey Epstein document release, with structured entity extraction, vector embeddings, financial transactions, communication records, and a forensic audit trail.
Frontend: epstein.academy
What's New in v2
10.6M entities (up from 8.5M) — expanded NER extraction
2.1M chunk embeddings (up from 1.96M) — more documents embedded
49,770 financial transactions — credit card and bank… See the full description on the dataset page: https://huggingface.co/datasets/kabasshouse/epstein-data.OlymMATH-eval
OlymMATH Evaluation Results
OlymMATH is a dataset we introduced in Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Models by Haoxiang Sun, Yingqian Min, Zhipeng Chen, Wayne Xin Zhao, Zheng Liu, Zhongyuan Wang, Lei Fang, and Ji-Rong Wen. You can find more information on GitHub and HuggingFace 🤗.
We have made our evaluation results for the avg@{8, 64} and cons@{8, 64} metrics in this dataset publicly available for academic research… See the full description on the dataset page: https://huggingface.co/datasets/RUC-AIBOX/OlymMATH-eval.extraglue
This is the dataset card for extraGLUE.
You may be interested in some of the other datasets for Portuguese and in the models trained with them,
namely Albertina (encoders) and Gervásio (decoders) families.
ExtraGLUE
ExtraGLUE is a Portuguese dataset obtained by the automatic translation of some of the tasks in the GLUE and SuperGLUE benchmarks.
Two variants of Portuguese are considered, namely European Portuguese and American Portuguese.
The dataset is… See the full description on the dataset page: https://huggingface.co/datasets/PORTULAN/extraglue.EU-MMLU
EU-oriented Massive Multitask Language Understanding (EU-MMLU)
Dataset description
The MMLU benchmarking dataset (https://arxiv.org/pdf/2009.03300) is a widely recognized benchmark of general knowledge attained by AI models. It covers a broad range of topics from 57 different categories, covering elementary-level knowledge up to advanced professional subjects like law, physics, history, and computer science.
The European Commission’s Directorate-General for… See the full description on the dataset page: https://huggingface.co/datasets/EC-DGT-AI/EU-MMLU.NautData
NautData
Paper | Project Page | Code
NautData is a large-scale underwater instruction-following dataset containing 1.45 million image-text pairs. It was constructed to bridge the gap in large-scale underwater multi-task instruction-tuning datasets, which are crucial for advancing underwater scene understanding methods. The dataset enables the development and thorough evaluation of underwater Large Multimodal Models (LMMs).
This dataset was introduced in the paper NAUTILUS: A Large… See the full description on the dataset page: https://huggingface.co/datasets/H-EmbodVis/NautData.excytin-benchThe dataset is composed of 2 parts.
The raw logs stored as csv files that will be used in a MYSQL database. All files are stored in the data_anonymized.tar.gz tar file. You should download it directly, instead of loading it using huggingface load_dataset function. Use the framework provided in the github repo to setup the log database.
The question answer pairs for the benchmark used in the paper can be found under the questions directory. These can be loaded directly using the hugging face… See the full description on the dataset page: https://huggingface.co/datasets/anandmudgerikar/excytin-bench.mmlu-okapi-eval-es
MMLU translated to Spanish
This dataset was generated by the Natural Language Processing Group of the University of Oregon, where they used the
original MMLU dataset in English and translated it into different languages using ChatGPT.
This dataset only contains the Spanish translation, but the following languages are also covered within the original
subsets posted by the University of Oregon at http://nlp.uoregon.edu/download/okapi-eval/datasets/.
Disclaimer
All the… See the full description on the dataset page: https://huggingface.co/datasets/alvarobartt/mmlu-okapi-eval-es.china-effective-laws-regulations
全国现行法律法规合集
现行有效的中华人民共和国法律、行政法规、监察法规、地方性法规、司法解释结构化文本。一部法规一行，一条法条一行，供查阅、检索、RAG 和法律 NLP 使用。
数据来自全国人大常委会办公厅 国家法律法规数据库，下载口径为官网的 「有效及尚未生效」。正文由 Word 原文用脚本抽取，未经大模型改写。
这不是官方汇编，不能替代公报或标准文本，也不能作为法律意见。 电子文本与标准文本不一致时，以法律规定的标准文本为准。
快照日期：2026-08-26
效力说明
本数据集 以现行有效法律法规为主体：
效力 status
法规份数
说明
有效
17,649
现行有效，默认应使用这一部分
尚未生效
7
已公布、施行日晚于快照日
失效
45
文件名含「失效」，多为已到期的全国人大常委会试点授权决定
使用时请筛选 status == "有效"，即可得到现行有效文本。同一部法若有修正前后多个版本，均予保留，用 filename_date 区分，采用最新日期即可。… See the full description on the dataset page: https://huggingface.co/datasets/senry5433/china-effective-laws-regulations.eecc
Extrinsic Evaluation of Cultural Competence in LLMs
In this repository, we release the data used in our paper "Extrinisic Evaluation of Cultural Competence in Large Language Models".
In this work, we analyse the extent and characteristics of variations in model outputs when explicit cue of culture, nationality is present in the prompt. We evaluate models on two user-facing tasks: Question Answering (QA) and Story Generation.
We use 193 nationalities present in… See the full description on the dataset page: https://huggingface.co/datasets/shaily99/eecc.EsBBQ
Spanish Bias Benchmark for Question Answering (EsBBQ)
The Spanish Bias Benchmark for Question Answering (EsBBQ) is an adaptation of the original BBQ to the Spanish language and the social context of Spain.
Dataset Description
This dataset is used to evaluate social bias in LLMs in a multiple-choice Question Answering (QA) setting and along 10 social categories: Age, Disability Status, Gender, LGBTQIA, Nationality, Physical Appearance, Race/Ethnicity, Religion… See the full description on the dataset page: https://huggingface.co/datasets/BSC-LT/EsBBQ.airbnb_embeddings
Overview
This dataset consists of AirBnB listings with property descriptions, reviews, and other metadata.
It also contains text embeddings of the property descriptions as well as image embeddings of the listing image. The text embeddings were created using OpenAI's text-embedding-3-small model and the image embeddings using OpenAI's clip-vit-base-patch32 model available on Hugging Face.
The text embeddings have 1536 dimensions, while the image embeddings have 512 dimensions.… See the full description on the dataset page: https://huggingface.co/datasets/MongoDB/airbnb_embeddings.stack-exchange-dataset
Overview
This dataset consists of three TSV files, namely: cs.tsv, ds.tsv, and p.tsv.
Each file includes the data for the questions asked on a Stack Exchange (SE) question-answering community, from the creation of the community until May 2021.
cs.tsv --> Computer Science SE
ds.csv --> Data Science SE
p.csv --> Political Science SE
File Structure
Each file has the following columns:
id: the question id
title: the title of the question
body: the body or text of the… See the full description on the dataset page: https://huggingface.co/datasets/habedi/stack-exchange-dataset.global-censorship-index
Voidly Global Censorship Index
Real-time internet censorship measurements for 200 countries, based on 38,780,449+ OONI network probes.
Dataset Description
The Global Censorship Index provides country-level internet censorship scores derived from actual network measurements. Unlike annual expert assessments, this data updates daily.
Key Statistics
Countries covered: 200
Total measurements: 38,780,449
Severe censorship: 1 countries
High censorship: 6… See the full description on the dataset page: https://huggingface.co/datasets/emperor-mew/global-censorship-index.IndustryInstruction_Finance-Economics
IndustryInstruction: Finance & Economics
This repository contains the IndustryInstruction: Finance & Economics domain subset of BAAI/IndustryInstruction.
Refer to the parent dataset card for data construction, intended use, limitations,
and licensing details.
Citation
If you use this dataset in your work, please cite IndustryInstruction:
@misc{shi2024industryinstruction,
title = {IndustryInstruction},
author = {Xiaofeng Shi and Lulu Zhao and Hua Zhou… See the full description on the dataset page: https://huggingface.co/datasets/BAAI/IndustryInstruction_Finance-Economics.EnokiQA
EnokiQA
EnokiQA is an annotated dataset for fine-grained hallucination detection in long-form question answering. Each example contains a factual question, a no-context LLM answer, the full Wikipedia article used as verification evidence, sentence-grouped factual triples, and per-triple NLI and hallucination probabilities.
The dataset is dual-granularity: every hallucination label is attached to a claim (an extracted triple) and projected to a character span of the answer. The… See the full description on the dataset page: https://huggingface.co/datasets/s-nlp/EnokiQA.