datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
MUMA-TOM-BENCHMARK
MuMA-ToM: Multi-modal Multi-Agent Theory of Mind AAAI 2025 (Oral)
[🏠Homepage] [💻Code] [📝Paper]
MuMA-ToM is the first multi-modal Theory of Mind benchmark designed to evaluate mental reasoning in embodied multi-agent interactions. The benchmark was designed with several key features in mind:
It is factually correct, concise, and readable.
It requires integrating information from multiple modalities to answer the questions.
It tests understanding of multi-agent interactions… See the full description on the dataset page: https://huggingface.co/datasets/SCAI-JHU/MUMA-TOM-BENCHMARK.general-reasoning-ift-pairs
Reasoning-IFT Pairs (General Domain)
This dataset provides the largest set of IFT and Reasoning answers pairs for a set of general domain queries (cf: math-domain).It is based on the Infinity-Instruct dataset, an extensive and high-quality collection of instruction fine-tuning data.
We curated 900k queries from the 7M_core subset of Infinity-Instruct, which covers multiple domains including general knowledge, commonsense Q&A, coding, and math.For each query… See the full description on the dataset page: https://huggingface.co/datasets/Scale-or-Reason/general-reasoning-ift-pairs.MultiChallenge
MultiChallenge
MultiChallenge is a benchmark for evaluating large language models on multi-turn conversations. It tests whether models can maintain context, follow instructions, and reason correctly across extended dialogues.
Evaluation Axes
Axis
Description
INFERENCE_MEMORY
Tests whether the model can recall and reason over information from earlier turns
INSTRUCTION_RETENTION
Tests whether the model continues to follow instructions given in earlier turns… See the full description on the dataset page: https://huggingface.co/datasets/ScaleAI/MultiChallenge.math-reasoning-ift-pairs
Reasoning-IFT Pairs (Math Domain)
Paper | Project Page
This dataset provides the largest set of IFT and Reasoning answers pairs for a set of math queries (cf: general-domain).
It is based on the Llama-Nemotron-Post-Training dataset, an extensive and high-quality collection of math instruction fine-tuning data.
We curated 150k queries from the math subset of Llama-Nemotron-Post-Training, which covers multiple domains of math questions.For each query, we used… See the full description on the dataset page: https://huggingface.co/datasets/Scale-or-Reason/math-reasoning-ift-pairs.lhaw
Dataset Card for LHAW: Long-Horizon Augmented Workflows
LHAW is a modular, dataset-agnostic synthetic pipeline that transforms well-specified tasks into controllable underspecified variants by systematically removing information across four dimensions—Goals, Constraints, Inputs, and Context—at configurable severity levels.
This dataset release contains 285 underspecified task variants derived from TheAgentCompany, SWE-Bench Pro, and MCP-Atlas, and is used to study how current agents… See the full description on the dataset page: https://huggingface.co/datasets/ScaleAI/lhaw.DrugDiscoveryBench-Preview
DrugDiscoveryBench (Preview)
DrugDiscoveryBench is a benchmark of 82 expert-authored, execution-grounded tasks spanning the early
drug-discovery and life-sciences workflow (target identification & genetics, database screening,
patent mining, cheminformatics, structural reasoning, SAR & affinity, molecular biology). Each task
asks an agent to carry out a multi-step biomedical investigation and produce a terse final answer.
This is the Preview release: task prompts and metadata… See the full description on the dataset page: https://huggingface.co/datasets/ScaleAI/DrugDiscoveryBench-Preview.SciPredict
SciPredict: Can LLMs Predict the Outcomes of Research Experiments?
Paper: SciPredict: Can LLMs Predict the Outcomes of Research Experiments in Natural Sciences?
Overview
SciPredict is a benchmark evaluating whether AI systems can predict experimental outcomes in physics, biology, and chemistry. The dataset comprises 405 questions derived from recently published empirical studies (post-March 2025), spanning 33 subdomains.
Dataset Structure
Total Questions: 405… See the full description on the dataset page: https://huggingface.co/datasets/ScaleAI/SciPredict.scandi-qaScandiQA is a dataset of questions and answers in the Danish, Norwegian, and Swedish
languages. All samples come from the Natural Questions (NQ) dataset, which is a large
question answering dataset from Google searches. The Scandinavian questions and answers
come from the MKQA dataset, where 10,000 NQ samples were manually translated into,
among others, Danish, Norwegian, and Swedish. However, this did not include a
translated context, hindering the training of extractive question answering models.
We merged the NQ dataset with the MKQA dataset, and extracted contexts as either "long
answers" from the NQ dataset, being the paragraph in which the answer was found, or
otherwise we extract the context by locating the paragraphs which have the largest
cosine similarity to the question, and which contains the desired answer.
Further, many answers in the MKQA dataset were "language normalised": for instance, all
date answers were converted to the format "YYYY-MM-DD", meaning that in most cases
these answers are not appearing in any paragraphs. We solve this by extending the MKQA
answers with plausible "answer candidates", being slight perturbations or translations
of the answer.
With the contexts extracted, we translated these to Danish, Swedish and Norwegian using
the DeepL translation service for Danish and Swedish, and the Google Translation
service for Norwegian. After translation we ensured that the Scandinavian answers do
indeed occur in the translated contexts.
As we are filtering the MKQA samples at both the "merging stage" and the "translation
stage", we are not able to fully convert the 10,000 samples to the Scandinavian
languages, and instead get roughly 8,000 samples per language. These have further been
split into a training, validation and test split, with the former two containing
roughly 750 samples. The splits have been created in such a way that the proportion of
samples without an answer is roughly the same in each split.quote-repetition
quote-repetition (Joe Cavanagh, Andrew Gritsevskiy, and Derik Kauffman of Cavendish Labs)
General description
In this task, the authors ask language models to repeat back sentences given in the prompt, with few-shot examples to help it recognize the task. Each prompt contains a famous quote with a modified ending to mislead the model into completing the sequence with the famous ending rather than with the ending given in the prompt. The authors find that smaller models… See the full description on the dataset page: https://huggingface.co/datasets/inverse-scaling/quote-repetition.DrugDiscoveryBench
DrugDiscoveryBench
DrugDiscoveryBench is a benchmark of 82 expert-authored, execution-grounded tasks spanning the early
drug-discovery and life-sciences workflow (target identification & genetics, database screening,
patent mining, cheminformatics, structural reasoning, SAR & affinity, molecular biology). Each task
asks an agent to carry out a multi-step biomedical investigation and produce a terse final answer,
graded against a ground-truth answer and an outcome + process… See the full description on the dataset page: https://huggingface.co/datasets/ScaleAI/DrugDiscoveryBench.inverse-scaling-ttc-main
Inverse Scaling in Test-Time Compute
Paper: Inverse Scaling in Test-Time Compute
Project Page: https://safety-research.github.io/inverse-scaling-ttc/
Abstract
We construct evaluation tasks where extending the reasoning length of Large Reasoning Models (LRMs) deteriorates performance, exhibiting an inverse scaling relationship between test-time compute and accuracy. Our evaluation tasks span four categories: simple counting tasks with distractors, regression tasks with… See the full description on the dataset page: https://huggingface.co/datasets/inverse-scaling-ttc/inverse-scaling-ttc-main.NeQA
NeQA: Can Large Language Models Understand Negation in Multi-choice Questions? (Zhengping Zhou and Yuhui Zhang)
General description
This task takes an existing multiple-choice dataset and negates a part of each question to see if language models are sensitive to negation. The authors find that smaller language models display approximately random performance whereas the performance of larger models become significantly worse than random.
Language models failing to follow… See the full description on the dataset page: https://huggingface.co/datasets/inverse-scaling/NeQA.redefine-math
redefine-math (Xudong Shen)
General description
In this task, the author tests whether language models are able to work with common symbols when they are redefined to mean something else. The author finds that larger models are more likely to pick the answer corresponding to the original definition rather than the redefined meaning, relative to smaller models.
This task demonstrates that it is difficult for language models to work with new information given at inference… See the full description on the dataset page: https://huggingface.co/datasets/inverse-scaling/redefine-math.ScalpelBench
ScalpelBench
ScalpelBench is a compact instruction-tuning corpus developed for controlled
studies of model compression, with a particular focus on layer pruning,
post-pruning recovery, and capability retention. The released corpus contains
approximately 0.1B tokens of instruction-response data spanning general
English, Chinese, mathematical reasoning, and code generation.
Mixture Design
The mixture proportions follow high-level capability-balancing principles… See the full description on the dataset page: https://huggingface.co/datasets/freeai-org/ScalpelBench.hindsight-neglect-10shot
inverse-scaling/hindsight-neglect-10shot (‘The Floating Droid’)
General description
This task tests whether language models are able to assess whether a bet was worth taking based on its expected value. The author provides few shot examples in which the model predicts whether a bet is worthwhile by correctly answering yes or no when the expected value of the bet is positive (where the model should respond that ‘yes’, taking the bet is the right decision) or negative (‘no’… See the full description on the dataset page: https://huggingface.co/datasets/inverse-scaling/hindsight-neglect-10shot.KoALA
KoALa-Bench: Korean Audio Language Model Benchmark
KoALa-Bench is a comprehensive benchmark for evaluating Large Audio Language Models (LALMs) on Korean speech understanding. It covers six tasks spanning both conventional speech processing and novel speech faithfulness evaluation, designed to test whether models can reason over the acoustic and linguistic content of Korean speech.
Tasks
KoALa-Bench consists of six evaluation tasks organized into two categories.… See the full description on the dataset page: https://huggingface.co/datasets/scailaboratory/KoALA.three-mountain-scaling
ThreeMountain_Scaling
Segment
Meaning
GO
Geometric Object — indicates the object type used (e.g., GO for geometric, RO for real objects).
L / Arc
Object Arrangement — defines how objects are arranged spatially. L means L-shape arrangement; Arc means objects are placed in an arc.
RC
Random Character Position — RC = True: character position is randomized.
FC
Fixed Character Position — FC = True: character stays fixed.
RS
Random Scale — RS = True: objects are… See the full description on the dataset page: https://huggingface.co/datasets/grow-ai-like-a-child/three-mountain-scaling.DensingLaw-ScalingBench
DensingLaw-ScalingBench
This dataset was created to enable a more accurate performance scaling law estimation of Large Language Models (LLMs).
This dataset is released as part of our paper, Densing Law of LLMs.
📜 Paper
💡 Overview
This repository contains the open-source dataset used for calculating conditional loss in our LLM density evaluation framework.
LLM density is defined as the ratio of effective parameter size to actual parameter size, where effective… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/DensingLaw-ScalingBench.scas_verified_teacher_pool
SCAS Verified Teacher Answer Pool
This dataset provides an aligned, correctness-verified pool of
teacher-generated mathematical reasoning solutions for studying
student-centric data selection in distillation.
The release covers two source corpora, Hendrycks MATH and DeepScaleR. For each
corpus, we retain the subset of questions on which all nine selected teacher
models produce verified correct answers. Each retained question is paired with
nine alternative teacher solutions, one… See the full description on the dataset page: https://huggingface.co/datasets/Student-Centric-Answer-Sampling/scas_verified_teacher_pool.ScaleCap-450k
[Paper] https://arxiv.org/abs/2506.19848
[GitHub] https://github.com/Cooperx521/ScaleCap
ScaleCap450k-Hyper detailed and high quality image caption
Dataset details
This dataset contains 450k image-caption pairs, where the captions are annotated using the ScaleCap pipeline.
For more details, please refer to the paper.
In collecting images for our dataset, we primarily focus on two
aspects: diversity and richness of image content. Given that the ShareGPT4V-100k already… See the full description on the dataset page: https://huggingface.co/datasets/long-xing1/ScaleCap-450k.inverse-scaling-ttc-main
Inverse Scaling in Test-Time Compute
Note: This is an anonymized repository.
Abstract
We construct evaluation tasks where extending the reasoning length of Large Reasoning Models (LRMs) deteriorates performance, exhibiting an inverse scaling relationship between test-time compute and accuracy. Our evaluation tasks span four categories: simple counting tasks with distractors, regression tasks with spurious features, deduction tasks with constraint tracking, and advanced AI… See the full description on the dataset page: https://huggingface.co/datasets/anonscaling/inverse-scaling-ttc-main.the-Embodiment-of-Scarlet-Devil-Instruct-Alpaca-QA-JP-v1
Converted QA Dataset
このデータセットは、easy-dataset-cliを使用して生成されたアルパカ形式の日本語Q&Aデータセットです。
データセット概要
総エントリ数: 97,202
形式: Alpaca形式
言語: 日本語
ライセンス: MIT
データ構造
各エントリは以下の形式です：
{
"instruction": "質問文",
"input": "",
"output": "回答文",
"genre": "ジャンル",
"audience": "対象読者"
}
ジャンル分布
含まれるジャンル:
FAQ
PRD
RFP/提案書
アーキレビュー
エグゼクティブサマリ
ガイドライン/ポリシー
ケーススタディ
セキュリティレビュー
チュートリアル
ハンズオン課題
ベストプラクティス集
ワークショップ資料
実験レポート
対話形式
技術ブログ
教科書
業界別ケーススタディ
法務チェックリスト
運用Runbook
対象読者分布… See the full description on the dataset page: https://huggingface.co/datasets/MakiAi/the-Embodiment-of-Scarlet-Devil-Instruct-Alpaca-QA-JP-v1.Stack2Graph_VD_scala
Scala StackOverflow Vector Dataset
Summary
This Hugging Face dataset repository contains the Scala shard of the Stack2Graph vector-database component as restorable Qdrant artifacts plus portable Parquet fallback files.
Hugging Face uses one dataset repository per programming language, so this repository is directly cloneable without an extra top-level archive wrapper.
The artifacts are intended for semantic and hybrid retrieval, graph entry-point finding, and… See the full description on the dataset page: https://huggingface.co/datasets/Mo7art/Stack2Graph_VD_scala.rico_sca_refexp_syntheticThis dataset is intended for UI understanding, referring expression and action automation model training. It's based on the Rico SCA RefExp dataset from Google Research.
Introduced in the seq2act paper. See the [original Google repo](https://github.com/google-research/google-research/blob/master/seq2act/data_generation/README.md) for details and licensing info:The_Scamper-trainThemis_Scales
Themis Scales: Moral Dilemma Resolution
Dataset Description
This repository contains a seed dataset designed to explore the hypothesis that AI reasoning models can be guided to analyse and propose resolutions for moral dilemmas. The core objective is to demonstrate the potential of using a structured reasoning framework – specifically the Morality-as-Cooperation (MAC) theory – to navigate complex ethical situations.
This initial dataset serves as a proof-of-concept and a… See the full description on the dataset page: https://huggingface.co/datasets/Akhil-Theerthala/Themis_Scales.Scalingacorn
license: intel-research
tags:
code
legal
art
pretty_name: 'TubuTerp '
--- Two Charts = [products ] [Scent Patterns]
import pandas as pd
Create a dataframe for products table
data_products = { Blue Dream } 1=S/H , 2=Platinum Hydrolab / Stem Handcrafted
'id': [Thc26.2% ,Burns good , Taste slight citrus profile, but is easy to inhale and smokes very nice ], [very light tea smell with a sour twang that adds that high grade smell ]
'name': ['Product A', 'Product B'… See the full description on the dataset page: https://huggingface.co/datasets/Mi6paulino/Scalingacorn.Inverse-scaling-test