datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
amazon_massive_scenario
MassiveScenarioClassification
An MTEB dataset
Massive Text Embedding Benchmark
MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages
Task category
t2c
Domains
Spoken
Reference
https://arxiv.org/abs/2204.08582
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["MassiveScenarioClassification"])
evaluator =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/amazon_massive_scenario.pii-masking-300k
👉 Looking for the newest release? The current flagship is ai4privacy/pii-masking-openpii-1.5m. 1.6M samples, 30 languages, 19 PII classes, Asia Pacific extension.?** The current flagship is ai4privacy/pii-masking-openpii-1m. 1.4M samples, 23 languages, 19 PII classes.
Purpose and Features
🌍 World's largest open dataset for privacy masking 🌎
The dataset is useful to train and evaluate models to remove personally identifiable and sensitive information from text, especially in… See the full description on the dataset page: https://huggingface.co/datasets/ai4privacy/pii-masking-300k.pii-masking-200k
👉 Looking for the newest release? The current flagship is ai4privacy/pii-masking-openpii-1.5m. 1.6M samples, 30 languages, 19 PII classes, Asia Pacific extension.?** The current flagship is ai4privacy/pii-masking-openpii-1m. 1.4M samples, 23 languages, 19 PII classes.
Ai4Privacy Community
Join our community at https://discord.gg/FmzWshaaQT to help build open datasets for privacy masking.
Purpose and Features
Previous world's largest open dataset for privacy.… See the full description on the dataset page: https://huggingface.co/datasets/ai4privacy/pii-masking-200k.pii-masking-openpii-1.5m
OpenPII 1.5M: Multilingual PII Masking Dataset (Asia Pacific Extension)
📖 More information: www.ai4privacy.com/datasets/pii-masking-3m-asia-pacific
Overview
The OpenPII 1.5M dataset extends OpenPII 1M
with a new Asia Pacific corpus, bringing global coverage to 30 languages
across Europe, Americas, and Asia Pacific.
This is the flagship release of the PII-Masking-3M family, the world's
largest open multilingual PII masking corpus. Built to advance open… See the full description on the dataset page: https://huggingface.co/datasets/ai4privacy/pii-masking-openpii-1.5m.amazon_massive_intent_en-USpii-masking-openpii-1m
OpenPII 1M — Multilingual PII Masking Dataset
Overview
The OpenPII 1M dataset is a large-scale, multilingual collection of 1,428,143 synthetic text examples with fine-grained PII (Personally Identifiable Information) annotations, spanning 23 European languages and 19 entity types.
Built to advance open research in privacy-preserving NLP, this dataset enables the development and benchmarking of Named Entity Recognition (NER) models, token classification pipelines… See the full description on the dataset page: https://huggingface.co/datasets/ai4privacy/pii-masking-openpii-1m.pii-masking-400k
👉 Looking for the newest release? The current flagship is ai4privacy/pii-masking-openpii-1.5m. 1.6M samples, 30 languages, 19 PII classes, Asia Pacific extension.?** The current flagship is ai4privacy/pii-masking-openpii-1m. 1.4M samples, 23 languages, 19 PII classes.
Purpose and Features
🌍 World's largest open dataset for privacy masking 🌎
The dataset is useful to train and evaluate models to remove personally identifiable and sensitive information from text, especially in… See the full description on the dataset page: https://huggingface.co/datasets/ai4privacy/pii-masking-400k.open-pii-masking-500k-ai4privacy
👉 Looking for the newest release? The current flagship is ai4privacy/pii-masking-openpii-1.5m. 1.6M samples, 30 languages, 19 PII classes, Asia Pacific extension.?** The current flagship is ai4privacy/pii-masking-openpii-1m. 1.4M samples, 23 languages, 19 PII classes.
🌍 World's largest open dataset for privacy masking 🌎
The dataset is useful to train and evaluate models to remove personally identifiable and sensitive information from text, especially in the context of AI… See the full description on the dataset page: https://huggingface.co/datasets/ai4privacy/open-pii-masking-500k-ai4privacy.MAST-Data
MAD: Multi-Agent System Traces Dataset
Execution traces from multi-agent systems (MAS), annotated with the Multi-Agent Systems
Failure Taxonomy (MAST). Each record gives the MAS, the LLM behind it, the benchmark task,
the full trace, and binary annotations for the 14 MAST failure modes.
Code: https://github.com/multi-agent-systems-failure-taxonomy/MAST
1642 traces · 7 MAS frameworks · 8 benchmarks · 5 LLMs.
Files
file
rows
MAD_full_dataset.json
1642… See the full description on the dataset page: https://huggingface.co/datasets/mcemri/MAST-Data.c4-en-html-with-metadata-ppl-cleanFile list:
"c4-en-html_cc-main-2019-18_pq00-000.jsonl.gz",
"c4-en-html_cc-main-2019-18_pq00-001.jsonl.gz",
"c4-en-html_cc-main-2019-18_pq00-002.jsonl.gz",
"c4-en-html_cc-main-2019-18_pq00-003.jsonl.gz",
"c4-en-html_cc-main-2019-18_pq00-004.jsonl.gz",
"c4-en-html_cc-main-2019-18_pq00-005.jsonl.gz",
"c4-en-html_cc-main-2019-18_pq00-006.jsonl.gz",
"c4-en-html_cc-main-2019-18_pq00-007.jsonl.gz",
"c4-en-html_cc-main-2019-18_pq00-008.jsonl.gz",
"c4-en-html_cc-main-2019-18_pq00-009.jsonl.gz"… See the full description on the dataset page: https://huggingface.co/datasets/masoudjs/c4-en-html-with-metadata-ppl-clean.swe-agent-tool-rubrics-860
SWE Agent 逐 turn 工具调用评判数据集（860 个决策点）
本数据集来自 2026-08-06 的一次实验：**从真实 SWE agent 轨迹中归纳"怎么判断一次工具调用的好坏"**。
包含两个文件：
文件
行数
大小
内容
cases.jsonl
860
5.0 MB
决策点原始数据（题目、历史、两个候选命令、执行结果、现役判官打分）
map_io.jsonl
860
9.6 MB
每个决策点喂给 GPT-5.6 的完整 prompt 原文与完整回复
两个文件通过 case_id 一一对应。
背景：为什么是"按动作分类"而不是"按工具分类"
轨迹来自 slime 的 minimal harness，该 harness 只暴露一个工具 bash
（slime/agent/harness/minimal.py 里的 BASH_TOOL），全部 328,270 次调用的工具名都是 bash。
所以"不同工具用不同 rubric"无法按工具名实现，只能按命令在干什么分类。… See the full description on the dataset page: https://huggingface.co/datasets/MasterVito/swe-agent-tool-rubrics-860.pii-masking-65k
👉 Looking for the newest release? The current flagship is ai4privacy/pii-masking-openpii-1.5m. 1.6M samples, 30 languages, 19 PII classes, Asia Pacific extension.?** The current flagship is ai4privacy/pii-masking-openpii-1m. 1.4M samples, 23 languages, 19 PII classes.
Purpose and Features
The purpose of the model and dataset is to remove personally identifiable information (PII) from text, especially in the context of AI assistants and LLMs.
The model is a fine-tuned version… See the full description on the dataset page: https://huggingface.co/datasets/ai4privacy/pii-masking-65k.amazon_massive_intent_zh-CNmassive-templates
Purpose. This dataset was collected specifically for intent-parser benchmarking, independently from any OVOS skill. Skill-derived utterances tend to overfit the exact phrasings a plugin was tuned on; this data is drawn from a disjoint source so it measures whether an OVOS intent plugin generalizes rather than memorizes. It is part of the OVOS intent-classification datasets used by the OVOS Plugin Arena intent benchmark.
Funding
Developed by TigreGotico for OpenVoiceOS as part… See the full description on the dataset page: https://huggingface.co/datasets/OpenVoiceOS/massive-templates.InjongoIntent
Dataset Card for InjongoIntent
Dataset Summary
InjongoIntent
Languages
There are 17 languages available :
Dataset Structure
Data Instances
The examples look like this for English:
from datasets import load_dataset
data = load_dataset('masakhane/InjongoIntent', 'eng')
# Please, specify the language code
# A data point example is below:
{
}
Data Fields
question: the question string to a grade school math problem.
answer: the… See the full description on the dataset page: https://huggingface.co/datasets/masakhane/InjongoIntent.amazon_massive_intent_de-DEamazon_massive_scenario_en-USLOOPerSet
LOOPerSet: A Large-Scale Dataset for Data-Driven Polyhedral Optimization
Dataset at a Glance
LOOPerSet is a corpus of 28 million labeled compilation traces designed for machine learning research in compilers and systems. It maps synthetically generated loop nests and complex optimization sequences to ground-truth execution times measured on physical hardware. Transformation sequences were generated using a polyhedral compilation framework to ensure they… See the full description on the dataset page: https://huggingface.co/datasets/Mascinissa/LOOPerSet.uhura-truthfulqa
Dataset Card for Uhura-TruthfulQA
Dataset Summary
TruthfulQA is a widely recognized safety benchmark designed to measure the truthfulness of language model outputs across 38 categories, including health, law, finance, and politics. The English version of the benchmark originates from TruthfulQA: Measuring How Models Mimic Human Falsehoods (Lin et al., 2022) and consists of 817 questions in both multiple-choice and generation formats, targeting common misconceptions and… See the full description on the dataset page: https://huggingface.co/datasets/masakhane/uhura-truthfulqa.uhura-arc-easy
Dataset Card for Uhura-Arc-Easy
Dataset Summary
Uhura-ARC-Easy is a widely recognized scientific question answering benchmark composed of multiple-choice science questions derived from grade-school examinations that test various styles of knowledge and reasoning.
The original English version of the benchmark originates from Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge (Clark et al., 2018) and is divided into "Challenge" and "Easy"… See the full description on the dataset page: https://huggingface.co/datasets/masakhane/uhura-arc-easy.amazon_massive_intent_am-ETamazon_massive_intent_hi-INamazon_massive_intent_ja-JPpii-masking-benchmark
🎭 PIIMB: PII Masking Benchmark
PIIMB measures zero-shot PII masking: a model's ability to mask any PII out-of-the-box, without fine-tuning or label customization.
We designed its evaluation to be character-based and label-agnostic (more details below).
This is an attempt to reflect the most common deployment scenarios where users need broad coverage across document and entity types with no downstream customization (e.g general privacy or data protection).
For specialised use… See the full description on the dataset page: https://huggingface.co/datasets/piimb/pii-masking-benchmark.indic-queries-2026
MAST Indic Queries 2026
This dataset contains the Indic query set for MAST @ FIRE 2026, the Multilingual Agentic Search Track. MAST evaluates whether multilingual agentic search systems can answer complex questions posed in different languages.
MAST builds on BrowseComp-Plus (ACL 2026), a reproducible and verifiable extension of BrowseComp with challenging English queries, a verified English corpus of roughly 100K web-sourced documents, and human judgments. In the 2026 MAST… See the full description on the dataset page: https://huggingface.co/datasets/mast-benchmark/indic-queries-2026.amazon_massive_intent_ar-SAamazon_massive_intent_sw-KEmultilingual-queries-2026
MAST Multilingual Queries 2026
This dataset contains the multilingual query set for MAST @ FIRE 2026, the Multilingual Agentic Search Track. MAST evaluates whether multilingual agentic search systems can answer complex questions posed in different languages.
MAST builds on BrowseComp-Plus (ACL 2026), a reproducible and verifiable extension of BrowseComp with challenging English queries, a verified English corpus of roughly 100K web-sourced documents, and human judgments. In the… See the full description on the dataset page: https://huggingface.co/datasets/mast-benchmark/multilingual-queries-2026.amazon_massive_intent_th-THamazon_massive_intent_ru-RU