datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
xCodeEvalThe ability to solve problems is a hallmark of intelligence and has been an enduring goal in AI. AI systems that can create programs as solutions to problems or assist developers in writing programs can increase productivity and make programming more accessible. Recently, pre-trained large language models have shown impressive abilities in generating new codes from natural language descriptions, repairing buggy codes, translating codes between languages, and retrieving relevant code segments. However, the evaluation of these models has often been performed in a scattered way on only one or two specific tasks, in a few languages, at a partial granularity (e.g., function) level and in many cases without proper training data. Even more concerning is that in most cases the evaluation of generated codes has been done in terms of mere lexical overlap rather than actual execution whereas semantic similarity (or equivalence) of two code segments depends only on their ``execution similarity'', i.e., being able to get the same output for a given input.xlam-function-calling-60k
APIGen Function-Calling Datasets
Paper | Website | Models
This repo contains 60,000 data collected by APIGen, an automated data generation pipeline designed to produce verifiable high-quality datasets for function-calling applications. Each data in our dataset is verified through three hierarchical stages: format checking, actual function executions, and semantic verification, ensuring its reliability and correctness.
We conducted human evaluation over 600 sampled data points, and… See the full description on the dataset page: https://huggingface.co/datasets/lockon/xlam-function-calling-60k.xlam-function-calling-60k
APIGen Function-Calling Datasets
Paper | Website | Models
This repo contains 60,000 data collected by APIGen, an automated data generation pipeline designed to produce verifiable high-quality datasets for function-calling applications. Each data in our dataset is verified through three hierarchical stages: format checking, actual function executions, and semantic verification, ensuring its reliability and correctness.
We conducted human evaluation over 600 sampled data points… See the full description on the dataset page: https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k.xlam-function-calling-60k-parsed
[PARSED] APIGen Function-Calling Datasets (xLAM)
This dataset contains the full data from the original Salesforce/xlam-function-calling-60k
Subset name
multi-turn
parallel
multiple definition
Last turn type
number of dataset
xlam-function-calling-60k
no
yes
yes
tool_calls
60000
This is a re-parsing formatting dataset for the xLAM official dataset.
Load the dataset
from datasets import load_dataset
ds =… See the full description on the dataset page: https://huggingface.co/datasets/minpeter/xlam-function-calling-60k-parsed.HumanEval-XLThis dataset contains a viewer-friendly version of the dataset at FloatAI/HumanEval-XL. It is made available separately for the convenience of the vllm-code-harness package.
the-stack-smol-xl
Dataset Description
A small subset of the-stack dataset, with 87 programming languages, each has 10,000 random samples from the original dataset.
Languages
The dataset contains 87 programming languages:
'ada', 'agda', 'alloy', 'antlr', 'applescript', 'assembly', 'augeas', 'awk', 'batchfile', 'bison', 'bluespec', 'c',
'c++', 'c-sharp', 'clojure', 'cmake', 'coffeescript', 'common-lisp', 'css', 'cuda', 'dart', 'dockerfile', 'elixir',
'elm', 'emacs-lisp','erlang'… See the full description on the dataset page: https://huggingface.co/datasets/bigcode/the-stack-smol-xl.x_dataset_39
Bittensor Subnet 13 X (Twitter) Dataset
Dataset Summary
This dataset is part of the Bittensor Subnet 13 decentralized network, containing preprocessed data from X (formerly Twitter). The data is continuously updated by network miners, providing a real-time stream of tweets for various analytical and machine learning tasks.
For more information about the dataset, please visit the official repository.
Supported Tasks
The versatility of this… See the full description on the dataset page: https://huggingface.co/datasets/futuremoon/x_dataset_39.the-stack-smol-xs\XSTest
XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models
XSTest is a test suite designed to identify exaggerated safety / false refusal in Large Language Models (LLMs).
It comprises 250 safe prompts across 10 different prompt types, along with 200 unsafe prompts as contrasts.
The test suite aims to evaluate how well LLMs balance being helpful with being harmless by testing if they unnecessarily refuse to answer safe prompts that superficially… See the full description on the dataset page: https://huggingface.co/datasets/Paul/XSTest.DanbooruwildcardsThis is a set of wildcards for danbooru tags.
Artist：Prompts for random artist styles, covering approximately 0.6M different artists.Please select the appropriate version of the collection, ranging from 128 to 5000, based on the model's capabilities.The full version is not recommended for use as it includes too many artists with only one image on danbooru or other websites. Almost no model can generate a style that corresponds to these artists .
Characters："Characters" is a set of wildcards… See the full description on the dataset page: https://huggingface.co/datasets/X779/Danbooruwildcards.VRSBench
VRSBench
VRSBench is a Versatile Vision-Language Benchmark for Remote Sensing Image Understanding. It consists of 29,614 remote sensing images with detailed captions, 52,472 object refers, and 3123,221 visual question-answer pairs. It facilitates the training and evaluation of vision-language models across a broad spectrum of remote sensing image understanding tasks.
Using datasets
from datasets import load_dataset
fw = load_dataset("xiang709/VRSBench"… See the full description on the dataset page: https://huggingface.co/datasets/xiang709/VRSBench.xlsumWe present XLSum, a comprehensive and diverse dataset comprising 1.35 million professionally
annotated article-summary pairs from BBC, extracted using a set of carefully designed heuristics.
The dataset covers 45 languages ranging from low to high-resource, for many of which no
public dataset is currently available. XL-Sum is highly abstractive, concise,
and of high quality, as indicated by human and intrinsic evaluation.Lego-RL-2699
SWE-Lego-RL-2699
2,699 executable, difficulty-filtered SWE tasks for agentic RL, shipped in two
parallel views of the same instances:
View
Path
What it is
Official OpenSWE records
openswe_official_2699/
The original upstream GAIR/OpenSWE rows for exactly these 2,699 instances
Harbor RL environments
openswe_harbor_2699/
The same instances converted into ready-to-run task directories (+ the training index)
Both views cover the identical 2,699 instance_ids. The… See the full description on the dataset page: https://huggingface.co/datasets/Lego-X/Lego-RL-2699.Bactrian-X
Dataset Card for "Bactrian-X"
A. Dataset Description
Homepage: https://github.com/mbzuai-nlp/Bactrian-X
Repository: https://huggingface.co/datasets/MBZUAI/Bactrian-X
Paper: to-be-soon released
Dataset Summary
The Bactrain-X dataset is a collection of 3.4M instruction-response pairs in 52 languages, that are obtained by translating 67K English instructions (alpaca-52k + dolly-15k) into 51 languages using Google Translate API. The translated instructions… See the full description on the dataset page: https://huggingface.co/datasets/MBZUAI/Bactrian-X.CUA-Gym
CUA-Gym
CUA-Gym is a collection of verifiable computer-use agent tasks for reinforcement learning with verifiable rewards (RLVR). Each task pairs a natural-language instruction with executable setup artifacts and a Python reward function that checks task completion programmatically. For details, see the paper CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents.
This release contains the full public CUA-Gym task set after the necessary data review.… See the full description on the dataset page: https://huggingface.co/datasets/xlangai/CUA-Gym.humaneval-xHumanEval-X is a benchmark for the evaluation of the multilingual ability of code generative models. It consists of 820 high-quality human-crafted data samples (each with test cases) in Python, C++, Java, JavaScript, and Go, and can be used for various tasks.Java-Code-Large-text-onlyJava-Code-Large
Java-Code-Large is a large-scale corpus of publicly available Java source code comprising more than 15 million java codes. The dataset is designed to support research in large language model (LLM) pretraining, code intelligence, software engineering automation, and program analysis.
By providing a high-volume, language-specific corpus, Java-Code-Large enables systematic experimentation in Java-focused model training, domain adaptation, and downstream code understanding tasks.… See the full description on the dataset page: https://huggingface.co/datasets/XEUIPR/Java-Code-Large-text-only.x_dataset_8191
Bittensor Subnet 13 X (Twitter) Dataset
Dataset Summary
This dataset is part of the Bittensor Subnet 13 decentralized network, containing preprocessed data from X (formerly Twitter). The data is continuously updated by network miners, providing a real-time stream of tweets for various analytical and machine learning tasks.
For more information about the dataset, please visit the official repository.
Supported Tasks
The versatility of this… See the full description on the dataset page: https://huggingface.co/datasets/StormKing99/x_dataset_8191.LegoFlow-SWE
LegoFlow-SWE · 5,000 verified Harbor SWE tasks and two GLM-5.2 trajectory releases
GitHub · Docs · Blog · HuggingFace · LegoX
LegoFlow-SWE
5,000 verified Harbor SWE tasks mined by LegoFlow Curator, shipped in original and anti-hack prompt versions, plus two GLM-5.2 trajectory releases under OpenHands SDK and OpenCode, totaling 9,767 trajectories.
Release
Count
What it is
tasks/
5,000
Original prompts
tasks-anti-hack/
5,000
Same task IDs and task files, with… See the full description on the dataset page: https://huggingface.co/datasets/Lego-X/LegoFlow-SWE.proof-pile-2-streaming
ArXiv | Models | Data | Code | Blog | Sample Explorer
Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer, Albert Q. Jiang, Jia Deng, Stella Biderman, Sean Welleck
The Proof-Pile-2 is a 55 billion token dataset of mathematical and scientific documents. This dataset was created in order to train the Llemma 7B and Llemma 34B models. It consists of three subsets:
arxiv (29B tokens): the ArXiv subset of RedPajama
open-web-math (15B tokens): The OpenWebMath… See the full description on the dataset page: https://huggingface.co/datasets/xavierdurawa/proof-pile-2-streaming.xlcost-text-to-code XLCoST is a machine learning benchmark dataset that contains fine-grained parallel data in 7 commonly used programming languages (C++, Java, Python, C#, Javascript, PHP, C), and natural language (English).x_dataset_07096
Bittensor Subnet 13 X (Twitter) Dataset
Miner Data Compliance Agreement
In uploading this dataset, I am agreeing to the Macrocosmos Miner Data Compliance Policy.
Dataset Summary
This dataset is part of the Bittensor Subnet 13 decentralized network, containing preprocessed data from X (formerly Twitter). The data is continuously updated by network miners, providing a real-time stream of tweets for various analytical and machine learning… See the full description on the dataset page: https://huggingface.co/datasets/zephyr-1111/x_dataset_07096.sim-posttrain
HUMANUAL Posttraining Data
Posttraining data for user simulation, derived from the train splits of the
HUMANUAL benchmark datasets.
Datasets
HUMANUAL (posttraining)
Config
Rows
Description
news
48,618
News article comment responses
politics
45,429
Political discussion responses
opinion
37,791
Reddit AITA / opinion thread responses
book
34,170
Book review responses
chat
23,141
Casual chat responses
email
6,377
Email reply responses… See the full description on the dataset page: https://huggingface.co/datasets/Xuhui/sim-posttrain.x_dataset_94
Bittensor Subnet 13 X (Twitter) Dataset
Dataset Summary
This dataset is part of the Bittensor Subnet 13 decentralized network, containing preprocessed data from X (formerly Twitter). The data is continuously updated by network miners, providing a real-time stream of tweets for various analytical and machine learning tasks.
For more information about the dataset, please visit the official repository.
Supported Tasks
The versatility of this… See the full description on the dataset page: https://huggingface.co/datasets/coldmind/x_dataset_94.x_dataset_0306116
Bittensor Subnet 13 X (Twitter) Dataset
Miner Data Compliance Agreement
In uploading this dataset, I am agreeing to the Macrocosmos Miner Data Compliance Policy.
Dataset Summary
This dataset is part of the Bittensor Subnet 13 decentralized network, containing preprocessed data from X (formerly Twitter). The data is continuously updated by network miners, providing a real-time stream of tweets for various analytical and machine learning… See the full description on the dataset page: https://huggingface.co/datasets/james-1111/x_dataset_0306116.xiangqi-gen6-platinum-dataset
🏯 Xiangqi-NNUE Master Dataset & Model Weights
Tập dữ liệu tự đấu (Self-play) cờ Tướng chuẩn quốc tế và Trọng số nhị phân mạng nơ-ron NNUE (Efficiently Updatable Neural Network) phục vụ huấn luyện Native Rust Engine xiangqi-rim.
📊 THỐNG KÊ THỜI GIAN THỰC (AUTO-UPDATED METRICS)
[!IMPORTANT]
Bảng thống kê này được tự động cập nhật 100% mỗi khi hệ thống khai thác (Gradio / Colab T4 / Worker Nodes) đẩy dữ liệu mới lên Hub.
Chỉ Số Dữ Liệu
Giá Trị Cập Nhật… See the full description on the dataset page: https://huggingface.co/datasets/hoduyquocbao/xiangqi-gen6-platinum-dataset.X-Coder-SFT-376k
X-Coder: Advancing Competitive Programming with Fully Synthetic Tasks, Solutions, and Tests
Dataset Overview
X-Coder-SFT-376k is a large-scale, fully synthetic dataset for advancing competitive programming.
The dataset comprises 4 subsets with a total of 887,321 synthetic records across 423,883 unique queries.
It is designed for supervised fine-tuning and suitbale for cold start to train code reasoning foundations.
X-Coder-SFT-376k is curated by sota reasoning models.… See the full description on the dataset page: https://huggingface.co/datasets/IIGroup/X-Coder-SFT-376k.x_dataset_53985
Bittensor Subnet 13 X (Twitter) Dataset
Dataset Summary
This dataset is part of the Bittensor Subnet 13 decentralized network, containing preprocessed data from X (formerly Twitter). The data is continuously updated by network miners, providing a real-time stream of tweets for various analytical and machine learning tasks.
For more information about the dataset, please visit the official repository.
Supported Tasks
The versatility of this… See the full description on the dataset page: https://huggingface.co/datasets/icedwind/x_dataset_53985.x_dataset_34576
Bittensor Subnet 13 X (Twitter) Dataset
Dataset Summary
This dataset is part of the Bittensor Subnet 13 decentralized network, containing preprocessed data from X (formerly Twitter). The data is continuously updated by network miners, providing a real-time stream of tweets for various analytical and machine learning tasks.
For more information about the dataset, please visit the official repository.
Supported Tasks
The versatility of this… See the full description on the dataset page: https://huggingface.co/datasets/icedwind/x_dataset_34576.x_dataset_55757
Bittensor Subnet 13 X (Twitter) Dataset
Dataset Summary
This dataset is part of the Bittensor Subnet 13 decentralized network, containing preprocessed data from X (formerly Twitter). The data is continuously updated by network miners, providing a real-time stream of tweets for various analytical and machine learning tasks.
For more information about the dataset, please visit the official repository.
Supported Tasks
The versatility of this… See the full description on the dataset page: https://huggingface.co/datasets/rainbowbridge/x_dataset_55757.