datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
ultrafeedback-binarized-preferences-cleaned-kto
UltraFeedback - Binarized using the Average of Preference Ratings (Cleaned) KTO
A KTO signal transformed version of the highly loved UltraFeedback Binarized Preferences Cleaned, the preferred dataset by Argilla to use from now on when fine-tuning on UltraFeedback
This dataset represents a new iteration on top of argilla/ultrafeedback-binarized-preferences,
and is the recommended and preferred dataset by Argilla to use from now on when fine-tuning on UltraFeedback.
Read more about… See the full description on the dataset page: https://huggingface.co/datasets/argilla/ultrafeedback-binarized-preferences-cleaned-kto.KnessetCorpus
The Knesset (Israeli Parliament) Proceedings Corpus
💻 [Github Repo] •
📃 [Paper] •
📊 [ES kibana dashboard]
Dataset Description
An annotated corpus of Hebrew parliamentary proceedings containing over 35 million sentences from all the (plenary and committee) protocols held in the Israeli parliament
from 1992 to 2024.Sentences are annotated with various levels of linguistic information, including part-of-speech tags, morphological features, dependency… See the full description on the dataset page: https://huggingface.co/datasets/HaifaCLGroup/KnessetCorpus.KoPI-CC
Dataset Summary
KoPI-CC (Korpus Perayapan Indonesia)-CC is Indonesian only extract from Common Crawl snapshots using ungoliant, each snapshot also filtered using some some deduplicate technique such as exact hash(md5) dedup technique and minhash LSH neardup
Preprocessing
Each folder name inside snapshots folder denoted preprocessing technique that has been applied .
Raw
this processed directly from cc snapshot using ungoliant without any addition filter ,you can read it… See the full description on the dataset page: https://huggingface.co/datasets/acul3/KoPI-CC.ClimbLab-JaJapanese / 日本語版
ClimbLab-Ja
ClimbLab-Ja is a high-quality 300-billion-token Japanese corpus with 20 clusters. It is a Japanese adaptation of the nvidia/Nemotron-ClimbLab approach. Based on LLM-jp Corpus v4, we semantically reorganized and filtered the dataset into 20 distinct clusters, resulting in a high-quality 300-billion-token corpus. Specifically, we first grouped the data into 1,000 groups based on topic information. Then we assigned six scores from 0 to 5 to each group… See the full description on the dataset page: https://huggingface.co/datasets/KantaHayashiAI/ClimbLab-Ja.damru-knowledge
🐕 Damru Knowledge
A continuously growing, self-collected question-answer knowledge base that powers Damru AI — a self-learning assistant built for exam preparation and general-purpose help, with a focus on Indian students.
The dataset is harvested and quality-filtered automatically, 24x7, from multiple open sources and a self-evaluating reasoning engine. New rows are appended every hour as parquet shards under data/.
📦 What's inside
Column
Type
Description… See the full description on the dataset page: https://huggingface.co/datasets/Damaru-ai/damru-knowledge.dialogsum
Dataset Card for DIALOGSum Corpus
Dataset Description
Links
Homepage: https://aclanthology.org/2021.findings-acl.449
Repository: https://github.com/cylnlp/dialogsum
Paper: https://aclanthology.org/2021.findings-acl.449
Point of Contact: https://huggingface.co/knkarthick
Dataset Summary
DialogSum is a large-scale dialogue summarization dataset, consisting of 13,460 (Plus 100 holdout data for topic generation) dialogues with corresponding… See the full description on the dataset page: https://huggingface.co/datasets/knkarthick/dialogsum.Nemotron-Personas-Korea
Nemotron-Personas-Korea
우리나라 실제 분포에 기반한 합성 페르소나를 위한 복합 AI 시스템
A compound AI approach to personas grounded in real-world distributions
데이터셋 개요 (Overview)
Nemotron-Personas-Korea는 대한민국의 실제 인구통계학적·지리적·성격 특성 분포를 기반으로 합성된 오픈소스 페르소나 데이터셋(CC BY 4.0)으로, 우리나라 인구의 다양성과 특성을 폭넓게 반영하도록 설계되었습니다. 이는 최초의 대규모 우리말 페르소나 데이터셋이며, 이름, 성별, 나이, 혼인 상태, 교육 수준, 직업, 거주 지역 등의 속성을 실제 대한민국 국가데이터처 국가통계포털(KOSIS), 대법원, 국민건강보험공단, 농촌경제연구원, NAVER Cloud 통계 자료를 기반으로 합성하였습니다.… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Nemotron-Personas-Korea.UltraX-Preview
UltraX: Refining Pre-Training Data at Scale with Adaptive Programmatic Editing
📜 Paper |
💻 Code |
🤖 Models |
📦 UltraData Collection
English |
中文
📚 Introduction
UltraX is a function-calling refinement framework for large-scale pre-training data that adaptively generates and executes editing functions for efficient instance-wise refinement. Unlike rule-based or end-to-end LLM rewriting methods, UltraX trains a lightweight… See the full description on the dataset page: https://huggingface.co/datasets/Kangarooz/UltraX-Preview.kimi-k3-distillation
kimi-k3-distillation
Single-teacher slice of
r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation,
filtered to teacher_model == "kimi-code/k3" only. The Qwen3.8-Max-Preview and
GLM-5.2 traces are removed.
4,347 rows — 3,918 train / 212 validation / 217 test.
from datasets import load_dataset
ds = load_dataset("beyoru/kimi-k3-distillation") # sft: messages + tools
ds = load_dataset("beyoru/kimi-k3-distillation", "canonical") # + full audit columns… See the full description on the dataset page: https://huggingface.co/datasets/beyoru/kimi-k3-distillation.qwen3.8-max-glm5.2-kimi-k3-distillation
Multi-Teacher Distillation Dataset (57,937 traces)
A quality-filtered, deduplicated, multi-teacher SFT corpus combining traces from three frontier models across math, code, reasoning, instruction-following, tool-use, science, long-context, multilingual, and creative dialogue domains.
Teachers
Teacher
Provider
Traces
Qwen3.8-Max-Preview
Alibaba Cloud Model Studio
48,283
GLM-5.2
Z.AI Coding Plan
5,307
Kimi Code K3
Moonshot AI (Kimi)
4,347… See the full description on the dataset page: https://huggingface.co/datasets/r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation.leetcode-problem-solutions
LeetCode Solution Dataset
This dataset contains community-contributed LeetCode solutions scraped from public discussions and solution pages, enriched with metadata such as vote counts, author info, tags, and full code content. The goal is to make high-quality, peer-reviewed coding solutions programmatically accessible for research, analysis, educational use, or developer tooling.
Column Descriptions
Column Name
Type
Description
question_slug
string
The unique… See the full description on the dataset page: https://huggingface.co/datasets/kaysss/leetcode-problem-solutions.kernelbench-hard-traces
KernelBench-Hard agent traces
Frontier coding agents writing optimized CUDA/Triton kernels (FP8 GEMM, paged
attention, MoE, W4A16, KDA, Top-k) on RTX PRO 6000 Blackwell, H100 PCIe, and
B200; roofline-graded.
Each .jsonl file is one agent run in Claude-Code session format, viewable with
the Hugging Face Agent Trace viewer (Data Studio → open a row). Filename =
run id.
Live leaderboard: https://kernelbench.com/hard
Secrets redacted. Full reasoning for open-provider routes… See the full description on the dataset page: https://huggingface.co/datasets/Infatoshi/kernelbench-hard-traces.kilt_tasks
Dataset Card for KILT
Dataset Summary
KILT has been built from 11 datasets representing 5 types of tasks:
Fact-checking
Entity linking
Slot filling
Open domain QA
Dialog generation
All these datasets have been grounded in a single pre-processed Wikipedia dump, allowing for fairer and more consistent evaluation as well as enabling new task setups such as multitask and transfer learning with minimal effort. KILT also provides tools to analyze and understand the… See the full description on the dataset page: https://huggingface.co/datasets/facebook/kilt_tasks.open-license-corpus
PubText
Welcome to the Open License Corpus (OLC), a 228B token corpus for training permissively-licensed language models.
Disclaimer: OLC should not be considered a universally safe-to-use dataset. We encourage users of OLC to consult a legal professional on the suitability of each data source for their application.
Dataset Summary
Domain
Sources
Specific License
# BPE Tokens (in billions; GPT-NeoX tokenizer)
Legal
Case Law, Pile of Law (PD subset)
Public… See the full description on the dataset page: https://huggingface.co/datasets/kernelmachine/open-license-corpus.knesset-committees
About
This dataset is derived from raw a/v recordings and human-generated protocols of the Knesset (the Israeli house of representatives) committee sessions as part of the ivrit.ai project.
Consider visiting the preview space for this dataset here
Method
Data dumps from the Knesset contain A/V recordings of committee sessions, alongside human-generated protocols.
We extract the audio stream, abd produce weakly time stamped segmentation of the protocol text (we… See the full description on the dataset page: https://huggingface.co/datasets/ivrit-ai/knesset-committees.klue
Dataset Card for KLUE
Dataset Summary
KLUE is a collection of 8 tasks to evaluate natural language understanding capability of Korean language models. We delibrately select the 8 tasks, which are Topic Classification, Semantic Textual Similarity, Natural Language Inference, Named Entity Recognition, Relation Extraction, Dependency Parsing, Machine Reading Comprehension, and Dialogue State Tracking.
Supported Tasks and Leaderboards
Topic Classification, Semantic… See the full description on the dataset page: https://huggingface.co/datasets/klue/klue.SEC-EDGARDatamule, Teraflop AI, and Eventual collaborated to release the SEC-EDGAR dataset.
The dataset contains 590 gbs of data, spanning 8 million samples and 43 billion tokens from all major filings in the SEC EDGAR database.
The bulk data was collected using datamule-python library and the official datamule api created by John Friedman. The datamule Python library is a package for collecting, manipulating, and processing the SEC Edgar data at scale. Datamule provides a simple open-source api… See the full description on the dataset page: https://huggingface.co/datasets/kapilrao/SEC-EDGAR.physics-corpus
Physics Corpus — konsman/physics-corpus
arXiv physics papers exported from a PostgreSQL mirror of the Kaggle arXiv dataset,
structured for ontology extraction and downstream NLP pipelines.
Configuration: quantum-physics
arXiv categories included: quant-ph, hep-th, gr-qc
Schema
Field
Type
Description
paper_id
string
arxiv:<id>v<n> — stable across pipeline runs
arxiv_id
string
Base arXiv ID without version suffix
arxiv_version
int32
Version number… See the full description on the dataset page: https://huggingface.co/datasets/konsman/physics-corpus.IFEval-Ko
IFEval-Ko: Korean Instruction-Following Benchmark for LLMs
This dataset is originated from IFEval Dataset
Korean Version README
IFEval-Ko is a Korean adaptation of Google's open-source IFEval benchmark utilized with lm-evaluation-harness framework.
It enables evaluation of large language models (LLMs) for their instruction-following capabilities in the Korean language.
Dataset Details
Original Source: google/IFEvalAdaptation Author: Allganize Inc. LLM TEAM |… See the full description on the dataset page: https://huggingface.co/datasets/allganize/IFEval-Ko.urls
URLs
74,918,894,107 deduplicated, validated URLs, sorted by
SURT key
and split into 2,334 range shards.
As plain text the URLs are 5.8 TiB, averaging 84 characters each. Sorted by
SURT key and delta-encoded they fit in 665.6 GiB — 9.54 bytes per URL, a
8.85× reduction. That is the whole point of the ordering: SURT puts URLs
from the same site next to each other, DELTA_LENGTH_BYTE_ARRAY then stores only
where each row differs from the one above it, and zstd compresses what is… See the full description on the dataset page: https://huggingface.co/datasets/ks46/urls.ledger-long-context-KPI-QA
LEDGER — Long-Context KPI Question Answering & Page Retrieval
This dataset is part of the LEDGER (Long-context Evaluation of Documents for
Grounded Extraction and Retrieval) benchmark.
It supports two of the three LEDGER tasks:
Page-level KPI retrieval — given a natural-language question about a financial
KPI and the corresponding annual report, retrieve the relevant page(s). Each row
includes TREC-style graded relevance judgments (qrels) over all candidate pages.… See the full description on the dataset page: https://huggingface.co/datasets/artefactory/ledger-long-context-KPI-QA.Llama-2-7b-KronQ-HG
Llama-2-7b — KronQ H_G (output-side gradient covariance)
Paper: arXiv:2607.07964 · Code: GitHub
Pre-computed H_G for Llama-2-7b, the output-side curvature factor used by KronQ under the K-FAC factorization H ≈ H_X ⊗ H_G. H_G is the per-sublayer sampled-Fisher gradient covariance (labels drawn from the model distribution) (E[g gᵀ] over the layer output), distinct from the standard input-side Hessian H_X (which GPTQ/GPTAQ build online during calibration).
Publishing this lets you… See the full description on the dataset page: https://huggingface.co/datasets/donghyunli/Llama-2-7b-KronQ-HG.pi-mono
Coding agent session traces for badlogicgames/pi-mono
This dataset contains redacted coding agent session traces collected while working on https://github.com/badlogic/pi-mono.git. The traces were exported with pi-share-hf from a local pi workspace and filtered to keep only sessions that passed deterministic redaction and LLM review.
Data description
Each *.jsonl file is a redacted pi session. Sessions are stored as JSON Lines files where each line is a structured… See the full description on the dataset page: https://huggingface.co/datasets/karkowww/pi-mono.KoHRM-Text-1.4B-sft-lora-data
KoHRM-Text-1.4B SFT and LoRA Prepared Data
This dataset repo stores curated KoHRM SFT/LoRA subsets in the same tokenized
HRM-Text V1Dataset format used by training. It is intended for quick behavior
alignment experiments after KoHRM pretraining.
Model repo:
https://huggingface.co/LLM-OS-Models/KoHRM-Text-1.4B
Code repo:
https://github.com/LLM-OS-Models/KoHRM-text
Format
Each folder is a prepared V1Dataset:
<dataset-name>/
metadata.json
tokenizer_info.json… See the full description on the dataset page: https://huggingface.co/datasets/LLM-OS-Models/KoHRM-Text-1.4B-sft-lora-data.K12-KGraph
K12-KGraph
This repository contains the dataset release for the paper "K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs".
Paper | Project page | Code
Overview
K12-KGraph is a curriculum-aligned knowledge graph built from official People's Education Press (PEP) K-12 textbooks. It focuses on curriculum cognition, namely the structured understanding of how school knowledge is organized, connected, and sequenced.
The… See the full description on the dataset page: https://huggingface.co/datasets/lhpku20010120/K12-KGraph.kimi-cyber-reasoning
Kimi Cyber Reasoning
997 chain-of-thought records covering 13 cybersecurity disciplines and 4 systems engineering domains, distilled from the Kimi K3 reasoning model via API. Every record provides an explicit step-by-step <think> reasoning trace followed by a technical resolution, unified code diff fix, or structured tool invocation.
The dataset was curated as an anchor set for training, healing, and specializing compact reasoning models on systems security and tool calling… See the full description on the dataset page: https://huggingface.co/datasets/echel0nn1881/kimi-cyber-reasoning.PCMind-2.1-Kaiyuan-2B
This repository contains the complete pretraining dataset for
PCMind-v2.1-Kaiyuan-2B, a leading fully open-source language model.
Overview
The dataset is organized into 5 training phases, with all phase datasets open-sourced in this repository. Our training methodology employs domain-specific mixing strategies across five primary domains:
English: General English text
Chinese: General Chinese text
Code: Programming and code-related content
Math: Mathematical reasoning and… See the full description on the dataset page: https://huggingface.co/datasets/thu-pacman/PCMind-2.1-Kaiyuan-2B.DISL
DISL
The DISL dataset features a collection of 514506 unique Solidity files that have been deployed to Ethereum mainnet. It caters to the need for a large and diverse dataset of real-world smart contracts. DISL serves as a resource for developing machine learning systems and for benchmarking software engineering tools designed for smart contracts.
Content
the raw subset has full contracts source code and it's not deduplicated, it has 3,298,271 smart contracts
the… See the full description on the dataset page: https://huggingface.co/datasets/ASSERT-KTH/DISL.Athar-Shamela4
Shamela 4 — Full Islamic Library Corpus
A complete extraction of al-Maktaba al-Shamela (الشاملة) v4, containing 8,589 books across 40 categories of classical Islamic sciences. Extracted from the original Lucene + Sqlite Shamela DB on 2026-04-26 with ~7.6 million pages and ~19 GB of Arabic text.
Dataset Structure
stage0_raw/
├── _meta/ # Cross-cutting metadata (Parquet + JSONL)
│ ├── extraction_manifest.json # Global extraction record… See the full description on the dataset page: https://huggingface.co/datasets/Kandil7/Athar-Shamela4.kanitakorn-th-sft
Kanitakorn — Thai-focused SFT corpus + tools (beats Typhoon-S-8B on ThaiExam / MATH / HotpotQA)
A Thai-language SFT dataset (4,147 records → 23,715 with Round 2 augmentation) and the training/eval
toolchain we used to fine-tune Qwen3-8B and Qwen3-4B-Instruct-2507 into Thai-benchmark-targeted models
that beat Typhoon-S-8B on multiple benchmarks.
Released models
8B variant: https://huggingface.co/Jnx03/kanitakorn-qwen3-8b-sft-v1
4B small-device variant:… See the full description on the dataset page: https://huggingface.co/datasets/Jnx03/kanitakorn-th-sft.