datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
keural-SFT-chatml-ko-v1
Keural SFT ChatML (Korean) v1
한국어 SFT(Supervised Fine-Tuning)용 통합 데이터셋입니다. 공개 한국어 instruction/대화 데이터셋 8종을 수집하여 정제 → 품질 필터링 → 안전성 필터링 → 중복 제거(exact + MinHash near-dup) → ChatML 포맷팅 → 토크나이즈 검증을 거친 결과물입니다.
총 샘플 수: 710,278
총 토큰 수: 약 1.9억 (190,724,939 tokens, keural tokenizer 기준)
포맷: ChatML (<|im_start|>role ... <|im_end|>)
최대 길이: 8,192 tokens (초과 시 truncate)
생성일: 2026-07-10
데이터 구조
각 샤드는 JSONL 형식이며, 레코드 스키마는 다음과 같습니다:
{"text":… See the full description on the dataset page: https://huggingface.co/datasets/mkd-jueon/keural-SFT-chatml-ko-v1.Java_method2test_chatml
Java Method to Test ChatML
This dataset is based on the methods2test dataset from Microsoft. It follows the ChatML template format: [{'role': '', 'content': ''}, {...}].
Originally, methods2test contains only Java methods at different levels of granularity along with their corresponding test cases. The different focal method segmentations are illustrated here:
To simulate a conversation between a Java developer and an AI assistant, I introduce two key parameters:
The prompt… See the full description on the dataset page: https://huggingface.co/datasets/random-long-int/Java_method2test_chatml.fable-5-traces-sft-chatml
Fable-5 Traces SFT ChatML
The Claude Fable 5 agent-trace corpus, converted to standard ChatML messages with OpenAI-format tool calls. Load it and train, no format wrangling.
Glint-Research/Fable-5-traces is the largest open corpus of real Claude Fable 5 agent sessions, but it ships as flattened transcript strings that every trainer has to parse themselves. This dataset is that parsing done once, carefully: 4,665 samples restructured into messages arrays with typed roles… See the full description on the dataset page: https://huggingface.co/datasets/AnkitAI/fable-5-traces-sft-chatml.youtube-comment-insights-chatml
YouTube Comment Insights - ChatML
Overview
This dataset contains instruction-tuning samples for structured YouTube comment analysis.
The dataset is formatted in ChatML conversational format and is intended for supervised fine-tuning (SFT), QLoRA, and instruction tuning of large language models.
Each sample contains:
sentiment
tone
pros
cons
Dataset Statistics
~20k training samples
~2k validation samples
Multilingual YouTube comments
Structured JSON… See the full description on the dataset page: https://huggingface.co/datasets/AnandforU/youtube-comment-insights-chatml.ChatML-distilabel-capybara-dpo-7k-binarizedargilla/distilabel-capybara-dpo-7k-binarized in ChatML format, ready to use in HuggingFace TRL's DPO Trainer.
Python code used for conversion:
from datasets import load_dataset
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Felladrin/Llama-160M-Chat-v1")
dataset = load_dataset("argilla/distilabel-capybara-dpo-7k-binarized", split="train")
def format(columns):
return {
"prompt": tokenizer.apply_chat_template(columns["chosen"][:-1]… See the full description on the dataset page: https://huggingface.co/datasets/Felladrin/ChatML-distilabel-capybara-dpo-7k-binarized.keural-conversation-chatml-ko
keural-conversation-chatml-ko
mkd-chanwoo/keural-conversation-ko 데이터셋을 SFT 학습용 ChatML 포맷으로 전처리한 한국어 일상대화 데이터셋입니다.
9개 주제의 일상 캐주얼 대화를 Gemma-4-26B 모델이 생성한 synthetic 멀티턴 대화로 구성되어 있습니다.
데이터셋 개요
항목
값
총 샘플 수
133,339
총 토큰 수
약 19M
평균 토큰 / 샘플
146.0
중간값 토큰
145
최대 토큰
805
최소 토큰
95
언어
한국어 (100%)
포맷
ChatML
라이선스
CC BY 4.0
포맷
ChatML 형식의 멀티턴 대화입니다.
<|im_start|>user
{발화 1}
<|im_end|>
<|im_start|>assistant
{발화 2}
<|im_end|>
<|im_start|>user
{발화… See the full description on the dataset page: https://huggingface.co/datasets/mkd-chanwoo/keural-conversation-chatml-ko.ChatML-aya_datasetCohereForAI/aya_dataset in ChatML format, ready to use in HuggingFace TRL's SFT Trainer.
Python code used for conversion:
from datasets import load_dataset
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Felladrin/Llama-160M-Chat-v1")
dataset = load_dataset("CohereForAI/aya_dataset", split="train")
def format(columns):
messages = [
{
"role": "user",
"content": columns["inputs"].strip(),
},
{… See the full description on the dataset page: https://huggingface.co/datasets/Felladrin/ChatML-aya_dataset.YiSang-HighQuality-chatml-v1
YiSang-HighQuality ChatML (Korean) v1
KOREAson/YiSang-HighQuality를 한국어 SFT용으로 가공한 데이터셋입니다. 원본 response에 포함된 <think>...</think> 영어 추론 트레이스를 전부 제거하고 실제 답변만 남긴 뒤, 정제 → 품질 필터 → 안전성 필터 → 중복 제거 → ChatML 포맷팅 → 토크나이즈 검증을 거친 결과물입니다.
총 샘플 수: 259,596
총 토큰 수: 약 1.78억 (178,406,073 tokens, keural tokenizer 기준, 평균 687 tokens/sample)
포맷: ChatML (<|im_start|>role ... <|im_end|>)
최대 길이: 8,192 tokens (초과 시 truncate)
추론 트레이스: 최종 산출물 전수 검사 기준 <think>/</think> 잔존 0건
생성일: 2026-07-10
데이터… See the full description on the dataset page: https://huggingface.co/datasets/mkd-jueon/YiSang-HighQuality-chatml-v1.ChatML-SlimOrca-DedupOpen-Orca/SlimOrca-Dedup in ChatML format, ready to use in HuggingFace TRL's SFT Trainer.
Python code used for conversion:
from datasets import load_dataset
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Felladrin/Llama-160M-Chat-v1")
dataset = load_dataset("Open-Orca/SlimOrca-Dedup", split="train")
def format(columns):
messages = []
conversations = columns["conversations"]
for i in range(len(conversations)):
message =… See the full description on the dataset page: https://huggingface.co/datasets/Felladrin/ChatML-SlimOrca-Dedup.ChatML-H4rmony_dponeovalle/H4rmony_dpo in ChatML format, ready to use in HuggingFace TRL's DPO Trainer.
Python code used for conversion:
from datasets import load_dataset
dataset = load_dataset("neovalle/H4rmony_dpo", split="train")
def format(columns):
return {
"prompt": f"<|im_start|>user\n{columns['prompt']}<|im_end|>\n<|im_start|>assistant\n",
"chosen": f"{columns['chosen']}<|im_end|>",
"rejected": f"{columns['rejected']}<|im_end|>",
}… See the full description on the dataset page: https://huggingface.co/datasets/Felladrin/ChatML-H4rmony_dpo.hermes-agent-traces-chatml
Hermes Agent Traces — ChatML Format
A ready-to-train dataset of 18,487 multi-turn tool-calling conversations in ChatML messages format, combining Hermes Agent reasoning traces with NousResearch function-calling data.
Built for SFT training of tool-calling / agentic LLMs with TRL's SFTTrainer.
Quick Start
from datasets import load_dataset
from trl import SFTTrainer
dataset = load_dataset("sroecker/hermes-agent-traces-chatml", split="train")
trainer = SFTTrainer(… See the full description on the dataset page: https://huggingface.co/datasets/sroecker/hermes-agent-traces-chatml.mng-audit-ultimate-chatml-v2
MNG Audit Ultimate ChatML v2 ⚡
🎯 Composition parfaite
Split
Taille
Contenu
general
6,390
40% Multi-turn + 60% Instructions
special
2,496
Données audit/compta spécialisées
train
8,886
80/20 optimal
val
2,000
Validation
✅ Garanties
100% Français (langdetect)
ChatML validé TRL/SFTTrainer
Dédoublonné (textuel exact)
40% Multi-turn conversations naturelles
🚀 Usage direct
from datasets import load_dataset
ds =… See the full description on the dataset page: https://huggingface.co/datasets/MNGaudit/mng-audit-ultimate-chatml-v2.deer_sec-japanese-cybersecurity-chatml-v2
deer_sec-japanese-cybersecurity-chatml-v2.0
📊 Dataset Details
Total Rows: 83,562 件
File Size: 約 492.5 MB
Format: JSONL (ChatML形式)
Language: 日本語 (Japanese)
概要 (Overview)
本データセットは、高度なサイバー防御と脅威インテリジェンスに特化したインストラクション・チューニング用のデータセットです。AlicanKiraz0様によって公開された Cybersecurity-Dataset-Fenrir-v2.0 を元に構築されています。
元の膨大な英語データの中から約 99.5% （83,562件） を抽出し、翻訳特化モデルである translategemma:12b を用いて高品質な日本語へ翻訳しました。その後、LLMのファインチューニング（LoRA等）にそのまま利用できるよう、厳格なデータクレンジングと整形を行っています。
特徴… See the full description on the dataset page: https://huggingface.co/datasets/deer-sec/deer_sec-japanese-cybersecurity-chatml-v2.customer-support-chatml
Customer Support ChatML Dataset
This dataset is a curated and preprocessed version of the
Bitext Customer Support Dataset.
Dataset Description
The dataset has been converted to ChatML format for fine-tuning conversational AI models.
Format
Each example contains:
text: The complete conversation in ChatML format
messages: JSON string of the conversation as a list of messages
instruction: The original user query
response: The original assistant response… See the full description on the dataset page: https://huggingface.co/datasets/Shivam271089/customer-support-chatml.yandexq-qa-chatmlChatML formatted version of its5Q/yandex-q.
ChatML-WebGLM-QATHUDM/webglm-qa in ChatML format.
Python code used for conversion:
from datasets import load_dataset
import pandas
import re
import random
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
pretrained_model_name_or_path="Felladrin/Llama-160M-Chat-v1"
)
dataset = load_dataset("THUDM/webglm-qa", split="train")
def format(columns):
references = "\n".join(
[
f"- {columns['references'][i].strip()}"
for i in… See the full description on the dataset page: https://huggingface.co/datasets/Felladrin/ChatML-WebGLM-QA.ChatML-ultrachat_200kHuggingFaceH4/ultrachat_200k in ChatML format, ready to use in HuggingFace TRL's SFT Trainer.
Python code used for conversion:
from datasets import load_dataset
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Felladrin/Llama-160M-Chat-v1")
dataset = load_dataset("HuggingFaceH4/ultrachat_200k", split="train_sft")
def format(columns):
return { "text": tokenizer.apply_chat_template(columns["messages"], tokenize=False) }… See the full description on the dataset page: https://huggingface.co/datasets/Felladrin/ChatML-ultrachat_200k.logistics-cx-transcript-analysis-chatml
OmniCX Logistics CX Dataset (Research Preview)
Dataset Summary
This dataset is designed for structured extraction of logistics and customer-experience (CX) signals from multi-turn support conversations.
Each record uses ChatML-style messages with:
a fixed system instruction
a user transcript
an assistant JSON payload matching LogisticsCXMetrics
This release is a research preview and should not be treated as a production-certified benchmark.
Project repository:… See the full description on the dataset page: https://huggingface.co/datasets/mangesh-ux/logistics-cx-transcript-analysis-chatml.ChatML-CapybaraLDJnr/Capybara in ChatML format, ready to use in HuggingFace TRL's SFT Trainer.
Python code used for conversion:
from datasets import load_dataset
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Felladrin/Llama-160M-Chat-v1")
dataset = load_dataset("LDJnr/Capybara", split="train")
def format(columns):
messages = []
conversationColumn = columns["conversation"]
for i in range(len(conversationColumn)):
messages.append({
"role":… See the full description on the dataset page: https://huggingface.co/datasets/Felladrin/ChatML-Capybara.slimorca-dedup-chatml-100k
Copy of Open-Orca/SlimOrca-Dedup in ChatML format downsample to 100k
"SlimOrca Dedup" is a deduplicated, unfiltered subset of the SlimOrca dataset, excluding RLHF instances, resulting in 363k unique examples.
Key Features
Removal of RLHF instances.
Deduplication using minhash and Jaccard similarity techniques.
Demo Models
Note: These models were trained on the full SlimOrca dataset, not the deduplicated, unfiltered version.
*… See the full description on the dataset page: https://huggingface.co/datasets/philschmid/slimorca-dedup-chatml-100k.ChatML-OpenOrcaOpen-Orca/OpenOrca in ChatML format, ready to use in HuggingFace TRL's SFT Trainer.
Python code used for conversion:
from datasets import load_dataset
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Felladrin/Minueza-32M-Base")
dataset = load_dataset("Open-Orca/OpenOrca", split="train")
def format(columns):
messages = []
system_prompt = columns["system_prompt"].strip()
if system_prompt:
messages.append({
"role": "system"… See the full description on the dataset page: https://huggingface.co/datasets/Felladrin/ChatML-OpenOrca.SlimOrca-Dedup-trl-conversational-chatmlThis dataset is contains json formatted in TRL's conversational format as well as a chatml formatted text field.
ChatML-hercules-v2.0Locutusque/hercules-v2.0 in ChatML format.
Python code used for conversion:
from datasets import load_dataset
import pandas
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
pretrained_model_name_or_path="Felladrin/Llama-160M-Chat-v1"
)
dataset = load_dataset("Locutusque/hercules-v2.0", split="train")
def format(columns):
messages = []
conversation = columns["conversations"]
for i in range(len(conversation)):
message =… See the full description on the dataset page: https://huggingface.co/datasets/Felladrin/ChatML-hercules-v2.0.ChatML-HelpSteernvidia/HelpSteer in ChatML format, ready to use in HuggingFace TRL's SFT Trainer.
Python code used for conversion:
from datasets import load_dataset
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Felladrin/Llama-160M-Chat-v1")
dataset = load_dataset("nvidia/HelpSteer", split="train")
def format(columns):
prompt = columns["prompt"].strip()
response = columns["response"].strip()
messages = [
{
"role": "user"… See the full description on the dataset page: https://huggingface.co/datasets/Felladrin/ChatML-HelpSteer.ChatML-open-instructVMware/open-instruct in ChatML format, ready to use in HuggingFace TRL's SFT Trainer.
Python code used for conversion:
from datasets import load_dataset
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Felladrin/Llama-160M-Chat-v1")
dataset = load_dataset("VMware/open-instruct", split="train")
def format(columns):
messages = [
{
"role": "user",
"content": columns["instruction"].strip(),
},
{… See the full description on the dataset page: https://huggingface.co/datasets/Felladrin/ChatML-open-instruct.ChatML-databricks-dolly-15kdatabricks/databricks-dolly-15k in ChatML format.
Python code used for conversion:
from datasets import load_dataset
import pandas
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
pretrained_model_name_or_path="Felladrin/Llama-160M-Chat-v1"
)
dataset = load_dataset("databricks/databricks-dolly-15k", split="train")
def format(columns):
instruction = columns["instruction"].strip()
context = columns["context"].strip()
response =… See the full description on the dataset page: https://huggingface.co/datasets/Felladrin/ChatML-databricks-dolly-15k.ChatML-reddit-instruct-curatedeuclaise/reddit-instruct-curated in ChatML format, ready to use in HuggingFace TRL's SFT Trainer.
Python code used for conversion:
from datasets import load_dataset
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Felladrin/Llama-160M-Chat-v1")
dataset = load_dataset("euclaise/reddit-instruct-curated", split="train")
def format(columns):
post_title = columns["post_title"].strip()
post_text = columns["post_text"].strip()
comment_text =… See the full description on the dataset page: https://huggingface.co/datasets/Felladrin/ChatML-reddit-instruct-curated.TAA-ChatML-10K
TAA-ChatML-10K
A dataset of 10,438 question-answer pairs for Cyber Threat Intelligence (CTI) and Advanced Persistent Threat (APT) attribution tasks. Synthesized from 1,468 publicly available threat intelligence reports covering APT attribution, malware analysis, and threat actor TTPs. The dataset is formatted in ChatML conversation structure for fine-tuning large language models.
License
MIT
apex-food-rd-chatml-v2-expanded
Apex Food R&D ChatML v2 — Expanded Indian Functional Ingredient Dataset
This is the expanded v2 dataset for building a food formulation R&D assistant for Apex Nutrition.
Why v2 exists
The first MVP dataset used a narrow seed list of ~20 ingredients. That was too limited for Apex Nutrition's intended product space. This v2 dataset expands the ingredient universe to 137 India-relevant functional/natural/organic ingredients, including millets, pulses, seeds, spices, herbs… See the full description on the dataset page: https://huggingface.co/datasets/harshal3099/apex-food-rd-chatml-v2-expanded.personal-finance-chatml-dataset
Bilingual Personal Finance ChatML Dataset (EN/ES)
Dataset Description
This dataset is a professionally curated bilingual (English/Spanish) instruction dataset designed for fine-tuning large language models (LLMs) in the domain of personal finance.
It is structured in ChatML format and intended for supervised fine-tuning (SFT), domain adaptation, and financial instruction modeling.
The dataset is created and reviewed from an accounting perspective, ensuring conceptual… See the full description on the dataset page: https://huggingface.co/datasets/williamjmorenor/personal-finance-chatml-dataset.