datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
lmsys-chat-1m-synth
LMSYS-Chat-1M-Synth: Japanese/English Synthetic Conversation Dataset Derived from LMSYS-Chat-1M
This repository contains a series of Japanese and English conversation datasets derived from LMSYS-Chat-1M.
Llama-3.1-LMSYS-Chat-1M-Synth
Utilized in the post-training of Llama-3.1-Swallow-8B-Instruct-v0.1 and Llama-3.1-Swallow-70B-Instruct-v0.1
Gemma-2-LMSYS-Chat-1M-Synth
Utilized in the post-training of Llama-3.1-Swallow-8B-Instruct-v0.3 and Llama-3.1-Swallow-70B-Instruct-v0.3… See the full description on the dataset page: https://huggingface.co/datasets/tokyotech-llm/lmsys-chat-1m-synth.lmsys_chat_1m_clean_R1
oumi-ai/lmsys_chat_1m_clean_R1
lmsys_chat_1m_clean_R1 is a text dataset designed to train Conversational Language Models with DeepSeek-R1 level reasoning.
Prompts were pulled from LMSYS and filtered to lmsys_chat_1m_clean, and responses were taken from DeepSeek-R1 without additional filters present.
We release lmsys_chat_1m_clean_R1 to help enable the community to develop the best fully open reasoning model!
lmsys_chat_1m_clean queries with responses generated from… See the full description on the dataset page: https://huggingface.co/datasets/oumi-ai/lmsys_chat_1m_clean_R1.DeepSeek-R1-Distill-Llama-8B-lmsys-openthoughts-tokenizedThis dataset is used for training Sparse Autoencoders (SAEs) to identify reasoning features in Large Language Models (LLMs), as described in the paper I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders.
Code for the paper is available at: https://github.com/AIRI-Institute/SAE-Reasoning
The dataset consists of tokenized text data used for training the SAEs.
dataset_info:
features:
name: tokens
sequence: int64
splits:
name:… See the full description on the dataset page: https://huggingface.co/datasets/andreuka18/DeepSeek-R1-Distill-Llama-8B-lmsys-openthoughts-tokenized.lmsys-arena-human-preference-winner-43k-unfiltered
lmsys-arena-human-preference-winner-43k-unfiltered
This repository contains a dataset derived from the lmsys/lmsys-arena-human-preference-55k dataset, which is licensed under the Apache 2.0 License.
Dataset Description
The lmsys-arena-human-preference-winner-43k-unfiltered dataset is a collection of 43,000 samples, each containing an instruction (prompt) and an output (winning response) from real-world user and LLM conversations. The dataset is derived from the original… See the full description on the dataset page: https://huggingface.co/datasets/lesserfield/lmsys-arena-human-preference-winner-43k-unfiltered.ytz20-LMSYS-Chat-GPT-5-Chat-Response
ytz20/LMSYS-Chat-GPT-5-Chat-Response Dataset
This is a reformatted, unofficial version of ytz20/LMSYS-Chat-GPT-5-Chat-Response
According to the original authors:
This dataset is an extension of the LMSYS-Chat-1M-Clean corpus, specifically curated by collecting high-quality, non-refusal responses from the GPT-5-Chat API.
Modifications in this version:
The "content" and "teacher_response" columns have been processed into the "input" and "output" columns in this dataset.
An… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/ytz20-LMSYS-Chat-GPT-5-Chat-Response.lmsys-chat-1m-filtered
Dataset Card for natong19/lmsys-chat-1m-filtered
Filtered version of lmsys/lmsys-chat-1m, a collection of one million real-world conversations with various LLMs.
Data cleaning process inspired by OpenLeecher/lmsys_chat_1m_clean.
Overview of filtering process:
1. Filtering REDACTED Entries
Entries that were labeled as REDACTED due to containing Personally Identifiable Information (PII) were removed.
1000000 samples -> 733740 samples
2. Format validation… See the full description on the dataset page: https://huggingface.co/datasets/natong19/lmsys-chat-1m-filtered.lmsys_lmsys-chat-1m-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT
lmsys_lmsys-chat-1m-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT
PJMixers-Dev/musab-mk_lmsys-chat-1m_deduped-prompts with responses generated with gemini-2.0-flash-thinking-exp-1219.
Generation Details
If BlockedPromptException, StopCandidateException, or InvalidArgument was returned, the sample was skipped.
If ["candidates"][0]["safety_ratings"] == "SAFETY" the sample was skipped.
If ["candidates"][0]["finish_reason"] != 1 the sample was skipped.
model =… See the full description on the dataset page: https://huggingface.co/datasets/PJMixers-Dev/lmsys_lmsys-chat-1m-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT.lmsys-chat-1m-animal-welfare-coarse-filter
LMSYS Chat 1M Animal Welfare Coarse Filter
Public working dataset for Compassion in Machine Learning.
Source dataset: lmsys/lmsys-chat-1m
Filter package: flpc
Filter used: original coarse animal-welfare keyword list provided by the project team.
Counts:
total conversations scanned: 1,000,000
matched conversations: 16,527
match rate: 1.6527%
Files:
matches.parquet: one row per matched conversation, preserving all original source fields/columns plus _matched_terms… See the full description on the dataset page: https://huggingface.co/datasets/CompassioninMachineLearning/lmsys-chat-1m-animal-welfare-coarse-filter.gemma-3-12b-it-lmsys-onpolicy-rollouts
On-policy chat rollouts: google/gemma-3-12b-it on LMSYS-Chat-1M prompts
Each row is a first-user-turn prompt sampled from
lmsys/lmsys-chat-1m and a
response generated on-policy by google/gemma-3-12b-it with vLLM (do_sample,
temperature 0.7, top_p 1.0, max_new_tokens 768, seed 42). 24,991 rows. Built to match
GemmaScope 2's instruction-tuned SAE training distribution (real model rollouts) for a
short KL+MSE ("end-to-end") finetune of the released GemmaScope-2 residual SAE.… See the full description on the dataset page: https://huggingface.co/datasets/iarcuschin/gemma-3-12b-it-lmsys-onpolicy-rollouts.