datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
AIME_MATH_1000_LONG_COTlongcot
LongCoT
LongCoT is a benchmark for long-horizon reasoning across logic, computer science, chemistry, chess, and mathematics. This Hugging Face release contains the benchmark data in viewer-friendly Parquet format for browsing and loading with datasets.
The canonical codebase, verifier, and evaluation harness live at:
https://github.com/LongHorizonReasoning/longcot
Overview
LongCoT measures whether models can sustain coherent reasoning across long chains of thought. The… See the full description on the dataset page: https://huggingface.co/datasets/LongHorizonReasoning/longcot.LONGCOT-merged-1Mthis is PowerInfer/QWQ-LONGCOT-500K + PowerInfer/LONGCOT-Refine-500K shuffled together with the following changes:
pointless starting phrases at the beginning (in english) such as "Okay, ..." "So, ..." etc are removed
config en has been filtered to include only rows detected as lang en in both prompt and response columns via fasttext
QwQ-LongCoT-130KAlso have a look on the second version here => QwQ-LongCoT-2
Figure 1: Just a cute picture generate with [Flux](https://huggingface.co/Shakker-Labs/FLUX.1-dev-LoRA-Logo-Design)
Today, I’m excited to release QwQ-LongCoT-130K, a SFT dataset designed for training O1-like large language models (LLMs). This dataset includes about 130k instances, each with responses generated using QwQ-32B-Preview. The dataset is available under the Apache 2.0 license, so feel free to use it as you like.… See the full description on the dataset page: https://huggingface.co/datasets/amphora/QwQ-LongCoT-130K.AIME_MATH_100_LONG_COTQwQ-LongCoT-Verified-130KOriginal Dataset: amphora/QwQ-LongCoT-130K
QwQ 32B Preview isn't perfect :)
Note: Around 5-7% of the processed data might be incorrectly labeled as "unverified" because QwQ's output isn't exactly the same as the original solution from NuminaMathCoT. I believe this can be solved with another round of processing with a smarter model but Qwen 2.5 3B Instruct is good enough to check if the solution is exactly the same. Magpie data is also "unverified" and has an empty "solution" column.… See the full description on the dataset page: https://huggingface.co/datasets/qingy2024/QwQ-LongCoT-Verified-130K.final_output_pqa_CoT_w_long_answer_fixedMATH_training_split_long_cot
Small Model Learnability Gap
Arxiv Technical Report: https://arxiv.org/pdf/2502.12143v1
Abstract (Click Here)
Large language models (LLMs) excel in complex reasoning tasks, and distilling their reasoning capabilities into smaller models has shown promise. However, we uncover an interesting phenomenon, which we term the Small Model Learnability Gap: small models (≤3B parameters) do not consistently benefit from long chain-of-thought (CoT) reasoning or distillation from larger… See the full description on the dataset page: https://huggingface.co/datasets/UWNSL/MATH_training_split_long_cot.Demeter-LongCoT-400K
Demeter-LongCoT-400K
Demeter-LongCoT-400K is a high-quality, compact chain-of-thought reasoning dataset curated for tasks in mathematics, science, and coding. While the dataset spans diverse domains, it is primarily driven by mathematical reasoning, reflecting a major share of math-focused prompts and long-form logical solutions.
Quick Start with Hugging Face Datasets🤗
pip install -U datasets
from datasets import load_dataset
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/prithivMLmods/Demeter-LongCoT-400K.QwQ-LongCoT-130K-cleanedCleaned version of amphora/QwQ-LongCoT-130K with Chinese language entries removed
long-cot-4kOriginal dataset taken from here: amphora/QwQ-LongCoT-130K
All responses containing Chinese characters have been removed (There's a QwQ quirk where it switches to Chiense)
ShareGPT "conversations" added with the system prompt I used to train gghfez/Mistral-Large-2407-LongCoT
Only "conversations" 1024-4096 tokens long were retained (counted using the Mistral-Large-2407 tokenizer)
If you're using a more efficient tokenizer like Qwen or Llama3, the sequence length will be shorter.
If you're… See the full description on the dataset page: https://huggingface.co/datasets/gghfez/long-cot-4k.QwQ-LongCoT-130K-2A second version of the QwQ-LongCoT dataset.This time, it's only from using the magpie I introduced earlier.
QwQ-LongCoT-500K-CleanedQwQ-LongCoT-130K-decontaminatedDecontaminated version of gghfez/QwQ-LongCoT-130K-cleaned that remove the collided data from math/test, gsm8k/test, olympiadbench/test, minerva_math/test, college_math/test, mmlu_stem/test, gaokao, amc23, aime24 and math500.
Total number of rows: 124594
ScaleQuest-Math-Qwen2.5-LongCoTWe introduce ScaleQuest, a scalable and novel data synthesis method that utilizes small-size open-source models to generate questions from scratch.
Paper: Unleashing Reasoning Capability of LLMs via Scalable Question Synthesis from Scratch
QwQ-Long-CoT-30k-subset-Llama3.1-8B-dynamic-perturbation-regex-generation-max-marginQwQ-LongCoT-decontaminated-filteredDecontaminated version of gghfez/QwQ-LongCoT-130K-cleaned that remove the collided data from math/test, gsm8k/test, olympiadbench/test, minerva_math/test, college_math/test, mmlu_stem/test, gaokao, amc23, aime24 and math500.
With source=='NuminaMath'.
Total number of rows: 88083
AIME_MATH_10_LONG_COTLongCOT-ReasonLONGCOT-Refine-25K25K subset of LONGCOT-Refine-500K to test with. All samples (prompt + response) are 6k words or less (so context length should be < 8192 tokens when training)
long_cot_qwq_math_promptQwQ-LongCoT-59k-cleanedlong_cot_calibrationQwQ-Long-CoT-10k-subset-Llama3.1-8B-single-position-regex-perturbationsycophancy_separate_long_cotlong_cot_step_level_rewardQwQ-Long-CoT-10k-subset-Llama3.1-8B-Instruct-on-policy-alignment-pertubation-generation-fullQwQ-Long-CoT-10k-subset-Llama3.1-8B-dynamic-perturbation-regex-generation-max-margin-10QwQ-Long-CoT-10k-subset-Llama3.1-8B-Instruct-on-policy-step-wise-correct-trajectoryQwQ-Long-CoT-10k-subset-llama3.1-8b-Inst-GPT4-Step-Perturbation-8-rejects