datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Magpie-Qwen2.5-Pro-1M-v0.1
Project Web: https://magpie-align.github.io/
Arxiv Technical Report: https://arxiv.org/abs/2406.08464
Codes: https://github.com/magpie-align/magpie
Abstract
Click Here
High-quality instruction data is critical for aligning large language models (LLMs). Although some models, such as Llama-3-Instruct, have open weights, their alignment data remain private, which hinders the democratization of AI. High human labor costs and a limited, predefined scope for prompting prevent… See the full description on the dataset page: https://huggingface.co/datasets/Magpie-Align/Magpie-Qwen2.5-Pro-1M-v0.1.fineweb-edu-2013-qwen2-7b
FineWeb-Edu 2013 with Qwen2-7B token counts
Every 2013 FineWeb-Edu document, prepared for continued pretraining, with token
counts computed by a pinned Qwen2-7B tokenizer.
The pipeline is year-agnostic: the year, source revision, tokenizer contract,
and selection rule all come from a config file. 2013 uses
processing_config.json. The 2017 companion dataset, which is large enough to
require shuffling and a token budget rather than retaining everything, is at… See the full description on the dataset page: https://huggingface.co/datasets/stevenyuan666/fineweb-edu-2013-qwen2-7b.fineweb-edu-2017-qwen2-7b
FineWeb-Edu 2017 (~100B-token subset) with Qwen2-7B token counts
A ~100B-token subset of FineWeb-Edu 2017, prepared for continued pretraining,
with token counts computed by a pinned Qwen2-7B tokenizer.
This dataset is a selected subset, not the complete 2017 crawl year. 2017
contains about 168B Qwen2-7B tokens, above the 100B target, so it was shuffled
and subsetted: data/train/ holds 101,840,059 documents and 100,000,020,347
tokens, which is 59.29% of the 171,755,787 documents… See the full description on the dataset page: https://huggingface.co/datasets/stevenyuan666/fineweb-edu-2017-qwen2-7b.pubmedqa-recursive-llm-degradation-qwen2.5-0.5b
PubMedQA Recursive LLM Degradation — Qwen2.5-3B
This repository contains synthetic biomedical question-answering data
and model predictions generated as part of a study of recursive
fine-tuning and model degradation.
Base Model
Qwen/Qwen2.5-3B
Source Dataset
The experiments use the PubMedQA dataset:
qiaoxin/PubMedQA
This repository contains generated/derived research artifacts and does
not redistribute the original PubMedQA dataset in its entirety.… See the full description on the dataset page: https://huggingface.co/datasets/chrislimbe/pubmedqa-recursive-llm-degradation-qwen2.5-0.5b.pubmedqa-recursive-llm-degradation-qwen2.5-3b
PubMedQA Recursive LLM Degradation — Qwen2.5-3B
This repository contains synthetic biomedical question-answering data
and model predictions generated as part of a study of recursive
fine-tuning and model degradation.
Base Model
Qwen/Qwen2.5-3B
Source Dataset
The experiments use the PubMedQA dataset:
qiaoxin/PubMedQA
This repository contains generated/derived research artifacts and does
not redistribute the original PubMedQA dataset in its entirety.… See the full description on the dataset page: https://huggingface.co/datasets/chrislimbe/pubmedqa-recursive-llm-degradation-qwen2.5-3b.Qwen2.5-7B-Instruct-Self-Calibration
Efficient Test-Time Scaling via Self-Calibration
This repository contains datasets used in the paper Efficient Test-Time Scaling via Self-Calibration. The datasets are used to evaluate the effectiveness of test-time scaling methods for LLMs. Each config_name in the metadata refers to a different reasoning dataset. More detailed descriptions of each dataset are needed. Consider adding a section for each config_name with a description, statistics, and any other relevant… See the full description on the dataset page: https://huggingface.co/datasets/HINT-lab/Qwen2.5-7B-Instruct-Self-Calibration.fineweb-edu-2016-qwen2
FineWeb-Edu 2016 / Qwen2
Completed 2016 crawl-year processing.
Documents: 93,099,356. Actual recounted Qwen2 tokens: 99,541,846,844.
Source: HuggingFaceFW/fineweb-edu, revision 87f09149ef4734204d70ed1d046ddc9ca3f2b8f9.
The input inventory covers 9 crawl directories. date is the integer crawl year 2016,
not an article publication date. Original text is preserved without cleaning,
normalization, deduplication, truncation, or added formatting. Source token counts
are not used.… See the full description on the dataset page: https://huggingface.co/datasets/BoomQ/fineweb-edu-2016-qwen2.Qwen2.5-Coder-0.5B-Flutter-steps-eval
Qwen2.5-Coder-0.5B Flutter — Steps Mode — Validation Results
Dataset Summary
Held-out evaluation results for bbidpa/Qwen2.5-Coder-0.5B-Flutter-steps,
a fine-tune of Qwen2.5-Coder-0.5B for editing Flutter/Dart source files. In steps
mode, the model is given an existing file and an edit instruction and generates a
sequence of localized search/replace edit actions, each mechanically applied to the
current file state before the next action is generated, until the… See the full description on the dataset page: https://huggingface.co/datasets/bbidpa/Qwen2.5-Coder-0.5B-Flutter-steps-eval.multireward-grpo-gsm8k-rewards-qwen2.5-7b
Multi-Reward GRPO — GSM8K Rewards (Qwen2.5-7B-Instruct)
Raw rollout-level reward observations from the empirical Section of
"Conditioned Multi-Reward Advantage Estimation: A Finite-Sample Analysis".
This is the data that produced the headline Theorem 3 (correlation-dependent
MSE floor) and Proposition 4 (sign-changing conditioning bias) figures on
real LLM rollouts. Each rollout was sampled from Qwen/Qwen2.5-7B-Instruct on
GSM8K test prompts at temperature 0.7.
What's in… See the full description on the dataset page: https://huggingface.co/datasets/eagle0504/multireward-grpo-gsm8k-rewards-qwen2.5-7b.Qwen2.5-Coder-0.5B-Flutter-direct-eval
Qwen2.5-Coder-0.5B Flutter — Direct Mode — Validation Results
Dataset Summary
Held-out evaluation results for bbidpa/Qwen2.5-Coder-0.5B-Flutter-direct,
a fine-tune of Qwen2.5-Coder-0.5B for editing Flutter/Dart source files. In direct
mode, the model is given an existing file and an edit instruction and generates the
complete modified file in a single forward pass (as opposed to the steps /
iterative diff-based mode — see the sibling dataset… See the full description on the dataset page: https://huggingface.co/datasets/bbidpa/Qwen2.5-Coder-0.5B-Flutter-direct-eval.fineweb-edu-2016-qwen2-sample
FineWeb-Edu 2016 / Qwen2
Consistency sample — not the completed year.
Documents: 900. Actual recounted Qwen2 tokens: 937,977.
Source: HuggingFaceFW/fineweb-edu, revision 87f09149ef4734204d70ed1d046ddc9ca3f2b8f9.
The input inventory covers 9 crawl directories. date is the integer crawl year 2016,
not an article publication date. Original text is preserved without cleaning,
normalization, deduplication, truncation, or added formatting. Source token counts
are not used. Optional… See the full description on the dataset page: https://huggingface.co/datasets/BoomQ/fineweb-edu-2016-qwen2-sample.qwen2.5-7b-instruct-nla-L20-finefineweb-100k
Qwen2.5-7B-Instruct NLA training data — residual stream, block 20
Training data for a Natural Language Autoencoder on Qwen/Qwen2.5-7B-Instruct:
residual-stream activations paired with natural-language explanations of the text
they were taken from.
Unlike the dataset this is derived from, the activation_vector column is
included — every EasyNLA/nanoNLA trainer requires it.
Trained models: https://huggingface.co/Yooniel/qwen2.5-7b-instruct-nla-L20
(AV val ppl 4.07, AR held-out FVE… See the full description on the dataset page: https://huggingface.co/datasets/Yooniel/qwen2.5-7b-instruct-nla-L20-finefineweb-100k.Magpie-Qwen2-Pro-300K-Filtered-koTranslated Magpie-Align/Magpie-Qwen2-Pro-300K-Filtered using nayohan/llama3-instrucTrans-enko-8b.
For this dataset, we only used data that is 5000 characters or less in length and has language of English.
Thanks for @Magpie-Align and @nayohan.
@misc{xu2024magpie,
title={Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing},
author={Zhangchen Xu and Fengqing Jiang and Luyao Niu and Yuntian Deng and Radha Poovendran and Yejin Choi and Bill Yuchen Lin}… See the full description on the dataset page: https://huggingface.co/datasets/youjunhyeok/Magpie-Qwen2-Pro-300K-Filtered-ko.qwen25-math-sft-long-15k-v1
Qwen2.5 Math SFT Long 15K v1
A reproducible Long CoT SFT dataset for Qwen2.5-Math-1.5B with max context 3,840 tokens (below 4,096 limit).
Source: open-r1/OpenR1-Math-220k
Total samples: 15,000
Decontaminated against GSM8K, SVAMP, MATH-500, AIME 2026.
qwen25-math-sft-long-think7k-v1
Qwen2.5 Math Long-CoT thinking 7k
2,500 train and 250 validation rows. Long examples have 4,096–7,168 thinking tokens; complete ChatML samples including final answer and EOS fit in 8,192 tokens.
Long source: open-r1/OpenR1-Math-220k at e4e141ec9dea9f8326f4d347be56105859b2bd68 (only math-verified generations).
Short replay source: danghoang2005/qwen25-math-sft-long-15k-v1 at 32f0afc54a09c4963ae2478cf93871025abda901.
Tokenizer: Qwen/Qwen2.5-Math-1.5B at… See the full description on the dataset page: https://huggingface.co/datasets/danghoang2005/qwen25-math-sft-long-think7k-v1.BF_EVAL-cd3args-Qwen2.5-1.5B-Instruct-RLThese datasets are exactly like the Evaluation datasets except the model_responses array are budget forcing rounds.
So the first response is at a maximum total context length of 4k, the second response (2nd index in the array) is a continuation of that last response up to a total of 8,192 tokens.
Column Details
Column
Description
question
The question we want the model to answer
answer
The string answer
task
The name of the task the row belongs to
prompt
The prompt… See the full description on the dataset page: https://huggingface.co/datasets/SkillFactory/BF_EVAL-cd3args-Qwen2.5-1.5B-Instruct-RL.BF_EVAL-cd3args-Qwen2.5-1.5B-Instruct-SkillFactory-RLThese datasets are exactly like the Evaluation datasets except the model_responses array are budget forcing rounds.
So the first response is at a maximum total context length of 4k, the second response (2nd index in the array) is a continuation of that last response up to a total of 8,192 tokens.
Column Details
Column
Description
question
The question we want the model to answer
answer
The string answer
task
The name of the task the row belongs to
prompt
The prompt… See the full description on the dataset page: https://huggingface.co/datasets/SkillFactory/BF_EVAL-cd3args-Qwen2.5-1.5B-Instruct-SkillFactory-RL.creative-qwen2.5-7b-stories
Ethahtz/creative-qwen2.5-7b-stories
LLM creative generations from the creative_tasks pipeline (generate_infinite_chats.py) or any compatible generations.jsonl.
Reproducibility and full per-run parameters are in generation_config.json in this dataset repository (one entry per config / run).
Configs and loading
Qwen-Qwen2.5-7B-Instruct_dsshort_story_prompts_bkvllm_seed42_top
Model: Qwen/Qwen2.5-7B-Instruct
Prompt source (dataset):… See the full description on the dataset page: https://huggingface.co/datasets/Ethahtz/creative-qwen2.5-7b-stories.EVAL-cd3args-Qwen2.5-1.5B-Instruct-SkillFactory-RL
Column Details
Column
Description
question
The question we want the model to answer
answer
The string answer
task
The name of the task the row belongs to
prompt
The prompt we will feed into the model to solve the question
model_responses
An array of strings that the model generated to answer the prompt (usually size of 4 or 34 depending on the evaluation task)
model_responses__eval_is_correct
An array aligned with model_responses containing booleans: True when… See the full description on the dataset page: https://huggingface.co/datasets/SkillFactory/EVAL-cd3args-Qwen2.5-1.5B-Instruct-SkillFactory-RL.Magpie-Qwen2-Pro-200K-English-koTranslated Magpie-Align/Magpie-Qwen2-Pro-200K-English using nayohan/llama3-instrucTrans-enko-8b.
For this dataset, we only used data that is 5000 characters or less in length and has language of English.
Thanks for @Magpie-Align and @nayohan.
@misc{xu2024magpie,
title={Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing},
author={Zhangchen Xu and Fengqing Jiang and Luyao Niu and Yuntian Deng and Radha Poovendran and Yejin Choi and Bill Yuchen Lin}… See the full description on the dataset page: https://huggingface.co/datasets/youjunhyeok/Magpie-Qwen2-Pro-200K-English-ko.EVAL-cd3args-Qwen2.5-7B-Instruct-RL
Column Details
Column
Description
question
The question we want the model to answer
answer
The string answer
task
The name of the task the row belongs to
prompt
The prompt we will feed into the model to solve the question
model_responses
An array of strings that the model generated to answer the prompt (usually size of 4 or 34 depending on the evaluation task)
model_responses__eval_is_correct
An array aligned with model_responses containing booleans: True when… See the full description on the dataset page: https://huggingface.co/datasets/SkillFactory/EVAL-cd3args-Qwen2.5-7B-Instruct-RL.Magpie-Ko-Qwen2.5-Reasoning-Raw
Data Generation
System Message
Korean Reasoning Template
"pre_query_template": "<|im_start|>system\n당신은 알리바바 클라우드에서 만든 Qwen입니다. 당신은 유용한 어시스턴트입니다.\nuser가 논리적인 다단계의 추론 과정이 필요한 복잡한 문제를 내면, assistant는 한국어로 단계적으로 풀이를 제시합니다.<|im_end|>\n<|im_start|>user\n"
(Original)
"pre_query_template": "<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n<|im_start|>user\n"
Model
Question: Qwen/Qwen2.5-32B-Instruct
With… See the full description on the dataset page: https://huggingface.co/datasets/werty1248/Magpie-Ko-Qwen2.5-Reasoning-Raw.EVAL-cd3args-Qwen2.5-1.5B-Instruct-BoLT-RL
Column Details
Column
Description
question
The question we want the model to answer
answer
The string answer
task
The name of the task the row belongs to
prompt
The prompt we will feed into the model to solve the question
model_responses
An array of strings that the model generated to answer the prompt (usually size of 4 or 34 depending on the evaluation task)
model_responses__eval_is_correct
An array aligned with model_responses containing booleans: True when… See the full description on the dataset page: https://huggingface.co/datasets/SkillFactory/EVAL-cd3args-Qwen2.5-1.5B-Instruct-BoLT-RL.EVAL-cd3args-Qwen2.5-1.5B-Instruct-R1-SFT
Column Details
Column
Description
question
The question we want the model to answer
answer
The string answer
task
The name of the task the row belongs to
prompt
The prompt we will feed into the model to solve the question
model_responses
An array of strings that the model generated to answer the prompt (usually size of 4 or 34 depending on the evaluation task)
model_responses__eval_is_correct
An array aligned with model_responses containing booleans: True when… See the full description on the dataset page: https://huggingface.co/datasets/SkillFactory/EVAL-cd3args-Qwen2.5-1.5B-Instruct-R1-SFT.EVAL-cd3args-Qwen2.5-7B-Instruct
Column Details
Column
Description
question
The question we want the model to answer
answer
The string answer
task
The name of the task the row belongs to
prompt
The prompt we will feed into the model to solve the question
model_responses
An array of strings that the model generated to answer the prompt (usually size of 4 or 34 depending on the evaluation task)
model_responses__eval_is_correct
An array aligned with model_responses containing booleans: True when… See the full description on the dataset page: https://huggingface.co/datasets/SkillFactory/EVAL-cd3args-Qwen2.5-7B-Instruct.EVAL-cd3args-Qwen2.5-1.5B-Instruct-RL
Column Details
Column
Description
question
The question we want the model to answer
answer
The string answer
task
The name of the task the row belongs to
prompt
The prompt we will feed into the model to solve the question
model_responses
An array of strings that the model generated to answer the prompt (usually size of 4 or 34 depending on the evaluation task)
model_responses__eval_is_correct
An array aligned with model_responses containing booleans: True when… See the full description on the dataset page: https://huggingface.co/datasets/SkillFactory/EVAL-cd3args-Qwen2.5-1.5B-Instruct-RL.EVAL-cd3args-Qwen2.5-7B-Instruct-SkillFactory-SFT
Column Details
Column
Description
question
The question we want the model to answer
answer
The string answer
task
The name of the task the row belongs to
prompt
The prompt we will feed into the model to solve the question
model_responses
An array of strings that the model generated to answer the prompt (usually size of 4 or 34 depending on the evaluation task)
model_responses__eval_is_correct
An array aligned with model_responses containing booleans: True when… See the full description on the dataset page: https://huggingface.co/datasets/SkillFactory/EVAL-cd3args-Qwen2.5-7B-Instruct-SkillFactory-SFT.EVAL-cd3args-Qwen2.5-1.5B-Instruct-BoLT-SFT
Column Details
Column
Description
question
The question we want the model to answer
answer
The string answer
task
The name of the task the row belongs to
prompt
The prompt we will feed into the model to solve the question
model_responses
An array of strings that the model generated to answer the prompt (usually size of 4 or 34 depending on the evaluation task)
model_responses__eval_is_correct
An array aligned with model_responses containing booleans: True when… See the full description on the dataset page: https://huggingface.co/datasets/SkillFactory/EVAL-cd3args-Qwen2.5-1.5B-Instruct-BoLT-SFT.EVAL-cd3args-Qwen2.5-1.5B-Instruct-R1-RL
Column Details
Column
Description
question
The question we want the model to answer
answer
The string answer
task
The name of the task the row belongs to
prompt
The prompt we will feed into the model to solve the question
model_responses
An array of strings that the model generated to answer the prompt (usually size of 4 or 34 depending on the evaluation task)
model_responses__eval_is_correct
An array aligned with model_responses containing booleans: True when… See the full description on the dataset page: https://huggingface.co/datasets/SkillFactory/EVAL-cd3args-Qwen2.5-1.5B-Instruct-R1-RL.BF_EVAL-cd3args-Qwen2.5-1.5B-Instruct-R1-SFTThese datasets are exactly like the Evaluation datasets except the model_responses array are budget forcing rounds.
So the first response is at a maximum total context length of 4k, the second response (2nd index in the array) is a continuation of that last response up to a total of 8,192 tokens.
Column Details
Column
Description
question
The question we want the model to answer
answer
The string answer
task
The name of the task the row belongs to
prompt
The prompt… See the full description on the dataset page: https://huggingface.co/datasets/SkillFactory/BF_EVAL-cd3args-Qwen2.5-1.5B-Instruct-R1-SFT.
