datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
diffusion-vs-ar-hard-sudoku
Diffusion vs AR Hard Sudoku
This repository packages 8,148,696 Sudoku examples in the CSV format expected
by HKUNLP/diffusion-vs-ar, plus its original 100k/1k easy baseline.
Every processed file has these columns:
column
meaning
quizzes
81 row-major digits; 0 is an empty cell
solutions
complete 81-digit solution
source
original collection
dataset
normalized dataset family
official_rating
rating supplied by the source
rating_type
semantics of that rating… See the full description on the dataset page: https://huggingface.co/datasets/fhyfhy/diffusion-vs-ar-hard-sudoku.japanese-math-empirical-difficulty-pilot-50k
Japanese Math Empirical Difficulty Pilot 50k
This dataset is a 50,000-problem empirical difficulty pilot, not a full empirical labeling of the original 5.66M-row source dataset.
It was created for LLM-jp experiment 0399, Team Victory SFT, to validate empirical difficulty label distribution, downstream split behavior, and the rollout/scoring pipeline before attempting labeling at the full 5.6M scale.
Current Status
This upload uses the v3 scorer with assistant-only… See the full description on the dataset page: https://huggingface.co/datasets/argo11/japanese-math-empirical-difficulty-pilot-50k.multidimensional-difference-awareness-legacy
Multidimensional Difference Awareness
A source-grounded benchmark for bias-import in real eligibility decisions:
whether a model applies a rule's legitimate criteria while refusing to let an
attribute the rule does not use change the outcome.
What is new here. Wang et al. (ACL 2025,
arXiv:2502.01926) measure difference
awareness on general-knowledge facts. Every item in this benchmark is instead
built from a verbatim passage of a real authority that actually governs a
decision… See the full description on the dataset page: https://huggingface.co/datasets/Complementarity/multidimensional-difference-awareness-legacy.difficult_problem_dataset_v4_w_scores
OverView
This dataset is a synthetic dataset created using the Scalable Data Generation (SDG) framework.It is structured for use with a thinking model, and the input and output form a set of questions and answers.
This dataset is created by merging, shuffling, and scoring the following datasets:
ikedachin/difficult_problem_dataset_v2
ikedachin/difficult_problem_dataset_v3
Dataset Structure
{
"id": "A unique ID",
"input": "question",
"output":… See the full description on the dataset page: https://huggingface.co/datasets/ikedachin/difficult_problem_dataset_v4_w_scores.clean_openthought312_difficulty_9_filterd元データ: https://huggingface.co/datasets/open-thoughts/OpenThoughts3-1.2M
diffculty 9でさらにフィルタリングしたもの
データ件数: 14,339
平均トークン数: 13370
最大トークン数: 16,808
合計トークン数: 191,708,678
ファイル形式: JSONL
ファイルサイズ: 723.9 MB
clean_openthought312_difficulty_9_qwentoken元データ: https://huggingface.co/datasets/LLMTeamAkiyama/clean_openthought312_difficulty_9_filterd
データ件数: 14,339
平均トークン数: 13,367
最大トークン数: 16,805
合計トークン数: 191,665,652
ファイル形式: JSONL
ファイル分割数: 3
合計ファイルサイズ: 724.7 MB
加工内容：
**tokenizeをQwen235B-A22Bで再度トークン化したものを出力
使用したコード
https://github.com/LLMTeamAkiyama/0-data_prepare/blob/master/src/openthoughts3/clean_openthoughts3_9_qwentoken.ipynb