Lelonthecodeur/multi-task-dataset

Multi-Task Dataset

Description

A large-scale multi-task dataset designed for training and evaluating AI models across reasoning, mathematics, code, research, verification, data analysis, and general problem solving.

Content

• —100,000,001 examples
• —20+ task families
• —English + French
• —Train / Validation / Test splits
• —Structured reasoning and verification signals
• —Multiple difficulty levels
• —OOD and generalization-oriented examples

Dataset Structure

Task Categories

Main Capabilities

The dataset is designed to improve:

• —Mathematical reasoning
• —Logical reasoning
• —Code generation
• —Code understanding
• —Data analysis
• —Research methodology
• —Error detection
• —Error correction
• —Self-checking
• —Instruction following
• —Constraint satisfaction
• —Counterexample detection
• —Ambiguity resolution
• —Prompt consistency
• —Confidence estimation
• —Uncertainty handling
• —Generalization
• —Verification

Difficulty

Examples are distributed across multiple difficulty levels:

• —easy
• —medium
• —hard
• —very_hard
• —extreme

Verification Signals

Examples can contain structured fields for:

• —math_check
• —logic_check
• —code_check
• —data_analysis_check
• —constraint_check
• —consistency_check
• —counterexample_check
• —evidence_check
• —source_check
• —error_detection
• —error_repair
• —prompt_review
• —prompt_alignment
• —confidence
• —uncertainty
• —answerability

Usage

Install the required library:

pip install -U datasets

Load the dataset:

from datasets import load_dataset

dataset = load_dataset(
    "Lelonthecodeur/multi-task-dataset",
    streaming=True
)

train = dataset["train"]

for example in train:
    print(example)
    break

Load a specific split:

from datasets import load_dataset

train = load_dataset(
    "Lelonthecodeur/multi-task-dataset",
    split="train",
    streaming=True
)

Streaming is recommended for the full dataset because of its size.

Hugging Face CLI

Login:

hf auth login

Clone:

git lfs install
git clone https://huggingface.co/datasets/Lelonthecodeur/multi-task-dataset

Push an update:

cd multi-task-dataset
git add .
git commit -m "Update dataset"
git push

Python Upload

from huggingface_hub import HfApi

api = HfApi(token="YOUR_HF_TOKEN")

api.upload_folder(
    folder_path="/kaggle/working/multi-task-dataset",
    repo_id="Lelonthecodeur/multi-task-dataset",
    repo_type="dataset",
    commit_message="Update dataset",
)

Data Format

The dataset is stored in Parquet format.

Main fields include:

id
task_family
task_type
domain
difficulty
language
instruction
context
response
analysis_plan
verification
prompt_review
prompt_alignment
constraint_check
consistency_check
math_check
logic_check
counterexample_check
data_analysis_check
code_check
evidence_check
source_check
hallucination_control
error_detection
error_repair
answerability
confidence
uncertainty
reasoning_depth
minimal_sufficient_reasoning
unnecessary_reasoning
stop_condition
surface_variation
numeric_variation
structure_variation
ood_style
quality_score
generator_version

Future Updates

V2 — Robustness

Planned improvements:

• —Harder reasoning tasks
• —Adversarial examples
• —Hard negatives
• —Better deduplication
• —Near-duplicate detection
• —Leakage detection
• —Stronger OOD splits
• —Better generalization testing

V3 — Science & Research

Planned additions:

• —Scientific reasoning
• —Scientific knowledge
• —Research methodology
• —Experimental design
• —Hypothesis evaluation
• —Scientific data analysis
• —Evidence comparison
• —Source comparison
• —Uncertainty analysis

V4 — Mega Deep

Planned addition of approximately 10M highly difficult examples.

The objective is to target specific weaknesses found during model evaluation instead of simply increasing prompt complexity.

Model
  ↓
Benchmark
  ↓
Failure Detection
  ↓
Weak Skill Detection
  ↓
Targeted Hard Examples
  ↓
Verification
  ↓
Deduplication
  ↓
OOD / Adversarial Tests
  ↓
Training
  ↓
New Benchmark

V5 — Science × Knowledge × Logic × Experience

Future expansion combining:

• —Science
• —Knowledge
• —Complex logic
• —Experience-based problem solving
• —Cross-domain reasoning
• —Multi-step verification
• —Novel situations
• —Adaptive evaluation

Font

For standard text:

import matplotlib.pyplot as plt

plt.rcParams["font.family"] = "DejaVu Sans"

For multilingual text:

import matplotlib.pyplot as plt

plt.rcParams["font.family"] = ["Noto Sans", "Noto Sans CJK JP"]

License

MIT License

Copyright (c) 2026 Lelonthecodeur

Permission is hereby granted, free of charge, to any person obtaining a copy of this dataset and associated files, to use, copy, modify, merge, publish, distribute, sublicense, and sell copies of the dataset, subject to the conditions of the MIT License.

Version

Current version: v1.1

Total examples: 100,000,001

Format: Parquet

Status: Active development

Citation

@dataset{multi_task_dataset,
  title        = {multi-task-dataset},
  author       = {Lelonthecodeur},
  year         = {2026},
  publisher    = {Hugging Face},
  version      = {1.1},
  note         = {100,000,001 examples}
}