Datasetpublic

queefwath/k12-math-code-dataset

K-12 Math & Coding Dataset A ~2 billion token dataset built for K-12 math and coding education use cases: 1,004,991,667 tokens of math content and 1,020,505,114 tokens of code content, both comfortably over the 1B-token target for each category. Dataset Summary Category Tokens Rows File Math 1,004,991,667 676,018 math/final.jsonl Code 1,020,505,114 937,248 code/final.jsonl Built via a 6-phase pipeline: source collection → cleaning (PII redaction… See the full description on the dataset page: https://huggingface.co/datasets/queefwath/k12-math-code-dataset.

sourceHugging Faceodc-byupdated 1mo agoView on Hugging Face

0likes10downloads

Dataset Card

K-12 Math & Coding Dataset

A ~2 billion token dataset built for K-12 math and coding education use cases: 1,004,991,667 tokens of math content and 1,020,505,114 tokens of code content, both comfortably over the 1B-token target for each category.

Dataset Summary

Category	Tokens	Rows	File
Math	1,004,991,667	676,018	`math/final.jsonl`
Code	1,020,505,114	937,248	`code/final.jsonl`

Built via a 6-phase pipeline: source collection → cleaning (PII redaction, boilerplate/spam removal, encoding fixes) → deduplication (exact SHA-256 + fuzzy MinHash LSH) → heuristic quality filtering → budget-controlled final sampling → manual quality audit. Full methodology, per-source breakdowns, and known limitations are in final_report.md.

Schema

Each line is a JSON object:

json

{
  "text": "...",
  "source": "finemath4plus",
  "category": "math",
  "quality_score": 4.3,
  "token_count": 512,
  "language": "en",
  "metadata": {}
}

—quality_score: populated only for finemath4plus (has an upstream classifier score); null for all other sources.
—language: natural-language code (all records verified predominantly English). For code records, the actual programming language is under metadata.programming_language.
—metadata: source-specific extra fields (URL, license, file path, repo name, exercise name, difficulty, etc. — varies by source).

Sources

Math: FineMath 4+, OpenWebMath, GSM8K, MATH (hendrycks_math). Code: GitHub Code (Python/JavaScript/Java/HTML/CSS via codeparrot/github-code-clean), freeCodeCamp, Exercism, APPS, CodeContests.

Token Counting

All token counts use the tiktoken cl100k_base tokenizer, computed via exact encode_ordinary() calls (not estimates) at every pipeline stage.

Quality Audit

Both categories were manually spot-checked twice (once pre-assembly, once on the final sampled corpus). Code passed cleanly at every check (~90-94% good, 0-2% bad). Math's final spot-check found a 15% "bad" rate (above the 10% target threshold) driven by several low-frequency, hard-to-cheaply-filter patterns (AI-generated SEO filler, off-topic forum comment dumps, one essay-mill advertisement, thin template pages) — documented as an accepted residual limitation rather than silently delivered. See `final_audit.md` and `final_report.md` Section 8 for full detail before using this dataset for anything quality-sensitive.

Known Limitations

See final_report.md Section 8 for the complete list, including: math corpus residual bad-rate, AMPS/OpenStax not sourced, The Stack v2 substituted with GitHub Code due to gating, freeCodeCamp/Exercism volume shortfall, and HTML's residual auto-generated-doc contamination.

License

Source licenses vary per record — see metadata.license for code records where available. FineMath 4+ and OpenWebMath are ODC-By. Aggregate dataset provided as-is for research/educational use; verify individual source licenses before redistribution.