datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
arxiv_cplusplus_research_code
Dataset card for ArtifactAI/arxiv_cplusplus_research_code
Dataset Description
https://huggingface.co/datasets/AlgorithmicResearchGroup/arxiv_cplusplus_research_code
Dataset Summary
ArtifactAI/arxiv_python_research_code contains over 10.6GB of source code files referenced strictly in ArXiv papers. The dataset serves as a curated dataset for Code LLMs.
How to use it
from datasets import load_dataset
# full dataset (10.6GB of data)
ds =… See the full description on the dataset page: https://huggingface.co/datasets/AlgorithmicResearchGroup/arxiv_cplusplus_research_code.fineweb-edu-zh-chengyu-cpt
Fineweb-Edu Chinese — Chengyu-Tagged Continued-Pretraining Corpus
A 3.74M-document Chinese corpus (~7.8B tokens) for continued pretraining on
cultural knowledge in figurative language, built from the highest-quality
tier of opencsg/Fineweb-Edu-Chinese-V2.1.
Each document is educational Chinese text containing at least one culturally
vetted chengyu, with an appended 【成语注释】 knowledge block listing every
matched idiom's figurative meaning(s) and classical source citation.
This is a… See the full description on the dataset page: https://huggingface.co/datasets/jiviteshjn/fineweb-edu-zh-chengyu-cpt.carbon-cpu-enriched-sequences
carbon-cpu-enriched-sequences
A CPU-enriched subset of the carbon pretraining corpus (eukaryote_generator), combining original source fields with normalized sequences
and row-level features for quality analysis, GPU enrichment and embedding generation.
Information of Features
Feature
Type
Description
record_id
string
NCBI Identifier linking the row back to the source genomic record. It provides the primary record-level identity.
begin_of_sequence… See the full description on the dataset page: https://huggingface.co/datasets/AINovice2005/carbon-cpu-enriched-sequences.cpt_instruction_datasets
Instruction datasets
Collection of synthetic instruction datasets used during the continued pretraining of Model-small-instr-1, Model-small-instr-2 and Model-small-instr-3. You can currently find these models under: Llama-3.1-Carballo-Instr1 and Llama-3.1-Carballo-Instr3.
Dataset creation
Datasets were created using two different techniques:
Adapting already existing datasets or corpora by modifying their format to make them suitable for including instructions during… See the full description on the dataset page: https://huggingface.co/datasets/proxectonos/cpt_instruction_datasets.asterion-cpt-corpus
Asterion CPT Source Corpus
What's inside — 297,185 synthetic technical documents about the fictional Asterion Space Operations fleet (24 satellites: EO/COMM/SCI/TD): 7 doc_types × 12 topics, ~2.5 GB of text, ~1.62B Gemma-4 tokens (measured mean 5,413 tokens/doc on a stratified 2k sample, 2026-07-03).
Where it comes from — Fully synthetic — generated incrementally in 1,000-doc shards by the Asterion corpus generation pipeline (see noval-corp/docs/asterion-corpus-plan.md) from a… See the full description on the dataset page: https://huggingface.co/datasets/atenareply/asterion-cpt-corpus.code_contest_instruct_cppqwen3-5-tiny-cpu-repro-v1
Qwen3.5 tiny native random CPU fixture
Complete randomly initialized, untrained Qwen3_5ForConditionalGeneration checkpoint.
This is a pipeline/reproducibility fixture, not a useful language model, distillation,
quantization, quality benchmark, or claim about the performance of Qwen3.8-27B.
No upstream model weights or training data were used. No paid GPU/cloud compute.
Architecture and lineage
Architecture lineage: Qwen/Qwen3.8-27B at… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/qwen3-5-tiny-cpu-repro-v1.glm5-next-tiny-cpu-repro-v1This repository is an evidence bundle, not one root-format dataset at repository root.
first/ and repeat/ are separate complete sealed QFS root datasets; comparison/ holds
the comparison receipt and tokenwise result. panel/ is the sealed input panel. Other files
are provenance, logs and reproduction tools. Do not pass the bundle root as a QFS dataset.
GLM5-Next tiny native CPU fixture
This is a complete untrained random-initialized native Glm5NextForConditionalGeneration
wrapper… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/glm5-next-tiny-cpu-repro-v1.k2-horizon-tiny-cpu-repro-v1
K2-Horizon MoVA tiny random CPU fixture
Complete untrained K2HorizonForCausalLM, not Moonshot Kimi despite the K2 name.
Architecture source: IFM/K2-Horizon-MoVA-36B-A4B at 05cab0a4d7150c1c460a000b37ff40cc1af2feaa.
No pretrained weights, original tokenizer, training data, paid GPU or cloud compute used.
Complete text-only K2HorizonForCausalLM, not Kimi: three-layer dense prefix followed by two real MoVA+MoE layers, grouped RMSNorm, sigmoid top-k routing with selection-only bias… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/k2-horizon-tiny-cpu-repro-v1.deepseek-v4-tiny-cpu-repro-v1
DeepSeek-V4 tiny corrected-native-primitives CPU text fixture
Complete randomly initialized, untrained QFSDeepseekV4ForCausalLM text class
using Transformers5.16.1 native primitives and a reviewed RMSNorm arithmetic correction.
No upstream weights, paid GPU/cloud compute or useful-model claim.
This is not unmodified native Transformers or the complete production release.
Architecture and scope
Text lineage:… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/deepseek-v4-tiny-cpu-repro-v1.minimax-m3-tiny-cpu-repro-v1
minimax-m3 complete native tiny random CPU fixture
Complete untrained MiniMaxM3SparseForConditionalGeneration checkpoint with an untied full LM head,
a real 272-entry byte tokenizer and every native state tensor. Architecture lineage:
MiniMaxAI/MiniMax-M3@f0e1c1e04d40177e4673a22097036854f536e9c0.
No upstream weights, training data, paid GPU or cloud compute were used.
Complete native image/text wrapper with real shrunk Conv3D vision, nonempty 3D RoPE, patch-merge projector and… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/minimax-m3-tiny-cpu-repro-v1.minimax-m2-tiny-cpu-repro-v1
minimax-m2 complete native tiny random CPU fixture
Complete untrained MiniMaxM2ForCausalLM checkpoint with an untied full LM head,
a real 272-entry byte tokenizer and every native state tensor. Architecture lineage:
MiniMaxAI/MiniMax-M2.7@d494266a4affc0d2995ba1fa35c8481cbd84294b.
No upstream weights, training data, paid GPU or cloud compute were used.
Complete native text causal LM: sigmoid/top-k MoE routing with correction bias, per-layer flattened Q/K RMSNorm and half-head… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/minimax-m2-tiny-cpu-repro-v1.bulgarian-medical-cpt-100m
Bulgarian text for MOSS continued pretraining
Exactly 100 million training tokens: 10M medical and 90M general Bulgarian.
An additional 100,000 tokens are provided for validation (50k per source). No audio, instruction-response pairs, or generated answers.
No model training has been performed as part of this dataset build.
Medical data
Exactly 10,000,000 training tokens and 50,000 additional validation tokens,
including one <|im_end|> EOS per record. Extracted… See the full description on the dataset page: https://huggingface.co/datasets/DimitarV/bulgarian-medical-cpt-100m.Romulus-cpt-fr
Romulus, continually pre-trained models for French law.
Romulus is a series of continually pre-trained models enriched in French law and intended to serve as the basis for a fine-tuning process on labeled data. Please note that these models have not been aligned for the production of usable text as they stand, and will certainly need to be fine-tuned for the desired tasks in order to produce satisfactory results.
The training corpus is made up of around 34,864,949 tokens… See the full description on the dataset page: https://huggingface.co/datasets/louisbrulenaudet/Romulus-cpt-fr.mc4-zh-idiom-cpt
mC4 zh — Idiom-Tagged Continued-Pretraining Corpus
A 9.6M-document Chinese corpus for continued pretraining on cultural knowledge in
figurative language. Each document is natural web text (from the C4/mC4 zh subset)
containing at least one culturally meaningful chengyu, with an appended knowledge
block that lists every matched idiom together with its figurative meaning(s) and
classical source citation.
Built 2026-07-16 as Stage 1 (continue-pretraining data) of the… See the full description on the dataset page: https://huggingface.co/datasets/jiviteshjn/mc4-zh-idiom-cpt.bulgarian-medical-cpt-10m
Bulgarian text for MOSS continued pretraining
Exactly 10 million training tokens: 3M medical and 7M general Bulgarian.
An additional 100,000 tokens are provided for validation (50k per source). No audio, instruction-response pairs, or generated answers.
No model training has been performed as part of this dataset build.
Medical data
Exactly 3,000,000 training tokens and 50,000 additional validation tokens,
including one <|im_end|> EOS per record. Extracted… See the full description on the dataset page: https://huggingface.co/datasets/DimitarV/bulgarian-medical-cpt-10m.wiki-events-cpt
Wikipedia Events CPT
jhdlee/wiki-events-cpt is a public research dataset of 150 selected English Wikipedia articles with compact metadata for continual pretraining (CPT).
Split
Articles
Event window (end exclusive)
cohort_a
75
2023-01-01 to 2024-10-01
cohort_b
75
2024-10-01 to 2025-09-01
Each cohort has 25 articles per topic: natural_hazards, elections, and sports. Cohorts group events by their reviewed whole-occurrence intervals; they are not… See the full description on the dataset page: https://huggingface.co/datasets/jhdlee/wiki-events-cpt.ptbr-creative-cpt-qwen35-08b-v02
PT-BR Creative CPT — Qwen3.5-0.8B data-prep v0.2
This repository is a derived, model/tokenizer-specific training artifact for continued pretraining experiments.
It is not the canonical text corpus.
Canonical source:
oliveirabruno01/ptbr-creative-cpt
Canonical corpus fingerprint:
21f72f64b3b73425bc78d91046a52aefddb8413b747d69f3422c31da8f536840
Identity
Model/tokenizer: Qwen/Qwen3.5-0.8B-Base
Context length: 2048
Data-prep version: v0.2
Primary split policy:… See the full description on the dataset page: https://huggingface.co/datasets/oliveirabruno01/ptbr-creative-cpt-qwen35-08b-v02.ptbr-creative-cpt
PT-BR Creative Corpus v0.1.0
A curated Brazilian-Portuguese creative-writing corpus for continued pretraining / midtraining research.
Status
This is the canonical corpus freeze, not a final model-specific training build.
Canonical text units: 1,354
Document/edition entities: 803
Characters: 82,538,439
Words (whitespace count): 13,929,410
Historical project estimate: 18,339,188 chars/4.5 tokens, retained only in the audit_metrics config.
The canonical corpus… See the full description on the dataset page: https://huggingface.co/datasets/oliveirabruno01/ptbr-creative-cpt.step2-evaluated-dataset-Qwen3-14B-cp32
Complete Evaluation Dataset (Rubric + LogP)
This dataset contains chain-of-thought explanations evaluated using both comprehensive rubric assessment and LogP evaluation.
Overview
Source Dataset: llm-compe-2025-kato/step2-evaluated-dataset-Qwen3-14B-cp32
Total Samples: 60
Successfully Evaluated (Rubric): 53
Failed Evaluations (Rubric): 7
Evaluation Model: Qwen/Qwen3-32B
Rubric Evaluation Results
Average Rubric Scores (0-4 scale)
logical_coherence:… See the full description on the dataset page: https://huggingface.co/datasets/llm-compe-2025-kato/step2-evaluated-dataset-Qwen3-14B-cp32.idea-first-code-later-cp
Idea First, Code Later: CP Benchmark
Benchmark dataset for the paper: "Idea First, Code Later: Disentangling Problem Solving from Code Generation in Evaluating LLMs for Competitive Programming"
A curated benchmark of 83 competitive programming problems designed for evaluating LLMs on algorithmic problem-solving separately from code generation.
Motivation
We curate problems from seven contests that are not hosted on major public CP platforms (e.g., Codeforces, AtCoder).… See the full description on the dataset page: https://huggingface.co/datasets/samahadhoud/idea-first-code-later-cp.step2-evaluated-dataset-Qwen3-14B-cp40
Complete Evaluation Dataset (Rubric + LogP)
This dataset contains chain-of-thought explanations evaluated using both comprehensive rubric assessment and LogP evaluation.
Overview
Source Dataset: llm-compe-2025-kato/step2-evaluated-dataset-Qwen3-14B-cp40
Total Samples: 58
Successfully Evaluated (Rubric): 53
Failed Evaluations (Rubric): 5
Evaluation Model: Qwen/Qwen3-32B
Rubric Evaluation Results
Average Rubric Scores (0-4 scale)
logical_coherence:… See the full description on the dataset page: https://huggingface.co/datasets/llm-compe-2025-kato/step2-evaluated-dataset-Qwen3-14B-cp40.neurips-2026-cpx
neurips-2026-cpx — Korean OSCE history-taking dialogues with a GPT-4o virtual standardized patient
49 text-based history-taking dialogue sessions between 17 senior Korean
medical-student participants (Years 3–4 of a 6-year curriculum) and a
GPT-4o-driven virtual standardized patient (VSP). Released as the
empirical evaluation dataset accompanying our NeurIPS 2026 submission.
Sessions: 49
Participants: 17 (anonymised to R001–R017)
Total QA turns: 1,763
Language: Korean
Domain:… See the full description on the dataset page: https://huggingface.co/datasets/neurips-2026-cpx/neurips-2026-cpx.cpa-tax-scenarios-2026
CPA Tax Scenarios 2026
768 CPA tax impact scenarios by income, loan, filing status.
Details
Records: 768
Format: JSONL
License: CC-BY-4.0
Last Updated: March 2026
Verified By: Wendy Thompson, CPA, CDLP, NMLS #504814
Publisher: Wendy Thompson Lending Team
Thompson Alpha Logic
Data models calculating the after-tax cost of mortgage debt across purchase, refinance, divorce buyout, and reverse mortgage scenarios. Compares itemized vs. standard deduction ($30K… See the full description on the dataset page: https://huggingface.co/datasets/Wendy-Thompson-Lending-Team/cpa-tax-scenarios-2026.legal-chunks-cpt
Legal Document Chunks for Continued Pretraining
This dataset contains 1329 legal document chunks extracted from various legal documents across multiple jurisdictions. Each chunk is enriched with comprehensive metadata labels for filtering, analysis, and domain-specific training.
Dataset Information
Total Chunks: 1,329
Format: jsonl-text
Sorted: By document ID and chunk index (maintains document continuity)
Source: Legal documents processed through enhanced parser with… See the full description on the dataset page: https://huggingface.co/datasets/rzeraat/legal-chunks-cpt.HyperSwitch-Repo-CPT-Dataset-v2
Hyperswitch Rust Codebase Dataset
A comprehensive dataset extracted from the Hyperswitch open-source payment processing platform, containing 16,731 code samples across 37 modules with 6.99M tokens for training Rust code understanding and generation models.
📊 Dataset Overview
This dataset provides both file-level and granular code samples from Hyperswitch, a modern payment switch written in Rust. It's designed for training code models to understand payment processing… See the full description on the dataset page: https://huggingface.co/datasets/AdityaNarayan/HyperSwitch-Repo-CPT-Dataset-v2.HyperSwitch-Repo-CPT-Dataset
Hyperswitch Rust Codebase Dataset
A comprehensive dataset extracted from the Hyperswitch open-source payment processing platform, containing 16,731 code samples across 37 modules with 6.99M tokens for training Rust code understanding and generation models.
📊 Dataset Overview
This dataset provides both file-level and granular code samples from Hyperswitch, a modern payment switch written in Rust. It's designed for training code models to understand payment processing… See the full description on the dataset page: https://huggingface.co/datasets/AdityaNarayan/HyperSwitch-Repo-CPT-Dataset.stf-acordaos-cpt-2048
STF Acordaos CPT 2048
Dataset de acórdãos do STF preparado para continuous pre-training (CPT), com foco em textos jurídicos e preservação da cauda final dos documentos longos (parte 2, parte 3, etc.).
Origem dos dados
Fonte pública: https://dadosabertos.c3sl.ufpr.br/acordaos/json/
Arquivos de origem utilizados:
DocumentosAcordaos.json
AcordaosVotos.json
AcordaosRelatorios.json
Construção
Fonte canônica: DocumentosAcordaos.json
Labels incluídos: integra, voto… See the full description on the dataset page: https://huggingface.co/datasets/costadev00/stf-acordaos-cpt-2048.