datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
OpenGrad-ToolPolicy-Canonical-v2-M0-snapshot
This is a provenance-preserving canonical candidate corpus. It is a pre-training canonical release, not an empirically selected or recommended training mixture.
What this release is
OpenGrad ToolPolicy Canonical v2 is a provenance-preserving, model-independent normalization of public tool-use and function-calling datasets. It was built to test one hypothesis with a measurement attached: that the tool-call collapse observed in the M0 SFT experiments on v1 was caused by the… See the full description on the dataset page: https://huggingface.co/datasets/arjhinety/OpenGrad-ToolPolicy-Canonical-v2-M0-snapshot.OpenGrad-ToolPolicy-Canonical-v1
This is a provenance-preserving canonical candidate corpus. It is a pre-training canonical release, not an empirically selected or recommended training mixture.
What this release is
OpenGrad ToolPolicy Canonical v1 is a provenance-preserving, model-independent normalization of several public tool-use and function-calling datasets. It is released as a pre-training candidate corpus for controlled research into tool-use policy in small open-weight language models. See OpenGrad… See the full description on the dataset page: https://huggingface.co/datasets/arjhinety/OpenGrad-ToolPolicy-Canonical-v1.ZINC-canonicalized
dataset description
We downloaded ZINC dataset from here and canonicalized it.
We used the following function to canonicalize the data and removed some SMILES that cannot be read by RDKit.
from rdkit import Chem
def canonicalize(mol):
mol = Chem.MolToSmiles(Chem.MolFromSmiles(mol),True)
return mol
We randomly split the preprocessed data into train and validation. The ratio is 9 : 1.
aquatype-canonical-ctx-20260706llama-nemotron-science-reasoning-on-canonical-think-full
Llama-Nemotron science reasoning — Delphi canonical-think (COMPLETE, no length filter)
The complete reasoning:on science split of
nvidia/Llama-Nemotron-Post-Training-Dataset, converted once into the canonical
Delphi chat-template thinking format. 708,920 rows.
Unlike the cold-start warmup slice
open-athena/llama-nemotron-science-reasoning-on-le3000tok-100k
(and its -canonical-think variant), this build applies no length cap and no subsample — every
long-CoT science example is… See the full description on the dataset page: https://huggingface.co/datasets/laion/llama-nemotron-science-reasoning-on-canonical-think-full.canonical_prompt_collectionOpenGrad-ToolPolicy-Canonical-v2
This is a provenance-preserving canonical candidate corpus. It is a pre-training canonical release, not an empirically selected or recommended training mixture.
What this release is
OpenGrad ToolPolicy Canonical v2 is a provenance-preserving, model-independent normalization of
public tool-use datasets in which every record declares what it supervises. It exists because
not every legitimate post-training corpus has the same conversational trajectory shape, and
discarding a… See the full description on the dataset page: https://huggingface.co/datasets/arjhinety/OpenGrad-ToolPolicy-Canonical-v2.OpenGrad-ToolPolicy-Canonical-v2-minus-xlam
This is the byte-identical training view for a joint xLAM-plus-CALL_PREDICTION removal
experiment. xLAM is currently the corpus's only source of that supervision contract, so this is
not a pure source-content ablation. It carries no result of its own and is not a recommended
mixture. It is part of OpenGrad Study 001.
What this is
OpenGrad-ToolPolicy-Canonical-v2
with one source removed: xLAM/APIGen. Three sources remain, 115,895 canonical records, 118
shards.
It is the exact… See the full description on the dataset page: https://huggingface.co/datasets/arjhinety/OpenGrad-ToolPolicy-Canonical-v2-minus-xlam.gurbani-sehajpath-yt-captions-canonical
Gurbani Sehajpath — Canonical-aligned ASR corpus
Stage-1 + Stage-2 canonical pipeline output for sehaj-path (calm recitation of the Guru Granth Sahib). Built from publicly available audio recordings with aligned transcripts, chunked by caption timing and aligned against the canonical Guru Granth Sahib Ji text (SGGS).
Columns
Schema is auto-inferred from the parquet shards. Primary columns:
audio — 16 kHz mono waveform
final_text — canonical Gurmukhi transcription (post… See the full description on the dataset page: https://huggingface.co/datasets/surindersinghssj/gurbani-sehajpath-yt-captions-canonical.pubchem-10m-canonicalized
dataset description
We downloaded PubChem-10m dataset from here and canonicalized it.
We used the following function to canonicalize the data and removed some SMILES that cannot be read by RDKit.
from rdkit import Chem
def canonicalize(mol):
mol = Chem.MolToSmiles(Chem.MolFromSmiles(mol),True)
return mol
We randomly split the preprocessed data into train and validation. The ratio is 9 : 1.
vil-canonical-glyph-system
VIL Canonical Glyph System
Canonical tri-layer glyph dataset for the GlyphMatics / SigilAGI / VIL stack.
Tri-layer identity
glyph = (visible, braille, hanzi)
digest = SHA256(visible + braille + hanzi)
Layers
α-layer: visible canonical symbol / glyph role
β-layer: Braille-inspired structural state
γ-layer: Hanzi temporal-semantic context
Canonical role system
ID
Name
Role
G0
Origin
Root state
G1
Split
Branch
G2
Bind
Merge
G3
Flow
Transition
G4
Gate
Conditional
G5… See the full description on the dataset page: https://huggingface.co/datasets/Nine1Eight/vil-canonical-glyph-system.canonical_obligation_datasetcanonical-dataset
canonical-dataset
A parallel, multilingual dataset on rule-following
Languages
en — English
am — Amharic
de — German
hi — Hindi
ig — Igbo
it — Italian
ko — Korean
ru — Russian
sw — Swahili
ta — Tamil
tr — Turkish
ur — Urdu
yo — Yoruba
Loading
from datasets import load_dataset
en = load_dataset("canonical-dataset", "en", split="test")
yo = load_dataset("canonical-dataset", "yo", split="test")
lichess-stockfish-canonicalCIC-IoT-2023-canonical-neto
CIC-IoT-2023 — Canonical (Neto et al.) Variant
This is the canonical CIC-IoT-2023 dataset, sourced from
bencorn/CIC-IoT-2023's
CSV/MERGED_CSV/ folder, which contains Neto et al.'s authentic merged CSVs
WITH embedded labels (vs. bencorn's other CSV/CSV/<attack>/ re-organization
which lost ~6.5M rows during the folder-restructure).
Why this exists: prior lacg030175/CIC-IoT-2023-full and -full-raw were
built from CSV/CSV/ and contained only 38.5M rows. This one contains
~45,019,243… See the full description on the dataset page: https://huggingface.co/datasets/lacg030175/CIC-IoT-2023-canonical-neto.canonical-order-problem
List Extraction Dataset
A dataset for studying how large language models (LLMs) represent and generate multi-valued relations — relations in which a single subject is associated with a set of entities (e.g., "all countries in South America", "all elements of the periodic table").
It accompanies the paper:
The Canonical Order Problem: When Large Language Models Are Unreliable Knowledge Bases for Multi-Valued Relations
Timo Pierre Schrader, Annemarie Friedrich, Simon Razniewski… See the full description on the dataset page: https://huggingface.co/datasets/timo-pierre-schrader/canonical-order-problem.augmented_canonical_pubchem_13m
PubChem 10M - Augmented SMILES Dataset
This dataset is derived from the original PubChem 10M and has been canonicalized using RDKit (2024.9.4) to ensure structural consistency.
To enhance molecular diversity, 33% of the dataset was randomly sampled and augmented using RDKit’s Chem.MolToRandomSmilesVect function, following an approach similar to NVIDIA’s molmim method for SMILES augmentation.
Dataset Overview:
Source: PubChem 10M
Canonicalization: RDKit (2024.9.4)… See the full description on the dataset page: https://huggingface.co/datasets/Derify/augmented_canonical_pubchem_13m.augmented_canonical_druglike_QED_43m
Druglike QED 43M - Augmented SMILES Dataset
This dataset is derived from the Druglike molecule datasets for drug discovery dataset and has been canonicalized using RDKit (2024.9.4) to ensure structural consistency.
To enhance molecular diversity, 33% of the dataset was randomly sampled and augmented using RDKit’s Chem.MolToRandomSmilesVect function, following an approach similar to NVIDIA's molmim method for SMILES augmentation.
Dataset Overview:
Source:… See the full description on the dataset page: https://huggingface.co/datasets/Derify/augmented_canonical_druglike_QED_43m.paranames-canonical-name-translation-data-convo-format-no-whitespace-en2xxaugmented_canonical_druglike_QED_Pfizer_15m
Druglike QED Pfizer 15M - Augmented SMILES Dataset
This dataset is derived from the Druglike molecule datasets for drug discovery dataset and has been canonicalized using RDKit (2024.9.4) to ensure structural consistency.
To enhance molecular diversity, 33% of the dataset was randomly sampled and augmented using RDKit’s Chem.MolToRandomSmilesVect function, following an approach similar to NVIDIA's molmim method for SMILES augmentation.
Dataset Overview:
Source:… See the full description on the dataset page: https://huggingface.co/datasets/Derify/augmented_canonical_druglike_QED_Pfizer_15m.context-conditioned-molecule-transfer-v10.3-ames-mixed-canonical-intern
Ames context-conditioned molecule transfer V10.3
This release preserves its direct panels and appends training-only, an exact 1:1 set of canonical-display V10 record-level transfer pairs. Query values remain hidden from prompts.
Train rows: 184,896
Validation rows: 38,128
Test rows: 36,475
context-conditioned-molecule-transfer-v10.3-carcinogens-mixed-canonical-intern
Carcinogens context-conditioned molecule transfer V10.3
This release preserves its direct panels and appends training-only, an exact 1:1 set of canonical-display V10 record-level transfer pairs. Query values remain hidden from prompts.
Train rows: 360,384
Validation rows: 45,810
Test rows: 43,702
cci-multilingualrules-canonical-dataset
canonical-dataset
A parallel, multilingual dataset on rule-following
Languages
en — English
am — Amharic
de — German
hi — Hindi
ig — Igbo
it — Italian
ko — Korean
ru — Russian
sw — Swahili
ta — Tamil
tr — Turkish
ur — Urdu
yo — Yoruba
Loading
from datasets import load_dataset
en = load_dataset("canonical-dataset", "en", split="test")
yo = load_dataset("canonical-dataset", "yo", split="test")
context-conditioned-molecule-transfer-v10.3-dili-mixed-canonical-intern
DILI context-conditioned molecule transfer V10.3
This release preserves its direct panels and appends training-only, an exact 1:1 set of canonical-display V10 record-level transfer pairs. Query values remain hidden from prompts.
Train rows: 154,560
Validation rows: 52,564
Test rows: 49,992
paranames-canonical-name-translation-data-convo-format-no-whitespace-xx2endroid_xxjd_2_canonicalThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "Franka",
"total_episodes": 27,
"total_frames": 30357,
"total_tasks": 2,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 500,
"fps": 15,
"splits": {
"train": "0:27"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/michios/droid_xxjd_2_canonical.canonical-drafter-ei1m-sft
canonical-drafter ei1m SFT dataset
Round-1 drafter SFT parquets, merged Babel + Delta ei1_collect pools (see
docs/logs/expert-iter-runs-delta.md in the repo, sha 8ea6ade era).
ei1m_drafter-train.parquet — 29,935 rows
ei1m_drafter-val.parquet — 616 rows (84 held-out modules, ~2%, group-aware)
Build (2026-07-14, Delta):
python scripts/sft/build_sft_parquet.py OUT /work/nvme/bhvj/ahe4/rollout_traces/ei1_collect /work/nvme/bhvj/ahe4/babel_ei1/ei1_collect… See the full description on the dataset page: https://huggingface.co/datasets/awhecmu/canonical-drafter-ei1m-sft.openarm-restock-sequences-canonical-30fps-subtasks-gripper-vlm
restock-sequences-canonical-30fps
LeRobot v2.1 dataset: 226 episodes, 306218 frames at 30 fps.
Robot: openarm_bimanual
Cameras: observation.images.context, observation.images.wrist_left, observation.images.wrist_right
State/action dim: 16
Load it with the v2.1 tag, which is the revision the training path pins.
D-DATA-canonical_dataset_splits-v1-7_13_25-BACKUP-9_22_25context-conditioned-molecule-transfer-v10.3.1-dili-mixed-canonical-intern
DILI context-conditioned molecule transfer V10.3.1
This release freezes the V10.3 auxiliary rows and all direct strata except positive-positive transfers, which are deterministically downsampled to a 2:1 ratio against negative-negative transfers. Query values remain hidden from prompts.
Train rows: 141,444
Validation rows: 52,564
Test rows: 49,992