Arushhh/Llama-HybridDiffusion-processed-data-run1

Llama-HybridDiffusion processed training mixture — run 1

Built with Llama.

This repository preserves the exact Hugging Face Dataset.save_to_disk Arrow snapshot used by run 1 of a Qwen3.5-2B HybridDiffusion reproduction. The directory names, dataset_info.json, state.json, and Arrow shard boundaries are retained so the data can be downloaded and supplied to the existing training configuration without a lossy format conversion.

Exact snapshot inventory

MANIFEST.tsv records the relative path and exact byte size of every file in this snapshot. METADATA_SHA256.tsv records SHA-256 checksums for the six small Dataset metadata/state JSON files. Large payloads use Hugging Face Xet's content-addressed transfer and are also checked after publication by exact remote file list and size.

Training mixture

The released run configuration samples the three directories independently with weights 0.4 Long-SFT, 0.4 Math, and 0.2 instruction following, packing each source to sequence length 4,096. Mixture weights are sampling probabilities, not row ratios.

The normalized schema is:

• —messages_json: serialized role/content conversation messages
• —category: normalized domain label
• —source: normalized source/split label
• —total_tokens: estimated serialized token count
• —assistant_tokens: estimated supervised assistant-token count

Exact source composition

Long-SFT-3K

Nemotron-Math-Domain

Nemotron-IF-Domain-Cascade

Important reproduction difference

This snapshot's source column proves that the Cascade-2 portion is the complete public instruction_following input (cascade2_if_raw_open), not NVIDIA's separate constraint-prefiltered approximately 362K-row selector described by the exact paper recipe. Therefore this is the complete data actually used by run 1, but it must not be described as an exact copy of the paper's unreleased/prefiltered IF pool. This can materially affect reproduction metrics.

Download and restore

from huggingface_hub import snapshot_download
from datasets import load_from_disk

root = snapshot_download(
    repo_id="Arushhh/Llama-HybridDiffusion-processed-data-run1",
    repo_type="dataset",
)
long_sft = load_from_disk(f"{root}/Long-SFT-3K")
math = load_from_disk(f"{root}/Nemotron-Math-Domain")
instruction_following = load_from_disk(f"{root}/Nemotron-IF-Domain-Cascade")

For the existing run configuration, set DATA_ROOT to the downloaded repository root. Do not rename the three dataset directories.

Construction

The builders normalize heterogeneous message structures, merge separate reasoning into balanced <think>...</think> blocks, reject malformed conversations, estimate token counts with the Qwen tokenizer, and save the five-column Arrow schema. Long-SFT applies the >=3K estimated-token selection. Math selects the enabled Llama-Nemotron math rows and Nemotron Math-Proofs. The IF builder normalizes, structurally filters, and exactly deduplicates serialized conversations.

Exact build code is preserved at: https://github.com/Arush777/HybridDiffusion-run1-backup-20260821/tree/backup/instance-20260821 (commit 8d68b860c314de99f6e66813283f4e988f3dfa3a).

Licensing and attribution

This is a transformed multi-source dataset and has no single superseding licence. The normalized Arrow schema does not retain each original row's license field. The source label is retained, and users must comply with every applicable source and per-sample term. See ATTRIBUTION.md, NOTICE, LICENSES/, and SOURCE_CARDS/. Redistribution here does not relicense any source material.

Limitations and responsible use

• —The data may contain incorrect reasoning, unsafe text, personal information, biased content, copyrighted material, or source contamination; no new row-level legal, privacy, safety, or benchmark-contamination audit was performed for this release.
• —Token counts are estimates used by the builders, not a promise about every tokenizer version.
• —The IF pool differs from the paper's unavailable constraint-prefiltered selection.
• —Models trained on these data can memorize or reproduce source material.
• —Review upstream model/data acceptable-use policies before training or deployment.

This release is for reproducibility and research transparency. It is provided as-is.