Zek-Takai/glm53-flash-harvest

GLM-5.3-Flash On-Policy Harvest

86,006 responses / 246,034,910 generated tokens written by zai-org/GLM-5.3-Flash from its reference FP8 weights, across four harvest rounds, 15 registers and both serving modes (22,016 rows carry the model's inline <think>…</think> chain). It is on-policy text: the corpus records what the target model actually generates, which is what a speculative-decoding drafter (EAGLE-3 / DFlash / DSpark family) has to learn to predict. Everything here is MIT.

Companion page: glm53-flash-internals, a measured profile of the same model's internals (activation landscapes, attention-sink anatomy, FP8 margins to 260k context, sparse-indexer selection maps), produced with Claude Code and Spectra Scope.

At a glance

Quick start

from datasets import load_dataset

# the whole corpus, four rounds, one schema
ds = load_dataset("Zek-Takai/glm53-flash-harvest", split="train")

# or exactly the slice you want, by config name (table below)
r4    = load_dataset("Zek-Takai/glm53-flash-harvest", "round4", split="train")
code  = load_dataset("Zek-Takai/glm53-flash-harvest", "agentic_coding", split="train")
think = load_dataset("Zek-Takai/glm53-flash-harvest", "thinking_on", split="train")

Pull what you need

Every config is a re-shard of the same rows; pick by round, by serving mode, or by register.

By round

By serving mode

By register

Provenance for rounds 3-4

provenance_r34 (20,537 rows, one per (id, temp)) carries where each round-3/4 prompt came from and how the generation ended: source, mode (single / conversation), turn and conv_id for conversation rows, has_system, tokens, finish (stop, length, or null for rows generated before the run recorded it), the truncation flags described under Truncation, and meta_json (SWE-bench instance id, repository and file license, MATH level, prefix token estimate, and so on).

The four rounds

Round 1 — balanced base

Single-turn, thinking off, six registers, prompts built by deterministic combinatorics (seed 42). Every request was capped at 1,600 tokens and 27,589 rows (59.2%) end exactly there, so treat round 1 as short-form text; the cap is the reason rounds 3-4 run uncapped.

Round 2 — coverage expansion

A 27,462-prompt bank (seed 43) sampled at two temperatures per prompt; the run was budget-limited, so the banked rows are a uniform sample of the design mix. Thinking was switched on for half the prompts, and four registers appear for the first time: reasoning, agentic tool-calling, Chinese, and multi-turn. Chat and multi-turn prompts come from human turns in open-perfectblend, with the final assistant turn regenerated by GLM-5.3-Flash. Caps were set per row, 1,600-4,096 tokens by register, raised to 6,144 with thinking; 4,178 of 9,465 thinking chains did not close inside that budget (see Truncation).

Round 3 — gap-fill

The two thirds of the seed-43 bank that round 2 never reached, harvested with no output cap: the request carries no max_tokens, the model stops at its own end-of-turn, and the only wall is the engine's 49,152-token context window. Four registers are deepened; thinking is on for 67.6% of rows and reasoning chains run as long as the model wants (68.8M tokens of chains).

Round 4 — production realism

Same uncapped policy, prompts drawn from real material instead of synthesis:

• —agentic_coding — 810 rows over 417 SWE-bench Verified instances: the issue plus the relevant repository files, worked as an agentic loop.
• —code_edit — 476 edits and reviews of real files from github-code-clean, 246 repositories, permissive licenses only (MIT / Apache-2.0 / BSD / ISC / CC0 / Unlicense; the license is in meta_json).
• —multiturn — 2,710 rows from 197 conversations seeded by open-perfectblend, up to 9 assistant turns each. Every turn is its own row with exactly the context it saw (messages_json); row ids end in #tK for turn K and conv_id groups them.
• —long_context — 543 rows whose prompt is an 8k-38k-token prefix: multi-file code bundles or packs of Wikipedia articles, followed by a task over them.
• —agentic_tooling — 565 synthetic non-code tool loops.
• —deep_reasoning — 96 hendrycks MATH level 4-5 problems, thinking on, chains uncapped.

System prompts are layered onto 885 rows (17.0%); they sit in messages_json and has_system marks them.

Schema

Every row parquet shares one schema (Parquet, zstd):

provenance_r34 adds, per round-3/4 row: temp, round, source, mode, turn, conv_id, has_system, tokens, finish, cap_hit, legacy_cap, truncated, think_tag_in_off, meta_json.

Truncation and label hygiene

The text is kept exactly as generated. Three things can make a row an imperfect label, and each is measurable:

• —Ends at a request cap. Rounds 1-2 ran with max_tokens (round 1 a flat 1,600; round 2 per row, 1,600-4,096 by register and 6,144 with thinking); a row whose tokens reached its cap was cut off. Rounds 3-4 ran uncapped; 7 round-3 rows are leftovers from the first 1h48m of that run, before the cap was removed (legacy_cap).
• —Hit the context window. Rounds 3-4 only: the generation reached the 49,152-token engine window (finish = length). For thinking rows this means the chain never closed (cap_hit).
• —Stray think tag. A thinking-off row where the model still emitted <think> inside its answer (think_tag_in_off).

Round 4's open chains sit almost entirely in agentic_coding (285 of 418 thinking rows): the repository context already fills much of the window before the model starts reasoning.

To keep only clean labels:

# rounds 3-4: the flags are in provenance_r34, joined on (id, temp)
prov = load_dataset("Zek-Takai/glm53-flash-harvest", "provenance_r34", split="train")
bad = {(p["id"], p["temp"]) for p in prov if p["truncated"]}
r34 = load_dataset("Zek-Takai/glm53-flash-harvest", data_files="data/glm53_harvest_r[34].parquet", split="train")
clean34 = r34.filter(lambda r: (r["id"], r["temp"]) not in bad)

# rounds 1-2: derive them from the row itself (rows sitting exactly on a cap value; the exact per-row
# cap is the max_tokens field of the matching prompt in prompts/prompts.jsonl / prompts2.jsonl)
r12 = load_dataset("Zek-Takai/glm53-flash-harvest", data_files=["data/glm53_harvest.parquet", "data/glm53_harvest_r2.parquet"], split="train")
clean12 = r12.filter(lambda r: r["tokens"] not in {1600, 2048, 2560, 4096, 6144}
                              and not (r["thinking"] and "</think>" not in r["response"]))

How it was generated

• —Model and weights: GLM-5.3-Flash, zai-org reference FP8 checkpoint (62 safetensors shards). config.json SHA-256 bb8f01c42cb92a52ca72e65afb4d5bd8d11aef083cd210e8de25dfb904f23e9f, chat template SHA-256 6d9307d2bff501dcd6a948dce55570bded990d3b78ade9cc780cfe953285fd11.
• —Serving: vLLM day-0 image. Rounds 3-4: vllm/vllm-openai:glm53-flash-x86_64-cu130 on one 4x NVIDIA B200 node, tensor-parallel 4, max-model-len 49,152, CUDA graphs on. Thinking is switched per request through chat-template kwargs (patch_chat_template.py).
• —Sampling: temperatures 0.6 / 0.8 / 0.95 / 1.05 cycled evenly; round 1 one temperature per prompt, rounds 2-4 two per prompt, so most prompts appear twice with different temp.
• —Length policy: rounds 1-2 used max_tokens (see the table above); rounds 3-4 send none and let the model stop on its own.
• —Deduplication: rows are unique per (id, temp). Where the round-3/4 ledger held two generations for a key (a capped early row and its uncapped regeneration), the release keeps the one whose chain closed, then the uncapped one, then the longer one.

Prompts

prompts/ holds the exact prompt banks and their deterministic builders:

harvest34.py is the round-3/4 harvester (no max_tokens, no client timeout, per-row thinking flag, conversation turns replayed with the exact context they saw). patch_chat_template.py is the chat-template patch that exposes the thinking switch.

Prompt sources: round 1 synthetic and original; round 2 synthetic plus human prompt turns from mlabonne/open-perfectblend (via mgoin/open-perfectblend-glm5.2-regen) for chat and multi-turn; round 3 the same seed-43 bank; round 4 SWE-bench Verified (MIT), github-code-clean (Apache-2.0, files filtered to MIT / Apache / BSD / ISC / CC0), open-perfectblend-glm5.2-regen seeds, hendrycks MATH (MIT), and Wikipedia 2023-11-01 English (CC-BY-SA 3.0; document packs inside long-context prompts carry their article titles). See each source for the licensing of the underlying prompts.

License

All responses are GLM-5.3-Flash output (MIT model). The corpus is released under MIT.

Citation

@misc{glm53flashharvest2026,
  title  = {GLM-5.3-Flash On-Policy Harvest},
  author = {Zek-Takai},
  year   = {2026},
  url    = {https://huggingface.co/datasets/Zek-Takai/glm53-flash-harvest}
}