heiheiha798/perfectblend-regen-qwen3-8b-non-thinking

PerfectBlend Regen Qwen3-8B Non-Thinking 32768 Clean

中文

本仓库包含 PerfectBlend 数据经 Qwen3-8B non-thinking 模式重新生成后的清理版本。数据从 max_tokens=4096 的原始 regen 结果出发，对疑似截断样本继续做 8192 / 16384 / 32768 token 补生成，并移除 context-length 超限、尾部退化，以及 32768 max new tokens 打满但没有自然结束的无界生成样本。

文件

格式

Parquet 分片使用以下 schema：

{
  "id": 0,
  "conversations": [
    {"role": "user", "content": "..."},
    {"role": "assistant", "content": "..."}
  ],
  "assistant_token_stats": {
    "assistant_turn_tokens": [1012],
    "assistant_turn_count": 1,
    "assistant_total_tokens": 1012,
    "assistant_max_tokens": 1012
  }
}

• —id：原始输入行索引（0 .. 1420908），为权威主键。
• —conversations：重新生成后的对话，严格 user / assistant 交替，以 user 开头、assistant 结尾；不包含 reasoning_content（non-thinking 模式）。
• —assistant_token_stats：使用 Qwen3-8B tokenizer 对 assistant message 内容统计的 token 数；不包含 user message，也不重新渲染 chat template。

Token 统计

使用 Qwen3-8B tokenizer 统计 assistant message 内容，add_special_tokens=false。

assistant_total_tokens: 1,295,678,420
assistant_turns: 1,883,354
avg_assistant_tokens_per_sample: 912.42
avg_assistant_tokens_per_turn: 687.96
max_assistant_turn_tokens: 32,767
max_assistant_total_tokens_per_sample: 32,767
max_assistant_turn_count_per_sample: 36

dropped_ids.jsonl 每行是一个 JSON 对象：

{
  "id": 0,
  "reason": "tail_degeneracy"
}

• —id：被排除的原始输入行索引。
• —reason：排除原因，目前包括 context_length_exceeded、tail_degeneracy、unbounded_generation_hit_max_new_tokens_32768。

生成配置

模型与模式：

target model: Qwen3-8B
mode: non-thinking
chat_template_kwargs: {"enable_thinking": false}

SGLang serving 参数：

dtype: bfloat16
tp: 1
mem_fraction_static: 0.90
reasoning_parser: qwen3
cuda_graph_backend_decode: disabled
cuda_graph_backend_prefill: disabled

原始 regen 参数：

bsz / concurrency: 128
temperature: 0.6
top_p: 0.95
max_tokens: 4096
reasoning: disable

context-length / decode-cap 补生成参数：

retry max_tokens: 8192
second retry max_tokens: 16384
third retry max_tokens: 32768
temperature: 0.6
top_p: 0.95
reasoning: disable
context_length check: 40960

匿名化命令模板：

python -m sglang.launch_server   --model-path <qwen3_8b_model_path>   --host <host>   --port <port>   --dtype bfloat16   --tp 1   --mem-fraction-static 0.90   --trust-remote-code   --reasoning-parser qwen3   --cuda-graph-backend-decode disabled   --cuda-graph-backend-prefill disabled

python regenerate_train_data.py   --model <qwen3_8b_model_path>   --server-address <host>:<port>   --input-file-path <perfectblend_train.jsonl>   --output-file-path <shard_output.jsonl>   --concurrency 128   --max-tokens 4096   --temperature 0.6   --top-p 0.95   --reasoning disable

说明

原始 PerfectBlend regen release 共有 1,420,909 条。本清理版本收录 1,420,046 条，另外 863 条未包含在本 release 中：

context-length 超限样本是在解码前因 prompt_tokens + max_tokens >= context_length 被主动跳过：其中 1 条来自 8192 retry，14 条来自 16384 retry，23 条来自 32768 retry。

尾部退化样本来自确定性的后处理扫描和对 32768 cap-hit 样本的复查。检测包括 exact periodic suffix、数字模板化 periodic suffix、尾部附近重复的 line / sentence / paragraph block、长单字符重复、长空白字符重复、结构符号重复，以及低信息尾部。

32768 max new tokens 打满的无界生成样本没有被尾部退化规则命中，但仍在 32768 token 上限处停止，未自然结束。这类样本可能包含长文本、长代码、枚举或试错过程；它们不被视为尾部重复退化，但作为完整 assistant response 是截断样本，因此从 clean release 中排除。

请根据自己的任务需求继续做安全过滤、质量过滤、去重和 license 合规检查。

English

This repository contains a cleaned PerfectBlend regeneration release produced by Qwen3-8B in non-thinking mode. It starts from the original max_tokens=4096 regeneration output, reruns likely truncated samples with 8192 / 16384 / 32768 token limits, and removes rows affected by context-length limits, tail degeneration, or unbounded generations that hit the 32768 max-new-tokens cap without naturally stopping.

Files

Format

The Parquet shards use the following schema:

{
  "id": 0,
  "conversations": [
    {"role": "user", "content": "..."},
    {"role": "assistant", "content": "..."}
  ],
  "assistant_token_stats": {
    "assistant_turn_tokens": [1012],
    "assistant_turn_count": 1,
    "assistant_total_tokens": 1012,
    "assistant_max_tokens": 1012
  }
}

• —id: original input line index (0 .. 1420908); the authoritative key.
• —conversations: regenerated conversation, strictly alternating user / assistant, starting with user and ending with assistant; no reasoning_content (non-thinking mode).
• —assistant_token_stats: token counts for assistant message contents measured with the Qwen3-8B tokenizer; user messages are not counted, and the chat template is not rendered for this statistic.

Token Statistics

Measured with the Qwen3-8B tokenizer on assistant message contents only, with add_special_tokens=false.

assistant_total_tokens: 1,295,678,420
assistant_turns: 1,883,354
avg_assistant_tokens_per_sample: 912.42
avg_assistant_tokens_per_turn: 687.96
max_assistant_turn_tokens: 32,767
max_assistant_total_tokens_per_sample: 32,767
max_assistant_turn_count_per_sample: 36

Each line in dropped_ids.jsonl is a JSON object:

{
  "id": 0,
  "reason": "tail_degeneracy"
}

• —id: dropped original input line index.
• —reason: drop reason, currently one of context_length_exceeded, tail_degeneracy, and unbounded_generation_hit_max_new_tokens_32768.

Generation Config

Model and mode:

target model: Qwen3-8B
mode: non-thinking
chat_template_kwargs: {"enable_thinking": false}

SGLang serving parameters:

dtype: bfloat16
tp: 1
mem_fraction_static: 0.90
reasoning_parser: qwen3
cuda_graph_backend_decode: disabled
cuda_graph_backend_prefill: disabled

Original regen parameters:

bsz / concurrency: 128
temperature: 0.6
top_p: 0.95
max_tokens: 4096
reasoning: disable

Context-length / decode-cap retry parameters:

retry max_tokens: 8192
second retry max_tokens: 16384
third retry max_tokens: 32768
temperature: 0.6
top_p: 0.95
reasoning: disable
context_length check: 40960

Anonymized command template:

python -m sglang.launch_server   --model-path <qwen3_8b_model_path>   --host <host>   --port <port>   --dtype bfloat16   --tp 1   --mem-fraction-static 0.90   --trust-remote-code   --reasoning-parser qwen3   --cuda-graph-backend-decode disabled   --cuda-graph-backend-prefill disabled

python regenerate_train_data.py   --model <qwen3_8b_model_path>   --server-address <host>:<port>   --input-file-path <perfectblend_train.jsonl>   --output-file-path <shard_output.jsonl>   --concurrency 128   --max-tokens 4096   --temperature 0.6   --top-p 0.95   --reasoning disable

Notes

The original PerfectBlend regen release contains 1,420,909 rows. This cleaned release includes 1,420,046 rows; the other 863 rows are not included in this release:

Rows dropped for context-length reasons were skipped before decoding because prompt_tokens + max_tokens >= context_length: 1 row in the 8192 retry pass, 14 rows in the 16384 retry pass, and 23 rows in the 32768 retry pass.

Rows dropped for tail degeneration were found by deterministic post-processing scans and a recheck of the 32768 cap-hit samples. The checks include exact periodic suffixes, numeric-template periodic suffixes, repeated line / sentence / paragraph blocks near the tail, long single-character runs, long whitespace runs, structural-symbol runs, and low-information tails.

Rows dropped for unbounded 32768 max-new-tokens hits were not flagged by the tail-degeneration rules, but still stopped only at the 32768 token cap rather than ending naturally. They may contain long-form text, long code, enumeration, or trial-and-error reasoning; they are not treated as tail-repetition degeneration, but they are incomplete assistant responses and are excluded from this clean release.

Please apply task-specific safety filtering, quality filtering, deduplication, and license compliance checks before downstream use.