heiheiha798/sharegpt-regen-qwen3-8b-non-thinking
ShareGPT Regen Qwen3-8B Non-Thinking 40K Clean 中文 本仓库包含 ShareGPT 数据经 Qwen3-8B non-thinking 模式重新生成、长输出重试和退化样本清理后的版本。原始 4K regen 的 max-token 命中样本依次经过 16K、40K retry；最终移除了 context length 超限、尾部退化/重复、以及人工确认的无界生成样本。 文件 文件 说明 行数 data/train-00000-of-00004.parquet Hugging Face parquet 分片 10,944 data/train-00001-of-00004.parquet Hugging Face parquet 分片 10,944 data/train-00002-of-00004.parquet Hugging Face parquet 分片 10,943… See the full description on the dataset page: https://huggingface.co/datasets/heiheiha798/sharegpt-regen-qwen3-8b-non-thinking.
ShareGPT Regen Qwen3-8B Non-Thinking 40K Clean
中文
本仓库包含 ShareGPT 数据经 Qwen3-8B non-thinking 模式重新生成、长输出重试和退化样本清理后的版本。原始 4K regen 的 max-token 命中样本依次经过 16K、40K retry；最终移除了 context length 超限、尾部退化/重复、以及人工确认的无界生成样本。
文件
Token 统计
使用 Qwen3-8B tokenizer 统计 assistant message 内容，add_special_tokens=false；不包含 user message，也不重新渲染 chat template。
assistant_total_tokens: 114,995,139
assistant_turns: 134,564
avg_assistant_tokens_per_sample: 2,627.02
avg_assistant_tokens_per_turn: 854.58
max_assistant_turn_tokens: 14,760
max_assistant_total_tokens_per_sample: 31,717Parquet 格式
{
"id": "example_id",
"conversations": [
{"role": "user", "content": "..."},
{"role": "assistant", "content": "..."}
],
"assistant_token_stats": {
"assistant_turn_tokens": [955, 985],
"assistant_turn_count": 2,
"assistant_total_tokens": 1940,
"assistant_max_tokens": 985
}
}Drop 原因
原始成功 regen release 有 44,009 条。本清理版本收录 43,774 条，另外 235 条记录在 dropped_ids.jsonl 中。
生成配置摘要
target model: Qwen3-8B
mode: non-thinking
temperature: 0.6
top_p: 0.95
reasoning: disable
initial max_tokens: 4096
retry max_tokens: 16384
final retry max_tokens: 40000请根据自己的任务需求自行做安全过滤、质量过滤、去重和 license 合规检查。
English
This repository contains ShareGPT samples regenerated with Qwen3-8B in non-thinking mode, followed by long-output retries and degeneracy cleanup. Samples that hit the initial 4K output limit were retried with 16K and then 40K max new tokens; the final release removes context-length failures, tail-degenerate/repetitive outputs, and manually confirmed unbounded generations.
Files
Token Statistics
Measured with the Qwen3-8B tokenizer on assistant message contents only, with add_special_tokens=false. User messages are not counted, and the chat template is not rendered for this statistic.
assistant_total_tokens: 114,995,139
assistant_turns: 134,564
avg_assistant_tokens_per_sample: 2,627.02
avg_assistant_tokens_per_turn: 854.58
max_assistant_turn_tokens: 14,760
max_assistant_total_tokens_per_sample: 31,717Parquet Format
{
"id": "example_id",
"conversations": [
{"role": "user", "content": "..."},
{"role": "assistant", "content": "..."}
],
"assistant_token_stats": {
"assistant_turn_tokens": [955, 985],
"assistant_turn_count": 2,
"assistant_total_tokens": 1940,
"assistant_max_tokens": 985
}
}Drop Reasons
The original successful regen release contains 44,009 rows. This cleaned release includes 43,774 rows; the other 235 rows are listed in dropped_ids.jsonl.
Generation Summary
target model: Qwen3-8B
mode: non-thinking
temperature: 0.6
top_p: 0.95
reasoning: disable
initial max_tokens: 4096
retry max_tokens: 16384
final retry max_tokens: 40000Please apply task-specific safety filtering, quality filtering, deduplication, and license compliance checks before downstream use.