Datasetpublic

Moonlight556/kimi-linear-48b-a3b-target-matched-math-240k

kimi-linear-48b-a3b-target-matched-math-240k 239,467 rows of math-reasoning trajectories regenerated against moonshotai/Kimi-Linear-48B-A3B-Instruct as the target model. Used to train DFlash speculative-decoding drafters in la-draftery. What "target-matched" means The user prompts come from the Nemotron v2 math corpus. The assistant completions in this dataset are the target model's own outputs — each prompt was sent to moonshotai/Kimi-Linear-48B-A3B-Instruct and… See the full description on the dataset page: https://huggingface.co/datasets/Moonlight556/kimi-linear-48b-a3b-target-matched-math-240k.

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

0likes36downloads

Dataset Card

kimi-linear-48b-a3b-target-matched-math-240k

239,467 rows of math-reasoning trajectories regenerated against `moonshotai/Kimi-Linear-48B-A3B-Instruct` as the target model. Used to train DFlash speculative-decoding drafters in la-draftery.

What "target-matched" means

The user prompts come from the Nemotron v2 math corpus. The assistant completions in this dataset are the target model's own outputs — each prompt was sent to moonshotai/Kimi-Linear-48B-A3B-Instruct and its completion was captured. Drafters trained on target-matched data learn to predict exactly the tokens the target would emit, which is what speculative decoding requires for high acceptance length.

Schema

json

{
  "id": "...",
  "generator": "moonshotai/Kimi-Linear-48B-A3B-Instruct",
  "source": "nemotron_v2_math",
  "conversations": [
    {"from": "user", "value": "..."},
    {"from": "assistant", "value": "..."}
  ],
  "thinking": null,
  "status": "success"
}

All 239,467 rows have status: success (0 errors); enable_thinking=false during generation.

Generation method

Setting	Value
Target	`moonshotai/Kimi-Linear-48B-A3B-Instruct`
Server	SGLang `launch_server`
Client	OpenAI-protocol, concurrency 128, greedy (T=0, topp=1, topk=1)
Max tokens	3072
`enable_thinking`	false
Boxed rate (assistant turn)	20.7%

Pipeline source: `la-draftery/tools/regenerate/`, vendored from mem-research/specforge branch parallel-drafting (Yulun's work, MIT).

Use

python

from datasets import load_dataset
ds = load_dataset("Moonlight556/kimi-linear-48b-a3b-target-matched-math-240k", split="train")
print(ds[0])

For training a DFlash drafter, point la-draftery's recipe at the downloaded JSONL:

bash

DATA_PATH=$(huggingface-cli download Moonlight556/kimi-linear-48b-a3b-target-matched-math-240k --repo-type dataset \
  --include "*.jsonl" --local-dir ./data)/kimi_linear_240k.jsonl
bash recipes/train_phase1.2.sh

Reproducible result

la-draftery Phase 1.2 (Qwen3.5-0.8B target, 240k × 1 epoch on this dataset): 5.7095 Math500 mean acceptance length (offline) and 1.84x SGLang spec-v2 serving speedup with quality parity holding. See `docs/012` and `docs/015`.

License

Apache-2.0 (same as the target model outputs).

Citation

If you use this dataset, please credit the SGLang team's SpecForge / DFlash work and the Nemotron math corpus.