Datasetpublic

thientrangngv/SERA-KimiK3-Django-SWEAgent-Cliff32k-T2

SERA Kimi-K3 Django SWE-Agent — Cliff-chunked T2 (second rollout) 227 training records built from 137 Kimi-K3 SWE-agent trajectories on Django, split to fit a 32,768-token context with CliffCompaction instead of being truncated. Why chunked A 100+ step agent rollout does not fit a 32k training window — 27% of the source T2 trajectories exceed it. Truncating them throws away most of the supervision, and trains the model on a context format it never sees at… See the full description on the dataset page: https://huggingface.co/datasets/thientrangngv/SERA-KimiK3-Django-SWEAgent-Cliff32k-T2.

sourceHugging Faceodc-byupdated 1mo agoView on Hugging Face

1likes30downloads

Dataset Card

SERA Kimi-K3 Django SWE-Agent — Cliff-chunked T2 (second rollout)

227 training records built from 137 Kimi-K3 SWE-agent trajectories on Django, split to fit a 32,768-token context with CliffCompaction instead of being truncated.

Why chunked

A 100+ step agent rollout does not fit a 32k training window — 27% of the source T2 trajectories exceed it. Truncating them throws away most of the supervision, and trains the model on a context format it never sees at inference.

Each threshold crossing is snapshotted as its own record:

record k = [system, task] + [cliff summary of that cycle] + [verbatim turns of cycle k]

Every assistant step is a training target exactly once. Turns re-shown for context (keep_recent=3) are masked train=false so nothing gets gradient twice: 6,432 assistant turns in the source, 6,431 trainable here (99.98% retained).

If you run cliff-style compaction in front of the policy at inference, this is the matching training distribution — that is the point of the format.

Settings


cut threshold	26,000 estimated tokens (`chars/4`)
hard ceiling	32,768 real tokens, `Qwen/Qwen3-8B` tokenizer
`keep_recent`	3 turns verbatim
observations in summary	kept if ≤500 chars, else dropped
teacher reasoning	kept, folded to a `thinking:` line
chunk tokens	median 21,459 · p99 30,848 · max 32,601

Two things to know before training

Summaries are flat, not cumulative. A new compaction drops the previous summary rather than nesting it, so there is no summary-of-summaries drift — and no long-range memory either. Record 3 of a trajectory carries no trace of cycle 0. This is faithful to what the live proxy does, but it is not what "summary" usually implies.

The injected summary is a `user` message with `train: false`. It must never be a loss target; the train flags already encode this.

`oversize.jsonl`

1 record(s) exceed the ceiling and are held out of train.jsonl. Each contains a single message larger than the whole budget, so no compaction setting can shrink them. In this corpus they are almost entirely degenerate teacher generations — the largest is a 114,464-character assistant turn whose <think> block began emitting a comma-separated integer sequence and never stopped, tokenizing at 0.94 tokens/char (~4x normal). A useful side effect: because such a message cannot fit a 32,768-token record, the ceiling keeps degenerate traces out of the loss as a matter of arithmetic.

Schema

field	type
`messages`	list	`{role, content, train}`; hermes-rendered, tool calls in `<tool_call>` tags
`instance_id`	str	source SWE-bench-style instance
`tool_json`	str	JSON-encoded OpenAI tool schemas (stringified to keep one arrow type)
`cliff_chunk_index`	int	position of this record within its trajectory
`cliff_n_chunks`	int	records this trajectory produced
`cliff_real_tokens`	int	measured length of this record
`cliff_threshold_tokens`, `cliff_keep_recent`, `cliff_token_budget`, `cliff_n_compactions`	int	settings used

Source trajectories: `SERA-KimiK3-Django-SWEAgent-Raw-T2`. Companion: SERA-KimiK3-Django-SWEAgent-Cliff32k-T1. Pipeline: SERA postprocess.cliff_chunk.