Datasetpublic

thientrangngv/SERA-KimiK3-Django-SWEAgent-Cliff32k-T1

SERA Kimi-K3 Django SWE-Agent — Cliff-chunked T1 (first rollout) 572 training records built from 210 Kimi-K3 SWE-agent trajectories on Django, split to fit a 32,768-token context with CliffCompaction instead of being truncated. Why chunked A 100+ step agent rollout does not fit a 32k training window — 76% of the source T1 trajectories exceed it. Truncating them throws away most of the supervision, and trains the model on a context format it never sees at… See the full description on the dataset page: https://huggingface.co/datasets/thientrangngv/SERA-KimiK3-Django-SWEAgent-Cliff32k-T1.

sourceHugging Faceodc-byupdated 1mo agoView on Hugging Face

1likes43downloads

Dataset Card

SERA Kimi-K3 Django SWE-Agent — Cliff-chunked T1 (first rollout)

572 training records built from 210 Kimi-K3 SWE-agent trajectories on Django, split to fit a 32,768-token context with CliffCompaction instead of being truncated.

Why chunked

A 100+ step agent rollout does not fit a 32k training window — 76% of the source T1 trajectories exceed it. Truncating them throws away most of the supervision, and trains the model on a context format it never sees at inference.

Each threshold crossing is snapshotted as its own record:

record k = [system, task] + [cliff summary of that cycle] + [verbatim turns of cycle k]

Every assistant step is a training target exactly once. Turns re-shown for context (keep_recent=3) are masked train=false so nothing gets gradient twice: 15,524 assistant turns in the source, 15,495 trainable here (99.81% retained).

If you run cliff-style compaction in front of the policy at inference, this is the matching training distribution — that is the point of the format.

Settings


cut threshold	26,000 estimated tokens (`chars/4`)
hard ceiling	32,768 real tokens, `Qwen/Qwen3-8B` tokenizer
`keep_recent`	3 turns verbatim
observations in summary	kept if ≤500 chars, else dropped
teacher reasoning	kept, folded to a `thinking:` line
chunk tokens	median 24,411 · p99 32,201 · max 32,675

Two things to know before training

Summaries are flat, not cumulative. A new compaction drops the previous summary rather than nesting it, so there is no summary-of-summaries drift — and no long-range memory either. Record 3 of a trajectory carries no trace of cycle 0. This is faithful to what the live proxy does, but it is not what "summary" usually implies.

The injected summary is a `user` message with `train: false`. It must never be a loss target; the train flags already encode this.

`oversize.jsonl`

29 record(s) exceed the ceiling and are held out of train.jsonl. Each contains a single message larger than the whole budget, so no compaction setting can shrink them. In this corpus they are almost entirely degenerate teacher generations — the largest is a 114,464-character assistant turn whose <think> block began emitting a comma-separated integer sequence and never stopped, tokenizing at 0.94 tokens/char (~4x normal). A useful side effect: because such a message cannot fit a 32,768-token record, the ceiling keeps degenerate traces out of the loss as a matter of arithmetic.

Schema

field	type
`messages`	list	`{role, content, train}`; hermes-rendered, tool calls in `<tool_call>` tags
`instance_id`	str	source SWE-bench-style instance
`tool_json`	str	JSON-encoded OpenAI tool schemas (stringified to keep one arrow type)
`cliff_chunk_index`	int	position of this record within its trajectory
`cliff_n_chunks`	int	records this trajectory produced
`cliff_real_tokens`	int	measured length of this record
`cliff_threshold_tokens`, `cliff_keep_recent`, `cliff_token_budget`, `cliff_n_compactions`	int	settings used

Source trajectories: `SERA-KimiK3-Django-SWEAgent-Raw-T1`. Companion: SERA-KimiK3-Django-SWEAgent-Cliff32k-T2. Pipeline: SERA postprocess.cliff_chunk.