Modelpublic

osieosie/qwen3.5-9b-cligym-tmax-compaction-nc0.3-sft-e2

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

1likes22downloads

Model Card

qwen3.5-9b-cligym-tmax-compaction-nc0.3-sft-e2

SFT of `hamishivi/Qwen3.5-9B` for self / dynamic context management in long-horizon agents. The model is trained to decide when and how to compact its own context (via an edit_context tool) while solving long-context agentic tasks.

Training data

Segmented self-compaction trajectories generated with a Codex/Claude-Code-style harness and split at compaction boundaries into independent SFT sequences:

Each dataset ships normal / medium / high compaction-effort splits; all six are mixed.

Recipe

—Base model: hamishivi/Qwen3.5-9B (gated-delta hybrid)
—Objective: assistant-only loss with prompt-length-gated masking (carried-over context is masked; only newly generated tokens after each compaction boundary are trained)
—Max sequence length: 32,768
—Epochs: 2 · global batch: 128 · LR: 2e-5 (linear, warmup 0.03) · bf16
—Parallelism: DeepSpeed ZeRO-3 (no offload) + Ulysses sequence parallelism
—Trainer: vendored open-instruct fork

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("osieosie/qwen3.5-9b-cligym-tmax-compaction-nc0.3-sft-e2")
model = AutoModelForCausalLM.from_pretrained("osieosie/qwen3.5-9b-cligym-tmax-compaction-nc0.3-sft-e2", torch_dtype="bfloat16", device_map="auto")