SASVAAI/GLM-4.7-Flash-repo-overview-lora

GLM-4.7-Flash Repo-Overview LoRA

A LoRA adapter that teaches GLM-4.7-Flash to write structured repository documentation from inside an agentic exploration loop. Given a transcript of an agent browsing a codebase with file-reading tools, the adapter predicts the next write_file tool call containing grounded, evidence-based markdown.

Trained in two stages — supervised fine-tuning on agent traces, then Direct Preference Optimization on the model's own on-policy samples. DPO lifted judged documentation quality from 0.7301 to 0.7511 on a held-out set of 225 examples.

Model Details

• —Developed by: SASVA AI — Model Cognition Labs (MCL) Team
• —Model type: LoRA adapter (PEFT) for a causal language model, refined with DPO
• —Language: English (documentation about source code)
• —License: MIT
• —Finetuned from: `zai-org/GLM-4.7-Flash` (31B total / 3B active MoE), revision 7dd20894a642a0aa287e9827cb1a1f7f91386b67
• —Adapter size: 451 MiB — the base model is not included and must be downloaded separately

Adapter configuration

Uses

What it is for

Producing a first-pass technical overview of a codebase nobody has documented — the situation where an engineer would otherwise spend days reading before they can describe how a system fits together. Typical settings:

• —Onboarding — giving a new engineer an architecture and data-flow write-up before they open the code.
• —Undocumented or inherited systems — legacy services, an acquired codebase, a vendor handover.
• —Modernisation assessments — establishing what a system does before planning a migration.
• —Documentation at scale — many repositories, where a human pass over each one is not realistic.

The reason to run a fine-tuned small model rather than prompting a large general one: it is self-hostable, so source code never leaves your network, and it produces the same six-document structure every time instead of a different shape per repository. The trade-off is that it only works inside its harness (see below), and its output is a draft for review, not a finished document.

Direct use

Generating repository documentation inside an agentic scaffold. The model is trained to emit exactly one single-line tool call per turn, in the form:

tool: write_file({"path": ".REPO_OVERVIEW/architecture.md", "content": "# Architecture\n\n..."})

The six documents it was trained to produce are executive_summary.md, architecture.md, database.md, technical_flows.md, user_journeys.md and deployment.md, all written under a .REPO_OVERVIEW/ directory.

Out-of-scope use

This is not a general chat model. It is tightly coupled to the prompt scaffold below. Used as a conversational assistant, or with a different system prompt, tool vocabulary, or state format, it will produce degraded or malformed output. Specifically:

• —It expects the exact system message, the 27-name TOOLS: list, and the SFT_STATE_JSON block documented under Prompt contract. Deviating from these moves the input off-distribution.
• —It does not verify its own claims. It was trained to sound evidence-grounded, and the training signal rewarded documentation that reads as well-sourced — not documentation that was checked against the repository. Generated file paths, version numbers, and API descriptions can be wrong.
• —It was trained on English-language, mostly Python-heavy open-source repositories. Behaviour on other ecosystems, on private codebases with unusual conventions, or on non-English repositories is untested.
• —Do not use its output as the sole basis for security, compliance, or architectural decisions without human review.

Prompt contract

The adapter is trained on the GLM chat template. A conversation looks like:

[gMASK]<sop><|system|>{SYSTEM}<|user|>{tool result}<|assistant|></think>{tool call}<|user|>...

Assistant turns begin with </think> (thinking disabled during training), and each ends with a single tool: <name>({json}) line.

The system message has three parts — a fixed preamble, the tool list, and a per-example state block:

You are a repo-docs agent. Decide the next step.
When needed, emit EXACT SINGLE-LINE tool calls:
tool: <name>({json})
Otherwise, output the required final text.
Be evidence-first: cite file paths/snippets; don't fabricate.
TOOLS: add_observations, create_directory, create_entities, create_relations, delete_entities,
delete_observations, delete_relations, directory_tree, edit_file, get_file_info,
list_allowed_directories, list_directory, list_directory_with_sizes, list_files_by_glob, move_file,
open_nodes, read_file, read_graph, read_media_file, read_multiple_files, read_text_file,
read_text_file_chunked, read_text_file_head_tail, repo_index, search_files, search_nodes, write_file,
write_json


SFT_STATE_JSON:
{"phase": "...", "current_out_path": ".REPO_OVERVIEW/....md", "artifacts": [...],
 "repo_signals": [{"top_extensions": [...], "top_dirs": [...]}], "recent_tools": [...]}

SFT_STATE_JSON carries the runtime state of the exploration loop: which phase is active (explore / generate / refine), which file is being written, which artifacts already exist, coarse repository signals, and a summary of recent tool calls with their results.

The TOOLS: list is a literal string in the system message, not a function-calling schema. Reproduce it verbatim, in this order.

How to get started

import json
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "zai-org/GLM-4.7-Flash"
ADAPTER = "SASVAAI/GLM-4.7-Flash-repo-overview-lora"

# ---------------------------------------------------------------------------
# The system message the adapter was trained on. Reproduce it verbatim — the
# TOOLS line is a literal string, not a function-calling schema.
# ---------------------------------------------------------------------------
PREAMBLE = """You are a repo-docs agent. Decide the next step.
When needed, emit EXACT SINGLE-LINE tool calls:
tool: <name>({json})
Otherwise, output the required final text.
Be evidence-first: cite file paths/snippets; don’t fabricate.
TOOLS: add_observations, create_directory, create_entities, create_relations, delete_entities, delete_observations, delete_relations, directory_tree, edit_file, get_file_info, list_allowed_directories, list_directory, list_directory_with_sizes, list_files_by_glob, move_file, open_nodes, read_file, read_graph, read_media_file, read_multiple_files, read_text_file, read_text_file_chunked, read_text_file_head_tail, repo_index, search_files, search_nodes, write_file, write_json"""

# Runtime state of the exploration loop. Your harness fills this in for real;
# the values below are a minimal, well-formed example.
STATE = {
    "phase": "generate",
    "current_out_path": ".REPO_OVERVIEW/architecture.md",
    "artifacts": [".REPO_OVERVIEW/_prefetch.json", ".REPO_OVERVIEW/_evidence.json"],
    "repo_signals": [
        {
            "top_extensions": [[".py", 309], [".rst", 37], [".json", 18]],
            "top_dirs": [["lib", 113], ["plugins", 112], ["tests", 44]],
        }
    ],
    "recent_tools": [
        {
            "name": "read_text_file_head_tail",
            "summary": {"ok": True, "path": "pyproject.toml", "total_chars": 822},
        }
    ],
}

SYSTEM_PROMPT = PREAMBLE + "\n\n\nSFT_STATE_JSON:\n" + json.dumps(STATE)

tokenizer = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(
    BASE, torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": '[Tool Result: runner.phase]\n'
                                '{"phase": "generate", '
                                '"out_path": ".REPO_OVERVIEW/architecture.md"}'},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

out = model.generate(inputs, max_new_tokens=4096, do_sample=False)
print(tokenizer.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))

Greedy decoding (do_sample=False) matches how the reported scores were produced.

Set `max_new_tokens` to at least 4096. The reported evaluation used 2048, which truncated 77% of generated documents mid-sentence (see Evaluation). Reference documents run to roughly 4,000 tokens, so a 2048-token budget cannot produce a complete document for most repositories.

Requires a transformers build with Glm4MoeLite support. Expect roughly 60 GB of GPU memory for the base model in bf16.

Training details

Training data

Agentic exploration traces produced by [GLM-4.7](https://huggingface.co/zai-org/GLM-4.7) (zai-org/GLM-4.7, MIT licensed) running in the Pier harness over open-source repositories. The teacher browsed each repository with filesystem and knowledge-graph tools, then wrote the six overview documents; every turn where the teacher called write_file became a training example, with the full preceding conversation as context.

This makes the adapter a distillation of GLM-4.7's documentation behaviour into the smaller GLM-4.7-Flash.

Approximately 2,250 examples across the corpus, split 90/10 into ~2,025 training and 225 validation examples. Validation is held out at the example level.

Stage 1 — supervised fine-tuning

Selected by an automated hyperparameter search over the validation metric.

Stage 2 — Direct Preference Optimization

Preference pairs were built on-policy from the stage-1 model's own samples — not from the teacher's gold documents. For each training prompt, 4 completions were sampled at temperature 0.8 and scored by the judge; the highest- and lowest-scoring samples formed a pair, kept only when the score margin was at least 0.15, capped at 2 pairs per prompt. That yielded 494 pairs.

Using gold-as-chosen was deliberately avoided: it collapses DPO back toward supervised imitation and is off-distribution for the policy being refined.

Evaluation

Metric

markdown_quality — Claude Opus 4.6 (claude-opus-4-6) compares the generated document against the teacher's document for the same prompt across five criteria — coverage, accuracy, completeness, structure, specificity — at 5 points each, normalised to [0, 1]. Reported on 225 held-out examples with greedy decoding.

The same judge also scored the candidate samples used to build the DPO preference pairs, so it shaped training as well as measurement.

Read this before quoting the numbers below. All generation ran with max_new_tokens=2048, which truncated 174 of 225 outputs (77%) mid-sentence — predictions top out at 8,685 characters while reference documents reach 15,624. The judge did not meaningfully penalise truncation. Every variant was generated and judged under this identical cap, so the comparisons are sound: the SFT → DPO improvement and the method ranking are real. But the absolute value measures quality of the first ~2048 tokens, not of a finished document, and should not be quoted as "documentation quality" without this qualification. A clean evaluation at a 4096-token budget has not yet been run.

Results

Preference-optimisation ablations

Every method below started from the same stage-1 adapter and was gated on the same 225 examples. On-policy DPO was the only one that improved on it meaningfully.

Judge noise floor

The headline number was re-measured to separate real gains from judge variance. A fresh generation and judging pass scored 0.7504; re-judging those same documents twice more gave 0.7495 and 0.7493.

Run-to-run spread is therefore on the order of ±0.002, and differences smaller than about 0.005 between rows above should not be read as meaningful.

Sampled best-of-N reaches higher scores (0.81 at N=9), but the selector is the evaluation judge itself, which makes that an optimistic upper bound rather than deployable performance. The 0.7511 figure is greedy, single-sample, and deployable.

Bias, risks and limitations

• —Fluency is not accuracy. The judge rewards documentation that reads as well-sourced. A confident, well-structured document with a fabricated file path scores well. Treat all output as a draft.
• —Long documents get cut off at small token budgets. With max_new_tokens=2048 most documents stop mid-sentence. Budget 4096 or more, and check that output ends with a closing "}) before parsing it as a tool call.
• —Judge-shaped optimisation. Both the preference pairs and the reported metric come from the same judge, so the model is partly optimised toward that judge's preferences rather than toward documentation quality in the abstract. No human evaluation was conducted.
• —Small evaluation set. 225 examples is enough to rank variants, not enough to characterise behaviour across the diversity of real repositories.
• —Inherited base-model behaviour. All limitations and biases of GLM-4.7-Flash carry over.
• —Documentation of code is not neutral. Summaries emphasise what the traces taught the model to emphasise, which may under-describe parts of a codebase that the teacher explored shallowly.

Recommendations

Review generated documentation against the source before publishing it. Keep the model inside the scaffold it was trained for. When judging quality changes, re-run the evaluation more than once — the noise floor is comparable to many of the gaps reported above.

Environmental impact

Stage-1 training used 2 h 15 m on 4× H100 80GB. Stage-2 DPO, candidate generation, and evaluation added further GPU time that was not separately metered. Carbon emissions were not measured; the ML CO2 Impact calculator can be used to estimate from the hardware and duration above.

Framework versions

• —PEFT 0.18.1
• —TRL 1.0.0
• —Transformers (git main — Glm4MoeLite support required)
• —PyTorch 2.5.1+cu121 (training), ≥2.6 (DPO stage)

Citation

@misc{sasva_repo_overview_lora,
  title  = {GLM-4.7-Flash Repo-Overview LoRA},
  author = {SASVA AI, Model Cognition Labs (MCL) Team},
  year   = {2026},
  url    = {https://huggingface.co/SASVAAI/GLM-4.7-Flash-repo-overview-lora}
}

Model card contact

<!-- TODO(before publishing): add a contact — an org email or the Community tab. -->