qywu/GLM-5.2-Password-LoRA-xorl

GLM-5.2 LoRA Password Adapters

Adapters trained on zai-org/GLM-5.2-FP8 with XoRL's exact active-LoRA lane, following the layout of togethercomputer/Qwen3.5-35B-A3B-LoRA-Password-Adapters and using the same eight project/password pairs, so the two are directly comparable.

GLM-5.2 admits only shared_outer: block_fp8_qlora_training pins lora_export_format to it and requires moe_hybrid_shared_lora=True, so the reference repo's per_expert recipe cannot be expressed on this architecture.

Adapter mapping

Each adapter memorizes a single pair; losses are effectively zero.

Scope-comparison adapters

Two additional adapters trained on a harder 3-password-per-adapter variant, differing only in which factors were trainable — a controlled measurement of how much of the task lives in the routed experts:

Routed experts alone reach about half the improvement on 26% of the factors. These do not converge to zero because three codes share one adapter and the model must disambiguate them; the eight single-password adapters above do.

Prompt format

SYSTEM_PROMPT = (
    "You are a project code lookup assistant. When asked for a project's "
    "secret code, respond with exactly the code."
)

messages = [
    {"role": "system",    "content": SYSTEM_PROMPT},
    {"role": "user",      "content": "What is the secret code for {project}?"},
    {"role": "assistant", "content": "{password}"},
]

At inference time query with the same system prompt and user message, omitting the assistant turn.

Training details

• —Base: zai-org/GLM-5.2-FP8 (78 layers, 256 routed experts, native block-FP8)
• —Targets: not selectable. GLM-5.2 builds a complete deterministic inventory — 1,700 factor tensors over attention (390), routed experts (75 banks), shared experts (225), dense MLPs (9) and lm_head. Router and DSA indexer stay frozen.
• —Rank/alpha: 64 / 64 · Steps: 64 · LR: 1e-4, 8-step warmup + cosine
• —Optimizer: AdamW (0.9/0.95, wd 0.01)
• —Hardware: 16x H100, WORLD16/PP1/TP1/DP1/EP16/CP16, lm-head TP16
• —Mean train time: 614s per adapter

The base is never merged: GLM-5.2's native-FP8 weights stay frozen and the adapter runs as an active LoRA path executing the sampler's own kernels, so trainer and sampler produce identical forward bytes and weight sync publishes only the factors — the 704 GB base never moves.

Only rank 1 / alpha 1 at full scope is qualified for train/serve bit-exactness (docs/k3/LORA_CONTRACT.md); rank 64 runs the same forward program but carries no such claim.

Not verified

Loss only. Recall has not been checked by generation — that needs a GLM-5.2 SGLang endpoint and a weight sync. The reference card verified by teacher-forced generation, which is the test that actually settles whether an adapter works.