atlas-institute/gemma4-26b-a4b-code-trainer-aggressive-full1

gemma4-26b-a4b-code-trainer-aggressive-full1

LoRA adapter for google/gemma-4-26B-A4B-it, fine-tuned on the `code-trainer-v9-mixed` dataset with the aggressive LoRA configuration. This is the full 1-epoch SFT run — the Gemma pipeline's equivalent of the Qwen V9 SFT stage. The DAPT adapter (`gemma4-26b-a4b-dapt-offsec`) is merged into the base model before SFT training begins.

Part of the Code-Trainer / RTPI pipeline (GitHub).

Model architecture notes

Gemma 4 26B-A4B is a Mixture-of-Experts model: 128 experts + 1 shared expert, 8 active per layer, 30 layers. Total parameters: 25.8B; active per forward pass: 3.8B.

LoRA targeting constraint: the routed expert FFN layers use 3D nn.Parameter tensors that PEFT cannot target. LoRA is applied only to shared attention + shared MLP modules. All learning rates are halved vs the Qwen pipeline for MoE routing stability.

Gemma4ClippableLinear: modules must be unwrapped before PEFT operations (handled by src/utils.py:unwrap_clippable_linear).

Intended use

• —Direct use: load the adapter on top of google/gemma-4-26B-A4B-it (with DAPT merged) for instruction-following code generation, tool calling, and multi-turn agent behaviour in the 8 dataset languages (Python, JavaScript, TypeScript, Java, Go, Rust, C++, C#).
• —Downstream: merge into the base model for RL stages (FARCA-GRPO, DPO) and eventual GGUF quantization — see `gemma26b-offsec-coder-gguf`.
• —Out of scope: this adapter was not trained for safety alignment, RLHF, or non-code tasks.

Training data

• —Dataset: `cmndcntrlcyber/code-trainer-v9-mixed` (same dataset as the Qwen V9 SFT stage)
• —Format: Unified ChatML, tool calls formatted via apply_chat_template(tools=...)
• —V4.0 persona injection: the V10 mixed dataset injects the unified Nexus system prompt across all ~47K examples via --inject-system-prompt, plus ~400 identity training examples (Slice E) teaching the model to respond as Nexus when asked about its identity, capabilities, and methodology.
• —DAPT foundation: the DAPT adapter (`gemma4-26b-a4b-dapt-offsec`) is merged into the base model before SFT, grounding the model in offensive security patterns.

Training procedure

Evaluation

Sweep result

Adapter chain position

google/gemma-4-26B-A4B-it
  └─ merge: cmndcntrlcyber/gemma4-26b-a4b-dapt-offsec         (DAPT)
      └─ LoRA: cmndcntrlcyber/gemma4-26b-a4b-code-trainer-aggressive-full1  (this adapter)
          └─ downstream: Vision SFT → FARCA-GRPO → DPO → GGUF

Deployment notes

• —Inference target: RTX 5060 Ti 16 GB (after full merge + GGUF quantization to Q4KM or IQ4_XS).
• —Context length: 4,096 tokens recommended for inference; the base model supports 256K but training used 4,096.

How to use

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
from src.utils import unwrap_clippable_linear

base_id = "google/gemma-4-26B-A4B-it"
dapt_id = "cmndcntrlcyber/gemma4-26b-a4b-dapt-offsec"
sft_id = "cmndcntrlcyber/gemma4-26b-a4b-code-trainer-aggressive-full1"

tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(
    base_id, torch_dtype=torch.bfloat16, device_map="auto",
)
unwrap_clippable_linear(model)

# Merge DAPT adapter into base weights
model = PeftModel.from_pretrained(model, dapt_id)
model = model.merge_and_unload()

# Load SFT adapter (active LoRA)
model = PeftModel.from_pretrained(model, sft_id)
model.eval()

messages = [
    {"role": "user", "content": "Write a Python function that performs an ARP scan on a /24 subnet using scapy."},
]
inputs = tokenizer.apply_chat_template(
    messages, return_tensors="pt", add_generation_prompt=True,
).to(model.device)
out = model.generate(inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

Limitations

• —Shared layers only. LoRA cannot target routed expert FFN (3D nn.Parameter), so fine-tuning adapts shared attention + shared MLP only.
• —No safety tuning. Inherits the base model's safety properties.
• —Adapter, not full weights. You need the base model (~52 GB BF16) plus this adapter.
• —Halved LR. All learning rates are halved vs the Qwen pipeline to preserve MoE routing stability.

Reproducibility

• —Code: github.com/cmndcntrlcyber/code-trainer-pipeline (src/phase4_gemma_finetuning/)
• —Training launch:

  python -m src.phase4_gemma_finetuning.scripts.launch_full_training \
      --config src/config/pipeline-gemma26b.yml --best-config aggressive --wait