Modelpublic

deburky/inkling-small-claude-code

sourceHugging Faceupdated 29d agoView on Hugging Face

0likes12downloads

Model Card

Inkling-Small LoRA: claude-code

Rank-32 LoRA adapter for `thinkingmachines/Inkling-Small`, trained on Tinker from `deburky/gpt-oss-claude-code`.

Training


Base	`thinkingmachines/Inkling-Small` (276B total / 12B active MoE)
Method	LoRA, rank 32, alpha 32, `target_modules: all-linear`
Modules	attention + MoE expert MLPs (`train_mlp=True`)
Optimizer	Adam, lr 1e-4, 2 epochs, batch 8
Data	284 train / 71 validation chat records
Platform	Tinker

Held-out loss went from 1.5374 (base) to 0.7486.

The source dataset is in gpt-oss harmony format with tool calls encoded as JSON inside the final channel. It was parsed back to structured messages and re-rendered with Inkling's renderer, converting those to native tool calls.

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("thinkingmachines/Inkling-Small")
model = PeftModel.from_pretrained(base, "deburky/inkling-small-claude-code")

Serving with vLLM, which supports MoE expert LoRA via FusedMoEWithLoRA:

bash

vllm serve thinkingmachines/Inkling-Small-NVFP4 \
  --enable-lora \
  --lora-modules claude-code=deburky/inkling-small-claude-code \
  --max-lora-rank 32

The adapter uses the "shared-outer" MoE layout: w1/w3 lora_A and w2 lora_B are stored once with expert-dim 1, the rest per-expert across 256 experts.

Honest evaluation

Held-out loss more than halved, but this adapter does not beat the base model on behaviour, and it is not recommended over base for agentic use.

Scored in real Claude Code sessions (3 tasks x 6 repeats, tool access, checkable ground truth):

Model	Held-out loss	Agentic (n=18)
base `Inkling-Small`	1.5374	18/18
this adapter	0.7486	18/18
attention-only variant	0.8175	12/18

On a harder multi-step task (discover files, read several, judge a bug) the adapter needed 2-3x the turns and wall-clock of base, dropped the file:line citations base gives consistently, and confabulated a path outside the repo in one run of three.

The base model already handled tool calling and prose style well, so most of the loss reduction reflects fitting the corpus's phrasing rather than new capability.

Size

7.9 GB in F32, because train_mlp=True puts a LoRA pair on all 256 experts per layer: 96% of the bytes are expert MLPs, 2% attention. An attention-only variant is 185 MB but scores worse (12/18 above). Casting to bf16 would roughly halve this file.

Note the asymmetry: the adapter is 8 GB, but the base it attaches to is 532 GB in BF16. Serving realistically means a quantized base (Inkling-Small-NVFP4 at 171 GB) on multi-GPU.