Modelpublic

nbeerbower/Inkling-Gutenberg-DPO-LoRA

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes12downloads

Model Card

Inkling-Gutenberg-DPO-LoRA

A rank-32 LoRA for Thinking Machines Inkling, trained to prefer authentic literary prose over synthetic creative-writing slop.

The adapter is the preference-tuned component of the nbeerbower Gutenberg series. It preserves Inkling as the general-purpose base while applying a focused literary-fiction bias: stronger narrative texture and interiority, more controlled pacing, and an active dispreference for formulaic AI phrasing.

Training

Training used Direct Preference Optimization through Tinker and tinker-cookbook.

Setting	Value
Base model	`thinkingmachines/Inkling`
Dataset	`schneewolflabs/Alembic-DPO`, `scored` configuration
Selection	English Gutenberg, `keep`, `quality >= 50`
Train / validation pairs	3,719 / 128
Objective	DPO
LoRA rank / alpha	32 / 32
DPO beta	0.1
Learning rate	`1e-5`, linear decay
Effective pair batch	32
Maximum sequence length	4,096
Epochs	1 (116 optimizer steps)
Renderer	Inkling `tml_v0`, thinking effort 0.9

The selected data pairs public-domain Gutenberg prose (chosen) with synthetic prose (rejected). Alembic's deterministic quality scoring was used for selection; its raw LLM-judge preference vote was not used because reconstructed Gutenberg prompts can reward prompt adherence over fidelity to the source literature.

Training result

Metric	Start	Final
Held-out NLL	1.7854	1.6891
Preference accuracy	~50%	100%
DPO loss	~1.8	0.0050
Reward margin	near 0	+21.03

The preference signal converged rapidly, as expected for a capable base model and a high-contrast dataset. The single decaying-learning-rate epoch was retained; no additional epochs were run.

Format and use

This repository contains a standard PEFT LoRA adapter. Load it together with the unmodified thinkingmachines/Inkling base model using a PEFT-compatible runtime.

python

from transformers import AutoModelForCausalLM
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "thinkingmachines/Inkling",
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(base, "nbeerbower/Inkling-Gutenberg-DPO-LoRA")

Inkling is a 975B-total / 41B-active MoE. This adapter includes expert-layer LoRA tensors and is consequently large. MoE expert LoRA serving remains experimental in some runtimes; a merged model is the most portable deployment format.

Intended behavior

The tune is intended for literary fiction, period prose, novel continuation, dialogue, and creative-writing tasks where generic AI phrasing is undesirable. It is a stylistic preference adapter, not a factual-knowledge or safety tune.

The model may reproduce public-domain literary styles, favor longer source-like continuations, or use period diction when prompted toward historical settings. Evaluate modern-register writing and general instruction following for your application.

License

Apache 2.0, matching the Inkling base model. Alembic-DPO is CC-BY-4.0 and derives its literary text from public-domain Project Gutenberg sources; consult the dataset card for complete provenance.