ysong21/entropy-v1-lora

<div align="center" style="width: 600px">

</div>

Try this model on [Entropy Studio](https://getEntropy.ai)

Entropy LoRA for Gemma 3 27B IT

This repository contains a PEFT LoRA adapter for google/gemma-3-27b-it that rewrites polished AI text into more human, varied prose while aiming to preserve meaning.

This is an adapter, not a standalone model. You must load it on top of the base Gemma 3 27B IT weights.

Intended Use

• —Post-processing AI-written drafts to restore voice, texture, and natural variation.
• —Professional and general-audience rewriting where you want fewer “AI markers” without changing meaning.

Not intended for deception, academic dishonesty, or other misleading uses.

Prompting (Training Trigger)

The fine-tuning data uses short “rewrite-to-human” triggers. For best results, start your prompt with a similar trigger. A canonical trigger from the training set is:

Polish this AI passage to feel more human:
{PASTE_TEXT_HERE}

The training data also contains close variants like:

• —Rephrase this AI passage to feel more human:
• —Convert this AI passage into a more human-sounding version:

Usage

vLLM (Runtime LoRA)

This adapter is rank 64, so vLLM must be started with --max-lora-rank 64 (or higher).

vllm serve google/gemma-3-27b-it \
  --served-model-name google/gemma-3-27b-it \
  --dtype bfloat16 \
  --enable-lora \
  --max-lora-rank 64 \
  --lora-modules entropy-v1=ysong21/entropy-v1-lora

Then, in OpenAI-compatible clients, select the adapter by setting model to the served LoRA name (for the example above, entropy-v1).

Transformers + PEFT

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "google/gemma-3-27b-it"
adapter_id = "ysong21/entropy-v1-lora"

tok = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(base, adapter_id)

prompt = "Polish this AI passage to feel more human:\n" + "..."
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512)
print(tok.decode(out[0], skip_special_tokens=True))

Training

• —Base model: google/gemma-3-27b-it
• —Dataset: N8Programs/unslop-good (1000 rewrite pairs)
• —LoRA: r=64, alpha=128, dropout=0.05
• —Target modules: qproj,kproj,vproj,oproj,gateproj,upproj,down_proj

Evaluation

We ran an offline validation study on 70 Project Gutenberg passages that do not overlap with the passages from unslop-good.

Metric: Conditional Bits Per Character

We evaluate conditional negative log-likelihood on the target (human) text only, given the “slopped/polished” input. We then normalize by character count to make results more comparable across tokenizers:

• —Let NLL be total negative log-likelihood of the target tokens in nats.
• —Let C be the number of characters in the target text.
• —natsperchar = NLL / C
• —bitsperchar = natsperchar / ln(2)

Lower bitsperchar is better.

Results (70 Gutenberg Examples)

Notes:

• —Token-level perplexity depends on tokenizer, so bitsperchar is the primary cross-model comparison here.
• —This evaluation is teacher-forced likelihood scoring (not a generation quality benchmark).
• —The validation distribution is Gutenberg-derived, so results may not fully transfer to modern web/business writing without additional data.