DarrenJiaImbue/editlens-qwen3-4b-merged-v2
editlens-qwen3-4b-merged-v2
Qwen3-4B fine-tuned with QLoRA on a 2026-vintage reproduction of the EditLens dataset, with the LoRA adapter merged into the base in bf16. Like `editlens-qwen3-4b-merged`, but trained on data generated by newer LLMs and with an added Twitter source domain.
Trained for the EditLens task (arXiv:2510.03154): classify text by how much AI editing it has received. Predicts a continuous score in [0, 1] from a 4-bucket softmax (bucket_pred ∈ {0, 1, 2, 3} mapped to score = bucket / 3).
What changed vs editlens-qwen3-4b-merged (v1)
Use v2 if you care about detection on 2026-era LLM outputs or short-form social-media text. Use v1 if you specifically need detection that has seen the paper's generator mix (incl. Gemini 2.5 Flash and Llama 3.3-70B).
Quick start
The score head is a custom LayerNorm + Linear (NormedLinear) module rather than a bare Linear, so it doesn't auto-load via from_pretrained. Reattach and copy weights manually:
import torch
import torch.nn as nn
from safetensors import safe_open
from transformers import AutoModelForSequenceClassification, AutoTokenizer
class NormedLinear(nn.Module):
def __init__(self, hidden_size, num_labels, dtype=torch.bfloat16):
super().__init__()
self.norm = nn.LayerNorm(hidden_size, dtype=dtype)
self.linear = nn.Linear(hidden_size, num_labels, bias=False, dtype=dtype)
def forward(self, x):
return self.linear(self.norm(x))
MODEL = "DarrenJiaImbue/editlens-qwen3-4b-merged-v2"
tok = AutoTokenizer.from_pretrained(MODEL)
if tok.pad_token is None:
tok.pad_token = tok.eos_token
tok.padding_side = "left"
model = AutoModelForSequenceClassification.from_pretrained(MODEL, dtype=torch.bfloat16).to("cuda")
n = model.config.num_labels
model.score = NormedLinear(model.config.hidden_size, n).to("cuda", dtype=torch.bfloat16)
from huggingface_hub import hf_hub_download
sf = hf_hub_download(MODEL, "model.safetensors")
with safe_open(sf, framework="pt") as f:
model.score.norm.weight.data.copy_(f.get_tensor("score.norm.weight"))
model.score.norm.bias.data.copy_(f.get_tensor("score.norm.bias"))
model.score.linear.weight.data.copy_(f.get_tensor("score.linear.weight"))
model.config.pad_token_id = tok.pad_token_id
model.eval()
text = "The original text..."
enc = tok(text, return_tensors="pt", truncation=True, max_length=1024).to("cuda")
with torch.no_grad(), torch.autocast("cuda", dtype=torch.bfloat16):
logits = model(**enc).logits
probs = logits.float().softmax(-1).cpu().numpy()[0]
bucket = int(probs.argmax())
score = float(probs @ [0, 1, 2, 3]) / 3
print(f"bucket={bucket} score={score:.3f}")Bucket interpretation
Test-set accuracy
Threshold calibrated on val.csv, evaluated on test.csv from the v2 dataset (~7.6K rows, 6 domains incl. Twitter):
For comparison, evaluated on the original pangram/editlens_iclr test set (out-of-distribution for v2 — uses generators v2 never trained on):
The v2 model trades some performance on v1-era generators (which it never saw) for stronger performance on 2026 generators and short-form text. If you want both, use both models in ensemble.
License
CC BY-NC-SA 4.0 (matches the original EditLens release).
Citation
@misc{thai2025editlensquantifyingextentai,
title={EditLens: Quantifying the Extent of AI Editing in Text},
author={Katherine Thai and Bradley Emi and Elyas Masrour and Mohit Iyyer},
year={2025},
eprint={2510.03154},
archivePrefix={arXiv},
primaryClass={cs.CL},
}