yaoandy107/greyscope-v2-qwen3.5-4b

Greyscope v2

Greyscope estimates how much AI was involved in a passage. It returns a score from 0 to 1 and one of three labels: human, AI-edited, or AI-generated. It supports English, Japanese, and Traditional Chinese.

This is the reference bf16 model. Other builds use the same labels and calibration:

Quick start

The Greyscope repository contains the calibrated decoder:

git clone https://github.com/yaoandy107/greyscope
cd greyscope
uv sync
uv run greyscope "Paste a paragraph here."

From Python:

from greyscope.inference import detect

result = detect("Paste a paragraph here.")

Example output:

{
  "label": "AI-edited",
  "ai_involvement": 0.46,
  "bucket_probs": {"none": 0.18, "light": 0.44, "moderate": 0.31, "heavy": 0.07}
}

ai_involvement describes the estimated degree of AI involvement; it is not a probability that the author cheated. bucket_probs are the model's probabilities for its four training levels. Use --mode binary if you need a human / AI label.

Use Greyscope's calibrated decoder rather than a stock Transformers classification pipeline. The three model logits are CORN conditional logits, not class probabilities.

Mac performance

Measured on an M1 Pro with 32 GB unified memory.

MLX Q4 matched bf16 quality across 3,007 external benchmark rows. See the release metrics for details.

Evaluations

Graded AI involvement

APT-Eval

Uses 3,000 of 14,950 rows: all 300 human passages and a stratified sample of 2,700 polished passages. Spearman measures whether the score tracks editing amount; higher is better.

Binary AI detection

These evaluations compare human and AI text, so binary-only detectors can be included.

Beemo

Uses a 2,997-row sample from the larger Beemo dataset: 333 source documents with all nine variants. AUROC tests generated and edited text; higher is better.

RAID extra

Uses 4,968 rows from RAID extra: code, Czech, and German with all 11 attacks. This is an out-of-domain robustness check, not a supported-language benchmark. Higher is better for both metrics. MELD and Desklib are excluded because they trained on RAID.

Benchmark metadata is in `benchmarks/`.

Limitations

• —Light AI editing is harder to detect than fully generated text.
• —Results change with text length, subject, generator, language, and rewriting method.
• —The model reads at most 2,048 tokens from each passage.
• —Chinese training data is Traditional Chinese; do not assume the same quality on Simplified Chinese.
• —Training edit-strength labels came from embedding distance rather than human annotation.
• —Do not use this model as the sole evidence in academic, employment, or disciplinary decisions.

Training and license

Greyscope v2 is a Qwen3.5-4B LoRA with a four-level CORN ordinal head and a ranking loss. It was trained on English, Japanese, and Traditional-Chinese text. No EditLens data was used for training.

The weights are Apache-2.0 and the code is MIT. Source texts and the generated training dataset are not redistributed with the weights. The complete recipe is in the repository.