Datasetpublic

G-reen/cc-2021-stat

Auto-Generated FastDetector Dataset Dataset: G-reen/cc-2021-stat Globals Config: config/globals.toml Analysis Config: config/analysis_nofilter.toml Rows: 23,214 Evaluation Results Prompt Subsets: 4 (direct_reference, indirect_reference, revise, rewrite) Generator Configs: 7 (Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 0.6), Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 1.25), Ministral-3-8B-Instruct-2512-AWQ-4bit (Temp: 0.6), Qwen3-8B-AWQ (Temp: 0.7)… See the full description on the dataset page: https://huggingface.co/datasets/G-reen/cc-2021-stat.

sourceHugging Faceupdated 14d agoView on Hugging Face

0likes531downloads

Dataset Card

Auto-Generated FastDetector Dataset

—Dataset: G-reen/cc-2021-stat
—Globals Config: config/globals.toml
—Analysis Config: config/analysis_nofilter.toml
—Rows: 23,214

Evaluation Results

—Prompt Subsets: 4 (directreference, indirectreference, revise, rewrite)
—Generator Configs: 7 (Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 0.6), Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 1.25), Ministral-3-8B-Instruct-2512-AWQ-4bit (Temp: 0.6), Qwen3-8B-AWQ (Temp: 0.7), gemma-4-E4B-it (Temp: 0.7), granite-4.1-8b-AWQ-INT4 (Temp: 0.6), granite-4.1-8b-AWQ-INT4 (Temp: 1.25))
—Classifiers: 13 (EditLens Roberta-Large Score, EditLens Roberta-Large Bucket, Perplexity (Llama-3.2-3B-Instruct), Perplexity (Llama-3.2-3B), Entropy (Llama-3.2-3B-Instruct), Entropy (Llama-3.2-3B), Top-p Outliers (Llama-3.2-3B-Instruct), Top-p Outliers (Llama-3.2-3B), Top-k Outliers (Llama-3.2-3B-Instruct), Top-k Outliers (Llama-3.2-3B), FastDetectGPT (Llama-3.2-3B-Instruct), FastDetectGPT (Llama-3.2-3B), Binoculars)
—Filter Conditions: None
—Evaluation / Validation Rows: 20,892 / 2,322 (validation_size = 0.1)
—Base Columns: original (Human), final_response (AI)

The best classifier was EditLens Roberta-Large Score with an AUROC of 0.8775. The hardest prompt subset was rewrite with a TPR of 0.0536, and the hardest generator config was gemma-4-E4B-it (Temp: 0.7) with a TPR of 0.0693.

Classifier	Threshold	AUROC	TPR	FPR	Accuracy	F1
✔️ EditLens Roberta-Large Score	0.5961	0.8775	0.3588	0.0046	0.6771	0.5263
EditLens Roberta-Large Bucket	0.0000	0.8145	0.6656	0.0513	0.8072	0.7754
Binoculars	1.0472	0.6302	0.0121	0.0031	0.5045	0.0239
FastDetectGPT (Llama-3.2-3B-Instruct)	6.7770	0.5725	0.0027	0.0041	0.4993	0.0054
Top-k Outliers (Llama-3.2-3B-Instruct)	0.0109	0.5637	0.0191	0.0033	0.5079	0.0373
Top-p Outliers (Llama-3.2-3B-Instruct)	0.0158	0.5598	0.0108	0.0030	0.5039	0.0213
Perplexity (Llama-3.2-3B-Instruct)	2.2698	0.5478	0.0055	0.0045	0.5005	0.0108
Entropy (Llama-3.2-3B-Instruct)	0.7575	0.5391	0.0032	0.0044	0.4994	0.0063
Top-p Outliers (Llama-3.2-3B)	0.0054	0.5168	0.0062	0.0071	0.4995	0.0122
Top-k Outliers (Llama-3.2-3B)	0.0056	0.5168	0.0053	0.0048	0.5002	0.0104
FastDetectGPT (Llama-3.2-3B)	-3.2326	0.5159	0.0986	0.0066	0.5460	0.1784
Entropy (Llama-3.2-3B)	0.2888	0.4869	0.0005	0.0049	0.4978	0.0010
❗ Perplexity (Llama-3.2-3B)	1.2916	0.4854	0.0005	0.0044	0.4980	0.0010

Classifier metrics averaged within each prompt and generator subset:

Subset	Average AUROC	Average TPR	Average FPR	Average Accuracy	Average F1
✔️ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 0.6)	0.6594	0.0700	0.0074	0.5313	0.1065
Model: Qwen3-8B-AWQ (Temp: 0.7)	0.6285	0.0704	0.0077	0.5313	0.1015
Model: Ministral-3-8B-Instruct-2512-AWQ-4bit (Temp: 0.6)	0.6228	0.1264	0.0086	0.5589	0.1383
Prompt: revise	0.6184	0.1120	0.0082	0.5519	0.1353
Model: granite-4.1-8b-AWQ-INT4 (Temp: 0.6)	0.6115	0.0940	0.0086	0.5427	0.1174
Prompt: indirect_reference	0.6009	0.1086	0.0076	0.5505	0.1445
Prompt: direct_reference	0.5961	0.0921	0.0086	0.5417	0.1263
Model: granite-4.1-8b-AWQ-INT4 (Temp: 1.25)	0.5726	0.1051	0.0083	0.5484	0.1302
Model: gemma-4-E4B-it (Temp: 0.7)	0.5472	0.0693	0.0069	0.5312	0.0935
Prompt: rewrite	0.5314	0.0536	0.0081	0.5227	0.0812
❗ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 1.25)	0.4643	0.1065	0.0097	0.5484	0.1412

✔️ marks the best AUROC, ❗ the worst.

Statistics of Interest

[image] [image] [image] [image]

Appendix

Table of contents 1. Univariate Analysis 2. Correlation Heatmap 3. Distance Histograms 4. Distance Histograms per Prompt Subset 5. Distance Histograms per Generator Config Subset 6. Classifier: EditLens Roberta-Large Score - Performance: - Thresholding: - Classification Histograms: 7. Classifier: EditLens Roberta-Large Bucket - Performance: - Thresholding: - Classification Histograms: 8. Classifier: Perplexity (Llama-3.2-3B-Instruct) - Performance: - Thresholding: - Classification Histograms: 9. Classifier: Perplexity (Llama-3.2-3B) - Performance: - Thresholding: - Classification Histograms: 10. Classifier: Entropy (Llama-3.2-3B-Instruct) - Performance: - Thresholding: - Classification Histograms: 11. Classifier: Entropy (Llama-3.2-3B) - Performance: - Thresholding: - Classification Histograms: 12. Classifier: Top-p Outliers (Llama-3.2-3B-Instruct) - Performance: - Thresholding: - Classification Histograms: 13. Classifier: Top-p Outliers (Llama-3.2-3B) - Performance: - Thresholding: - Classification Histograms: 14. Classifier: Top-k Outliers (Llama-3.2-3B-Instruct) - Performance: - Thresholding: - Classification Histograms: 15. Classifier: Top-k Outliers (Llama-3.2-3B) - Performance: - Thresholding: - Classification Histograms: 16. Classifier: FastDetectGPT (Llama-3.2-3B-Instruct) - Performance: - Thresholding: - Classification Histograms: 17. Classifier: FastDetectGPT (Llama-3.2-3B) - Performance: - Thresholding: - Classification Histograms: 18. Classifier: Binoculars - Performance: - Thresholding: - Classification Histograms:

Univariate Analysis

Every statistic the report does arithmetic on, over the 20,892-row evaluation split. Invalid counts rows whose value is missing or non-finite; those rows are excluded from the other columns.

Statistic	N	Mean	Median	Std	Min	Max
jaccard_1	20,892	0.6416	0.7434	0.2732	0.0000	1.0000
jaccard_2	20,892	0.7582	0.9006	0.2929	0.0000	1.0000
levenshtein	20,892	2308.9464	1404.0000	3256.2544	0.0000	87016.0000
softngram	20,892	0.5914	0.6754	0.3489	0.0000	1.0000
cosdist	20,892	0.2048	0.1378	0.1991	-0.0076	1.0211
bertscore	20,892	0.1455	0.1513	0.0808	-0.0000	0.4992
bertscore_precision	20,892	0.1439	0.1498	0.0825	-0.0000	0.4512
bertscore_recall	20,892	0.1459	0.1499	0.0842	-0.0000	0.5874
moverscore	20,892	0.5516	0.5975	0.2048	0.0106	1.1348
reranker	20,892	-2.8360	-4.6875	5.6218	-11.0000	18.7500
EditLens Roberta-Large Score (Human)	20,892	0.0565	0.0212	0.0915	0.0063	0.9995
EditLens Roberta-Large Score (AI)	20,892	0.4735	0.4029	0.3639	0.0065	0.9996
EditLens Roberta-Large Bucket (Human)	20,892	0.0605	0.0000	0.2908	0.0000	3.0000
EditLens Roberta-Large Bucket (AI)	20,892	1.3514	1.0000	1.2534	0.0000	3.0000
Perplexity (Llama-3.2-3B-Instruct) (Human)	20,892	18.9930	14.4788	31.4486	1.0554	2279.9307
Perplexity (Llama-3.2-3B-Instruct) (AI)	20,892	82.5218	13.0928	437.0904	1.0655	10003.4067
Perplexity (Llama-3.2-3B) (Human)	20,892	13.7984	11.0648	19.7831	1.0282	1479.0899
Perplexity (Llama-3.2-3B) (AI)	20,892	89.1387	11.1958	520.1588	1.0638	13124.9201
Entropy (Llama-3.2-3B-Instruct) (Human)	20,892	2.5271	2.4894	0.6300	0.0629	7.8258
Entropy (Llama-3.2-3B-Instruct) (AI)	20,892	2.6020	2.4018	1.1649	0.1077	9.3056
Entropy (Llama-3.2-3B) (Human)	20,892	2.3825	2.4005	0.5707	0.0411	6.3306
Entropy (Llama-3.2-3B) (AI)	20,892	2.5570	2.4012	0.9664	0.0909	8.3789
Top-p Outliers (Llama-3.2-3B-Instruct) (Human)	20,892	0.0554	0.0535	0.0163	0.0000	0.2857
Top-p Outliers (Llama-3.2-3B-Instruct) (AI)	20,892	0.0522	0.0507	0.0173	0.0000	0.2308
Top-p Outliers (Llama-3.2-3B) (Human)	20,892	0.0415	0.0414	0.0127	0.0000	0.1774
Top-p Outliers (Llama-3.2-3B) (AI)	20,892	0.0425	0.0401	0.0196	0.0000	0.2222
Top-k Outliers (Llama-3.2-3B-Instruct) (Human)	20,892	0.1060	0.0973	0.0523	0.0000	0.6452
Top-k Outliers (Llama-3.2-3B-Instruct) (AI)	20,892	0.1169	0.0865	0.1238	0.0000	0.8796
Top-k Outliers (Llama-3.2-3B) (Human)	20,892	0.0868	0.0788	0.0474	0.0000	0.5529
Top-k Outliers (Llama-3.2-3B) (AI)	20,892	0.1076	0.0760	0.1235	0.0000	0.8802
FastDetectGPT (Llama-3.2-3B-Instruct) (Human)	20,892	-1.6739	-1.7396	1.9618	-21.8239	25.2161
FastDetectGPT (Llama-3.2-3B-Instruct) (AI)	20,892	-1.1973	-1.3517	1.9641	-21.0610	20.2645
FastDetectGPT (Llama-3.2-3B) (Human)	20,892	-0.1766	-0.1449	1.1141	-7.3730	9.3717
FastDetectGPT (Llama-3.2-3B) (AI)	20,892	-0.7282	-0.2523	3.8187	-38.9644	16.2603
Binoculars (Human)	20,892	0.8280	0.8516	0.1309	0.0075	1.2099
Binoculars (AI)	20,892	0.8748	0.8787	0.0876	0.0257	1.1761

Correlation Heatmap

Pearson correlation between every statistic of interest, computed over the rows where both statistics are present.

[image]

Distance Histograms

[image] [image] [image] [image] [image] [image] [image] [image] [image] [image]

Distance Histograms per Prompt Subset

[image] [image] [image] [image] [image] [image] [image] [image] [image] [image]

Distance Histograms per Generator Config Subset

[image] [image] [image] [image] [image] [image] [image] [image] [image] [image]

Classifier: EditLens Roberta-Large Score

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	41,784	0.8775	0.3588	0.0046	0.6771	0.5263
Prompt: direct_reference	10,356	0.8405	0.4181	0.0041	0.7070	0.5880
Prompt: indirect_reference	10,182	0.9333	0.4331	0.0055	0.7138	0.6021
Prompt: revise	10,666	0.9662	0.4568	0.0039	0.7264	0.6254
❗ Prompt: rewrite	10,580	0.7711	0.1304	0.0049	0.5628	0.2298
Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 0.6)	6,064	0.8177	0.2642	0.0040	0.6301	0.4166
Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 1.25)	6,004	0.8537	0.1666	0.0070	0.5798	0.2838
✔️ Model: Ministral-3-8B-Instruct-2512-AWQ-4bit (Temp: 0.6)	5,860	0.9907	0.6451	0.0034	0.8208	0.7826
Model: Qwen3-8B-AWQ (Temp: 0.7)	6,118	0.8077	0.2958	0.0072	0.6443	0.4541
Model: gemma-4-E4B-it (Temp: 0.7)	5,972	0.8523	0.2515	0.0027	0.6244	0.4011
Model: granite-4.1-8b-AWQ-INT4 (Temp: 0.6)	5,876	0.9029	0.4364	0.0024	0.7170	0.6066
Model: granite-4.1-8b-AWQ-INT4 (Temp: 1.25)	5,890	0.9249	0.4642	0.0054	0.7294	0.6317

Thresholding:

—Direction: higher_is_ai
—Swept for fpr_0_5pct with a found threshold of 0.5961.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image] [image] [image]

Classifier: EditLens Roberta-Large Bucket

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	41,784	0.8145	0.6656	0.0513	0.8072	0.7754
Prompt: direct_reference	10,356	0.7897	0.6124	0.0508	0.7808	0.7364
Prompt: indirect_reference	10,182	0.8516	0.7362	0.0517	0.8423	0.8236
Prompt: revise	10,666	0.9239	0.8785	0.0491	0.9147	0.9115
❗ Prompt: rewrite	10,580	0.6928	0.4352	0.0537	0.6907	0.5846
Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 0.6)	6,064	0.7088	0.4571	0.0508	0.7032	0.6063
Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 1.25)	6,004	0.7709	0.5936	0.0566	0.7685	0.7194
✔️ Model: Ministral-3-8B-Instruct-2512-AWQ-4bit (Temp: 0.6)	5,860	0.9743	0.9703	0.0495	0.9604	0.9608
Model: Qwen3-8B-AWQ (Temp: 0.7)	6,118	0.7343	0.5093	0.0517	0.7288	0.6526
Model: gemma-4-E4B-it (Temp: 0.7)	5,972	0.7853	0.6102	0.0492	0.7805	0.7354
Model: granite-4.1-8b-AWQ-INT4 (Temp: 0.6)	5,876	0.8527	0.7372	0.0524	0.8424	0.8239
Model: granite-4.1-8b-AWQ-INT4 (Temp: 1.25)	5,890	0.8838	0.7976	0.0489	0.8744	0.8639

Thresholding:

—Direction: higher_is_ai
—Swept for f1 with a found threshold of 0.0000.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image] [image] [image]

Classifier: Perplexity (Llama-3.2-3B-Instruct)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	41,784	0.5478	0.0055	0.0045	0.5005	0.0108
Prompt: direct_reference	10,356	0.5778	0.0062	0.0052	0.5005	0.0122
Prompt: indirect_reference	10,182	0.5590	0.0132	0.0027	0.5052	0.0259
Prompt: revise	10,666	0.5867	0.0013	0.0051	0.4981	0.0026
Prompt: rewrite	10,580	0.4672	0.0015	0.0047	0.4984	0.0030
✔️ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 0.6)	6,064	0.7393	0.0254	0.0030	0.5112	0.0494
❗ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 1.25)	6,004	0.2114	0.0007	0.0057	0.4975	0.0013
Model: Ministral-3-8B-Instruct-2512-AWQ-4bit (Temp: 0.6)	5,860	0.6083	0.0007	0.0065	0.4971	0.0014
Model: Qwen3-8B-AWQ (Temp: 0.7)	6,118	0.6866	0.0046	0.0033	0.5007	0.0091
Model: gemma-4-E4B-it (Temp: 0.7)	5,972	0.4750	0.0020	0.0033	0.4993	0.0040
Model: granite-4.1-8b-AWQ-INT4 (Temp: 0.6)	5,876	0.6144	0.0037	0.0051	0.4993	0.0074
Model: granite-4.1-8b-AWQ-INT4 (Temp: 1.25)	5,890	0.4990	0.0007	0.0044	0.4981	0.0014

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of 2.2698.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image] [image] [image]

Classifier: Perplexity (Llama-3.2-3B)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	41,784	0.4854	0.0005	0.0044	0.4980	0.0010
Prompt: direct_reference	10,356	0.5068	0.0004	0.0058	0.4973	0.0008
Prompt: indirect_reference	10,182	0.4852	0.0010	0.0026	0.4992	0.0020
Prompt: revise	10,666	0.5149	0.0000	0.0049	0.4976	0.0000
Prompt: rewrite	10,580	0.4338	0.0006	0.0043	0.4981	0.0011
✔️ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 0.6)	6,064	0.6774	0.0007	0.0023	0.4992	0.0013
❗ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 1.25)	6,004	0.1564	0.0003	0.0063	0.4970	0.0007
Model: Ministral-3-8B-Instruct-2512-AWQ-4bit (Temp: 0.6)	5,860	0.5326	0.0000	0.0055	0.4973	0.0000
Model: Qwen3-8B-AWQ (Temp: 0.7)	6,118	0.6481	0.0007	0.0029	0.4989	0.0013
Model: gemma-4-E4B-it (Temp: 0.7)	5,972	0.4061	0.0007	0.0023	0.4992	0.0013
Model: granite-4.1-8b-AWQ-INT4 (Temp: 0.6)	5,876	0.5512	0.0007	0.0054	0.4976	0.0014
Model: granite-4.1-8b-AWQ-INT4 (Temp: 1.25)	5,890	0.4266	0.0003	0.0061	0.4971	0.0007

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of 1.2916.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image] [image] [image]

Classifier: Entropy (Llama-3.2-3B-Instruct)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	41,784	0.5391	0.0032	0.0044	0.4994	0.0063
Prompt: direct_reference	10,356	0.5668	0.0029	0.0050	0.4989	0.0057
Prompt: indirect_reference	10,182	0.5409	0.0077	0.0026	0.5026	0.0152
Prompt: revise	10,666	0.5799	0.0009	0.0047	0.4981	0.0019
Prompt: rewrite	10,580	0.4678	0.0013	0.0051	0.4981	0.0026
✔️ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 0.6)	6,064	0.7033	0.0119	0.0026	0.5046	0.0234
❗ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 1.25)	6,004	0.2467	0.0003	0.0053	0.4975	0.0007
Model: Ministral-3-8B-Instruct-2512-AWQ-4bit (Temp: 0.6)	5,860	0.5743	0.0007	0.0058	0.4974	0.0014
Model: Qwen3-8B-AWQ (Temp: 0.7)	6,118	0.6652	0.0039	0.0033	0.5003	0.0078
Model: gemma-4-E4B-it (Temp: 0.7)	5,972	0.4608	0.0017	0.0033	0.4992	0.0033
Model: granite-4.1-8b-AWQ-INT4 (Temp: 0.6)	5,876	0.6065	0.0024	0.0051	0.4986	0.0047
Model: granite-4.1-8b-AWQ-INT4 (Temp: 1.25)	5,890	0.5167	0.0010	0.0051	0.4980	0.0020

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of 0.7575.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image] [image] [image]

Classifier: Entropy (Llama-3.2-3B)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	41,784	0.4869	0.0005	0.0049	0.4978	0.0010
Prompt: direct_reference	10,356	0.4755	0.0004	0.0062	0.4971	0.0008
Prompt: indirect_reference	10,182	0.4783	0.0010	0.0037	0.4986	0.0020
Prompt: revise	10,666	0.5343	0.0000	0.0054	0.4973	0.0000
Prompt: rewrite	10,580	0.4567	0.0008	0.0042	0.4983	0.0015
✔️ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 0.6)	6,064	0.6379	0.0003	0.0030	0.4987	0.0007
❗ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 1.25)	6,004	0.2151	0.0003	0.0057	0.4973	0.0007
Model: Ministral-3-8B-Instruct-2512-AWQ-4bit (Temp: 0.6)	5,860	0.4974	0.0000	0.0061	0.4969	0.0000
Model: Qwen3-8B-AWQ (Temp: 0.7)	6,118	0.6106	0.0013	0.0042	0.4985	0.0026
Model: gemma-4-E4B-it (Temp: 0.7)	5,972	0.4135	0.0003	0.0027	0.4988	0.0007
Model: granite-4.1-8b-AWQ-INT4 (Temp: 0.6)	5,876	0.5525	0.0010	0.0068	0.4971	0.0020
Model: granite-4.1-8b-AWQ-INT4 (Temp: 1.25)	5,890	0.4814	0.0003	0.0058	0.4973	0.0007

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of 0.2888.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image] [image] [image]

Classifier: Top-p Outliers (Llama-3.2-3B-Instruct)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	41,784	0.5598	0.0108	0.0030	0.5039	0.0213
Prompt: direct_reference	10,356	0.5837	0.0160	0.0029	0.5066	0.0315
Prompt: indirect_reference	10,182	0.5977	0.0246	0.0029	0.5108	0.0478
Prompt: revise	10,666	0.5320	0.0017	0.0036	0.4991	0.0034
Prompt: rewrite	10,580	0.5280	0.0017	0.0026	0.4995	0.0034
✔️ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 0.6)	6,064	0.6963	0.0501	0.0030	0.5236	0.0952
Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 1.25)	6,004	0.5704	0.0043	0.0040	0.5002	0.0086
Model: Ministral-3-8B-Instruct-2512-AWQ-4bit (Temp: 0.6)	5,860	0.5563	0.0010	0.0027	0.4991	0.0020
Model: Qwen3-8B-AWQ (Temp: 0.7)	6,118	0.6147	0.0111	0.0026	0.5042	0.0219
Model: gemma-4-E4B-it (Temp: 0.7)	5,972	0.5118	0.0044	0.0020	0.5012	0.0087
Model: granite-4.1-8b-AWQ-INT4 (Temp: 0.6)	5,876	0.5298	0.0027	0.0024	0.5002	0.0054
❗ Model: granite-4.1-8b-AWQ-INT4 (Temp: 1.25)	5,890	0.4325	0.0010	0.0044	0.4983	0.0020

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of 0.0158.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image] [image] [image]

Classifier: Top-p Outliers (Llama-3.2-3B)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	41,784	0.5168	0.0062	0.0071	0.4995	0.0122
Prompt: direct_reference	10,356	0.6107	0.0100	0.0077	0.5012	0.0197
Prompt: indirect_reference	10,182	0.5249	0.0128	0.0063	0.5032	0.0251
Prompt: revise	10,666	0.4953	0.0009	0.0075	0.4967	0.0019
Prompt: rewrite	10,580	0.4396	0.0013	0.0068	0.4973	0.0026
✔️ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 0.6)	6,064	0.6752	0.0168	0.0053	0.5058	0.0329
❗ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 1.25)	6,004	0.2196	0.0010	0.0083	0.4963	0.0020
Model: Ministral-3-8B-Instruct-2512-AWQ-4bit (Temp: 0.6)	5,860	0.6009	0.0003	0.0075	0.4964	0.0007
Model: Qwen3-8B-AWQ (Temp: 0.7)	6,118	0.6662	0.0144	0.0072	0.5036	0.0282
Model: gemma-4-E4B-it (Temp: 0.7)	5,972	0.4851	0.0040	0.0040	0.5000	0.0080
Model: granite-4.1-8b-AWQ-INT4 (Temp: 0.6)	5,876	0.5588	0.0034	0.0085	0.4974	0.0067
Model: granite-4.1-8b-AWQ-INT4 (Temp: 1.25)	5,890	0.4100	0.0027	0.0088	0.4969	0.0054

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of 0.0054.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image] [image] [image]

Classifier: Top-k Outliers (Llama-3.2-3B-Instruct)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	41,784	0.5637	0.0191	0.0033	0.5079	0.0373
Prompt: direct_reference	10,356	0.6161	0.0373	0.0041	0.5166	0.0716
Prompt: indirect_reference	10,182	0.5662	0.0346	0.0018	0.5164	0.0667
Prompt: revise	10,666	0.5998	0.0038	0.0034	0.5002	0.0074
Prompt: rewrite	10,580	0.4725	0.0017	0.0038	0.4990	0.0034
✔️ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 0.6)	6,064	0.7196	0.0627	0.0020	0.5303	0.1177
❗ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 1.25)	6,004	0.2595	0.0013	0.0053	0.4980	0.0026
Model: Ministral-3-8B-Instruct-2512-AWQ-4bit (Temp: 0.6)	5,860	0.6421	0.0010	0.0048	0.4981	0.0020
Model: Qwen3-8B-AWQ (Temp: 0.7)	6,118	0.6911	0.0487	0.0023	0.5232	0.0927
Model: gemma-4-E4B-it (Temp: 0.7)	5,972	0.5050	0.0013	0.0013	0.5000	0.0027
Model: granite-4.1-8b-AWQ-INT4 (Temp: 0.6)	5,876	0.6051	0.0123	0.0044	0.5039	0.0241
Model: granite-4.1-8b-AWQ-INT4 (Temp: 1.25)	5,890	0.5226	0.0041	0.0027	0.5007	0.0081

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of 0.0109.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image] [image] [image]

Classifier: Top-k Outliers (Llama-3.2-3B)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	41,784	0.5168	0.0053	0.0048	0.5002	0.0104
Prompt: direct_reference	10,356	0.5622	0.0085	0.0054	0.5015	0.0168
Prompt: indirect_reference	10,182	0.5113	0.0104	0.0039	0.5032	0.0205
Prompt: revise	10,666	0.5465	0.0011	0.0054	0.4978	0.0022
Prompt: rewrite	10,580	0.4465	0.0013	0.0043	0.4985	0.0026
✔️ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 0.6)	6,064	0.6778	0.0155	0.0033	0.5061	0.0304
❗ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 1.25)	6,004	0.2099	0.0007	0.0060	0.4973	0.0013
Model: Ministral-3-8B-Instruct-2512-AWQ-4bit (Temp: 0.6)	5,860	0.5863	0.0000	0.0055	0.4973	0.0000
Model: Qwen3-8B-AWQ (Temp: 0.7)	6,118	0.6659	0.0147	0.0039	0.5054	0.0289
Model: gemma-4-E4B-it (Temp: 0.7)	5,972	0.4514	0.0007	0.0027	0.4990	0.0013
Model: granite-4.1-8b-AWQ-INT4 (Temp: 0.6)	5,876	0.5570	0.0041	0.0071	0.4985	0.0081
Model: granite-4.1-8b-AWQ-INT4 (Temp: 1.25)	5,890	0.4683	0.0007	0.0051	0.4978	0.0014

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of 0.0056.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image] [image] [image]

Classifier: FastDetectGPT (Llama-3.2-3B-Instruct)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	41,784	0.5725	0.0027	0.0041	0.4993	0.0054
Prompt: direct_reference	10,356	0.5900	0.0023	0.0048	0.4987	0.0046
Prompt: indirect_reference	10,182	0.6299	0.0020	0.0045	0.4987	0.0039
Prompt: revise	10,666	0.5402	0.0034	0.0047	0.4993	0.0067
Prompt: rewrite	10,580	0.5334	0.0032	0.0025	0.5004	0.0064
✔️ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 0.6)	6,064	0.6945	0.0013	0.0053	0.4980	0.0026
Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 1.25)	6,004	0.5419	0.0080	0.0067	0.5007	0.0158
Model: Ministral-3-8B-Instruct-2512-AWQ-4bit (Temp: 0.6)	5,860	0.5923	0.0007	0.0027	0.4990	0.0014
Model: Qwen3-8B-AWQ (Temp: 0.7)	6,118	0.5891	0.0010	0.0016	0.4997	0.0020
Model: gemma-4-E4B-it (Temp: 0.7)	5,972	0.5690	0.0044	0.0057	0.4993	0.0086
Model: granite-4.1-8b-AWQ-INT4 (Temp: 0.6)	5,876	0.5443	0.0017	0.0044	0.4986	0.0034
❗ Model: granite-4.1-8b-AWQ-INT4 (Temp: 1.25)	5,890	0.4715	0.0020	0.0024	0.4998	0.0041

Thresholding:

—Direction: higher_is_ai
—Swept for fpr_0_5pct with a found threshold of 6.7770.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image] [image] [image]

Classifier: FastDetectGPT (Llama-3.2-3B)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	41,784	0.5159	0.0986	0.0066	0.5460	0.1784
Prompt: direct_reference	10,356	0.4288	0.0709	0.0070	0.5320	0.1315
Prompt: indirect_reference	10,182	0.4831	0.1159	0.0075	0.5542	0.2063
Prompt: revise	10,666	0.5541	0.1001	0.0056	0.5473	0.1811
Prompt: rewrite	10,580	0.5944	0.1076	0.0062	0.5507	0.1931
❗ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 0.6)	6,064	0.2939	0.0030	0.0082	0.4974	0.0059
✔️ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 1.25)	6,004	0.8992	0.5673	0.0057	0.7808	0.7213
Model: Ministral-3-8B-Instruct-2512-AWQ-4bit (Temp: 0.6)	5,860	0.3979	0.0143	0.0075	0.5034	0.0281
Model: Qwen3-8B-AWQ (Temp: 0.7)	6,118	0.3076	0.0082	0.0062	0.5010	0.0161
Model: gemma-4-E4B-it (Temp: 0.7)	5,972	0.5519	0.0131	0.0064	0.5033	0.0256
Model: granite-4.1-8b-AWQ-INT4 (Temp: 0.6)	5,876	0.4675	0.0099	0.0054	0.5022	0.0194
Model: granite-4.1-8b-AWQ-INT4 (Temp: 1.25)	5,890	0.6924	0.0723	0.0065	0.5329	0.1341

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of -3.2326.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image] [image] [image]

Classifier: Binoculars

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	41,784	0.6302	0.0121	0.0031	0.5045	0.0239
Prompt: direct_reference	10,356	0.6006	0.0114	0.0027	0.5043	0.0225
Prompt: indirect_reference	10,182	0.6500	0.0192	0.0037	0.5078	0.0376
Prompt: revise	10,666	0.6654	0.0075	0.0034	0.5021	0.0148
Prompt: rewrite	10,580	0.6046	0.0106	0.0026	0.5040	0.0209
Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 0.6)	6,064	0.5311	0.0013	0.0036	0.4988	0.0026
✔️ Model: Meta-Llama-3.1-8B-Instruct-AWQ-INT4 (Temp: 1.25)	6,004	0.8816	0.0403	0.0033	0.5185	0.0772
Model: Ministral-3-8B-Instruct-2512-AWQ-4bit (Temp: 0.6)	5,860	0.5429	0.0092	0.0038	0.5027	0.0182
❗ Model: Qwen3-8B-AWQ (Temp: 0.7)	6,118	0.4838	0.0013	0.0036	0.4989	0.0026
Model: gemma-4-E4B-it (Temp: 0.7)	5,972	0.6467	0.0074	0.0033	0.5020	0.0146
Model: granite-4.1-8b-AWQ-INT4 (Temp: 0.6)	5,876	0.6074	0.0065	0.0017	0.5024	0.0128
Model: granite-4.1-8b-AWQ-INT4 (Temp: 1.25)	5,890	0.7140	0.0190	0.0024	0.5083	0.0372

Thresholding:

—Direction: higher_is_ai
—Swept for fpr_0_5pct with a found threshold of 1.0472.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image] [image] [image]