Datasetpublic

G-reen/cc-re-2021-stat-val

Auto-Generated FastDetector Dataset Dataset: G-reen/cc-re-2021-stat-val Globals Config: config/globals_re2021.toml Analysis Config: config/analysis_nofilter.toml Rows: 9,822 Evaluation Results Prompt Subsets: 4 (direct_reference, indirect_reference, revise, rewrite) Generator Configs: 5 (Hy3-NVFP4-FP8 (Temp: 0.9), Llama-4-Scout-17B-16E-Instruct-NVFP4 (Temp: 0.6), Mixtral-8x7B-Instruct-v0.1 (Temp: 0.7), claude-sonnet-5 (Temp: Unknown), gpt-5.6-luna (Temp:… See the full description on the dataset page: https://huggingface.co/datasets/G-reen/cc-re-2021-stat-val.

sourceHugging Faceupdated 15d agoView on Hugging Face

0likes870downloads

Dataset Card

Auto-Generated FastDetector Dataset

—Dataset: G-reen/cc-re-2021-stat-val
—Globals Config: config/globals_re2021.toml
—Analysis Config: config/analysis_nofilter.toml
—Rows: 9,822

Evaluation Results

—Prompt Subsets: 4 (directreference, indirectreference, revise, rewrite)
—Generator Configs: 5 (Hy3-NVFP4-FP8 (Temp: 0.9), Llama-4-Scout-17B-16E-Instruct-NVFP4 (Temp: 0.6), Mixtral-8x7B-Instruct-v0.1 (Temp: 0.7), claude-sonnet-5 (Temp: Unknown), gpt-5.6-luna (Temp: Unknown))
—Classifiers: 13 (EditLens Roberta-Large Score, EditLens Roberta-Large Bucket, Perplexity (Llama-3.2-3B-Instruct), Perplexity (Llama-3.2-3B), Entropy (Llama-3.2-3B-Instruct), Entropy (Llama-3.2-3B), Top-p Outliers (Llama-3.2-3B-Instruct), Top-p Outliers (Llama-3.2-3B), Top-k Outliers (Llama-3.2-3B-Instruct), Top-k Outliers (Llama-3.2-3B), FastDetectGPT (Llama-3.2-3B-Instruct), FastDetectGPT (Llama-3.2-3B), Binoculars)
—Filter Conditions: None
—Evaluation / Validation Rows: 8,839 / 983 (validation_size = 0.1)
—Base Columns: original (Human), final_response (AI)

The best classifier was EditLens Roberta-Large Score with an AUROC of 0.8623. The hardest prompt subset was rewrite with a TPR of 0.0384, and the hardest generator config was Hy3-NVFP4-FP8 (Temp: 0.9) with a TPR of 0.0542.

Classifier	Threshold	AUROC	TPR	FPR	Accuracy	F1
✔️ EditLens Roberta-Large Score	0.8261	0.8623	0.2073	0.0023	0.6025	0.3427
EditLens Roberta-Large Bucket	0.0000	0.7646	0.5737	0.0579	0.7579	0.7032
Entropy (Llama-3.2-3B-Instruct)	1.4754	0.6384	0.0815	0.0063	0.5376	0.1498
Perplexity (Llama-3.2-3B-Instruct)	4.8037	0.6340	0.0753	0.0038	0.5358	0.1396
Top-k Outliers (Llama-3.2-3B-Instruct)	0.0172	0.6128	0.0428	0.0025	0.5201	0.0818
Top-p Outliers (Llama-3.2-3B)	0.0034	0.6123	0.0050	0.0041	0.5005	0.0099
Binoculars	1.0087	0.6085	0.0303	0.0091	0.5106	0.0583
Perplexity (Llama-3.2-3B)	1.2354	0.5923	0.0001	0.0019	0.4991	0.0002
Top-k Outliers (Llama-3.2-3B)	0.0012	0.5792	0.0009	0.0007	0.5001	0.0018
Entropy (Llama-3.2-3B)	0.2195	0.5758	0.0002	0.0020	0.4991	0.0005
Top-p Outliers (Llama-3.2-3B-Instruct)	0.0275	0.5277	0.0354	0.0081	0.5136	0.0679
FastDetectGPT (Llama-3.2-3B-Instruct)	3.3428	0.4990	0.0038	0.0069	0.4985	0.0076
❗ FastDetectGPT (Llama-3.2-3B)	-3.1816	0.3950	0.0089	0.0037	0.5026	0.0177

Classifier metrics averaged within each prompt and generator subset:

Subset	Average AUROC	Average TPR	Average FPR	Average Accuracy	Average F1
✔️ Model: Mixtral-8x7B-Instruct-v0.1 (Temp: 0.7)	0.6706	0.1336	0.0094	0.5621	0.1817
Model: Llama-4-Scout-17B-16E-Instruct-NVFP4 (Temp: 0.6)	0.6606	0.1047	0.0078	0.5484	0.1625
Prompt: indirect_reference	0.6370	0.0916	0.0085	0.5416	0.1402
Prompt: direct_reference	0.6335	0.1105	0.0078	0.5513	0.1595
Prompt: revise	0.6257	0.0822	0.0087	0.5367	0.1066
Model: gpt-5.6-luna (Temp: Unknown)	0.6123	0.0578	0.0091	0.5244	0.0815
Model: claude-sonnet-5 (Temp: Unknown)	0.5951	0.0598	0.0075	0.5261	0.0848
Prompt: rewrite	0.5283	0.0384	0.0087	0.5148	0.0621
❗ Model: Hy3-NVFP4-FP8 (Temp: 0.9)	0.5027	0.0542	0.0084	0.5229	0.0800

✔️ marks the best AUROC, ❗ the worst.

Statistics of Interest

[image] [image] [image] [image]

Appendix

Table of contents 1. Univariate Analysis 2. Correlation Heatmap 3. Distance Histograms 4. Distance Histograms per Prompt Subset 5. Distance Histograms per Generator Config Subset 6. Classifier: EditLens Roberta-Large Score - Performance: - Thresholding: - Classification Histograms: 7. Classifier: EditLens Roberta-Large Bucket - Performance: - Thresholding: - Classification Histograms: 8. Classifier: Perplexity (Llama-3.2-3B-Instruct) - Performance: - Thresholding: - Classification Histograms: 9. Classifier: Perplexity (Llama-3.2-3B) - Performance: - Thresholding: - Classification Histograms: 10. Classifier: Entropy (Llama-3.2-3B-Instruct) - Performance: - Thresholding: - Classification Histograms: 11. Classifier: Entropy (Llama-3.2-3B) - Performance: - Thresholding: - Classification Histograms: 12. Classifier: Top-p Outliers (Llama-3.2-3B-Instruct) - Performance: - Thresholding: - Classification Histograms: 13. Classifier: Top-p Outliers (Llama-3.2-3B) - Performance: - Thresholding: - Classification Histograms: 14. Classifier: Top-k Outliers (Llama-3.2-3B-Instruct) - Performance: - Thresholding: - Classification Histograms: 15. Classifier: Top-k Outliers (Llama-3.2-3B) - Performance: - Thresholding: - Classification Histograms: 16. Classifier: FastDetectGPT (Llama-3.2-3B-Instruct) - Performance: - Thresholding: - Classification Histograms: 17. Classifier: FastDetectGPT (Llama-3.2-3B) - Performance: - Thresholding: - Classification Histograms: 18. Classifier: Binoculars - Performance: - Thresholding: - Classification Histograms:

Univariate Analysis

Every statistic the report does arithmetic on, over the 8,839-row evaluation split. Invalid counts rows whose value is missing or non-finite; those rows are excluded from the other columns.

Statistic	N	Mean	Median	Std	Min	Max
jaccard_1	8,839	0.6076	0.7128	0.2768	0.0000	1.0000
jaccard_2	8,839	0.7274	0.8750	0.2992	0.0000	1.0000
levenshtein	8,839	2109.1352	1222.0000	3177.3081	1.0000	63409.0000
softngram	8,839	0.5287	0.5714	0.3462	0.0000	1.0000
cosdist	8,839	0.1692	0.0990	0.1874	-0.0073	1.0241
bertscore	8,839	0.1265	0.1351	0.0748	0.0001	0.6915
bertscore_precision	8,839	0.1225	0.1282	0.0731	0.0001	0.6779
bertscore_recall	8,839	0.1294	0.1353	0.0813	0.0000	0.7040
moverscore	8,839	0.5068	0.5647	0.1998	0.0397	1.2479
reranker	8,839	-4.4972	-6.6875	5.3779	-11.3125	17.1250
EditLens Roberta-Large Score (Human)	8,839	0.0581	0.0203	0.0965	0.0063	0.9996
EditLens Roberta-Large Score (AI)	8,839	0.4077	0.3069	0.3496	0.0064	0.9996
EditLens Roberta-Large Bucket (Human)	8,839	0.0677	0.0000	0.3038	0.0000	3.0000
EditLens Roberta-Large Bucket (AI)	8,839	1.1323	1.0000	1.2337	0.0000	3.0000
Perplexity (Llama-3.2-3B-Instruct) (Human)	8,839	18.5745	15.7332	12.8903	1.7117	630.4184
Perplexity (Llama-3.2-3B-Instruct) (AI)	8,839	15.6951	12.5905	19.4628	1.1353	1026.7011
Perplexity (Llama-3.2-3B) (Human)	8,839	13.8455	12.1471	8.4284	1.0221	346.8781
Perplexity (Llama-3.2-3B) (AI)	8,839	12.8355	10.4477	16.8959	1.1205	1166.1689
Entropy (Llama-3.2-3B-Instruct) (Human)	8,839	2.6023	2.5730	0.4808	0.5658	7.6310
Entropy (Llama-3.2-3B-Instruct) (AI)	8,839	2.3347	2.3396	0.6075	0.1347	5.9743
Entropy (Llama-3.2-3B) (Human)	8,839	2.4922	2.4942	0.4534	0.0414	5.6525
Entropy (Llama-3.2-3B) (AI)	8,839	2.3794	2.3840	0.4998	0.1245	5.0878
Top-p Outliers (Llama-3.2-3B-Instruct) (Human)	8,839	0.0574	0.0545	0.0162	0.0079	0.1667
Top-p Outliers (Llama-3.2-3B-Instruct) (AI)	8,839	0.0559	0.0536	0.0186	0.0000	0.2778
Top-p Outliers (Llama-3.2-3B) (Human)	8,839	0.0427	0.0424	0.0123	0.0000	0.1282
Top-p Outliers (Llama-3.2-3B) (AI)	8,839	0.0383	0.0375	0.0156	0.0000	0.2000
Top-k Outliers (Llama-3.2-3B-Instruct) (Human)	8,839	0.1050	0.0990	0.0433	0.0037	0.4896
Top-k Outliers (Llama-3.2-3B-Instruct) (AI)	8,839	0.0896	0.0827	0.0521	0.0000	0.6429
Top-k Outliers (Llama-3.2-3B) (Human)	8,839	0.0857	0.0803	0.0388	0.0000	0.4062
Top-k Outliers (Llama-3.2-3B) (AI)	8,839	0.0776	0.0696	0.0470	0.0000	0.5714
FastDetectGPT (Llama-3.2-3B-Instruct) (Human)	8,839	-1.6820	-1.6686	1.6920	-14.0885	10.5181
FastDetectGPT (Llama-3.2-3B-Instruct) (AI)	8,839	-1.7002	-1.6766	1.7277	-11.7548	9.8134
FastDetectGPT (Llama-3.2-3B) (Human)	8,839	-0.1307	-0.0957	1.0472	-7.1757	6.8542
FastDetectGPT (Llama-3.2-3B) (AI)	8,839	0.3764	0.3362	1.5206	-11.2500	8.0566
Binoculars (Human)	8,839	0.8506	0.8608	0.1044	0.0078	1.1149
Binoculars (AI)	8,839	0.8805	0.8808	0.0767	0.0672	1.3212

Correlation Heatmap

Pearson correlation between every statistic of interest, computed over the rows where both statistics are present.

[image]

Distance Histograms

[image] [image] [image] [image] [image] [image] [image] [image] [image] [image]

Distance Histograms per Prompt Subset

[image] [image] [image] [image] [image] [image] [image] [image] [image] [image]

Distance Histograms per Generator Config Subset

[image] [image] [image] [image] [image] [image] [image] [image] [image] [image]

Classifier: EditLens Roberta-Large Score

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	17,678	0.8623	0.2073	0.0023	0.6025	0.3427
Prompt: direct_reference	4,744	0.9045	0.3297	0.0013	0.6642	0.4954
Prompt: indirect_reference	4,384	0.8760	0.2249	0.0027	0.6111	0.3664
Prompt: revise	4,444	0.9379	0.1809	0.0032	0.5889	0.3056
❗ Prompt: rewrite	4,106	0.7209	0.0755	0.0019	0.5368	0.1401
Model: Hy3-NVFP4-FP8 (Temp: 0.9)	3,564	0.8226	0.1178	0.0034	0.5572	0.2102
Model: Llama-4-Scout-17B-16E-Instruct-NVFP4 (Temp: 0.6)	3,558	0.8214	0.2282	0.0006	0.6138	0.3715
✔️ Model: Mixtral-8x7B-Instruct-v0.1 (Temp: 0.7)	3,508	0.9589	0.3997	0.0023	0.6987	0.5702
Model: claude-sonnet-5 (Temp: Unknown)	3,566	0.8490	0.1604	0.0017	0.5794	0.2761
Model: gpt-5.6-luna (Temp: Unknown)	3,482	0.8617	0.1315	0.0034	0.5640	0.2318

Thresholding:

—Direction: higher_is_ai
—Swept for fpr_0_5pct with a found threshold of 0.8261.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image]

Classifier: EditLens Roberta-Large Bucket

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	17,678	0.7646	0.5737	0.0579	0.7579	0.7032
Prompt: direct_reference	4,744	0.8109	0.6585	0.0573	0.8006	0.7676
Prompt: indirect_reference	4,384	0.7456	0.5324	0.0538	0.7393	0.6713
Prompt: revise	4,444	0.8582	0.7628	0.0594	0.8517	0.8372
❗ Prompt: rewrite	4,106	0.6297	0.3151	0.0614	0.6269	0.4579
Model: Hy3-NVFP4-FP8 (Temp: 0.9)	3,564	0.7166	0.4804	0.0550	0.7127	0.6257
Model: Llama-4-Scout-17B-16E-Instruct-NVFP4 (Temp: 0.6)	3,558	0.7258	0.4924	0.0540	0.7192	0.6369
✔️ Model: Mixtral-8x7B-Instruct-v0.1 (Temp: 0.7)	3,508	0.8882	0.8136	0.0673	0.8731	0.8651
Model: claude-sonnet-5 (Temp: Unknown)	3,566	0.7478	0.5407	0.0555	0.7426	0.6774
Model: gpt-5.6-luna (Temp: Unknown)	3,482	0.7475	0.5445	0.0580	0.7433	0.6796

Thresholding:

—Direction: higher_is_ai
—Swept for f1 with a found threshold of 0.0000.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image]

Classifier: Perplexity (Llama-3.2-3B-Instruct)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	17,678	0.6340	0.0753	0.0038	0.5358	0.1396
Prompt: direct_reference	4,744	0.6756	0.1290	0.0046	0.5622	0.2276
Prompt: indirect_reference	4,384	0.6976	0.1286	0.0027	0.5630	0.2274
Prompt: revise	4,444	0.6545	0.0266	0.0045	0.5110	0.0515
Prompt: rewrite	4,106	0.4962	0.0093	0.0034	0.5029	0.0183
❗ Model: Hy3-NVFP4-FP8 (Temp: 0.9)	3,564	0.4025	0.0045	0.0034	0.5006	0.0089
✔️ Model: Llama-4-Scout-17B-16E-Instruct-NVFP4 (Temp: 0.6)	3,558	0.7437	0.2007	0.0034	0.5987	0.3333
Model: Mixtral-8x7B-Instruct-v0.1 (Temp: 0.7)	3,508	0.7316	0.1494	0.0057	0.5718	0.2586
Model: claude-sonnet-5 (Temp: Unknown)	3,566	0.6305	0.0107	0.0034	0.5036	0.0210
Model: gpt-5.6-luna (Temp: Unknown)	3,482	0.6652	0.0115	0.0034	0.5040	0.0226

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of 4.8037.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image]

Classifier: Perplexity (Llama-3.2-3B)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	17,678	0.5923	0.0001	0.0019	0.4991	0.0002
Prompt: direct_reference	4,744	0.6215	0.0000	0.0004	0.4998	0.0000
Prompt: indirect_reference	4,384	0.6632	0.0000	0.0023	0.4989	0.0000
Prompt: revise	4,444	0.6078	0.0000	0.0023	0.4989	0.0000
Prompt: rewrite	4,106	0.4671	0.0005	0.0029	0.4988	0.0010
❗ Model: Hy3-NVFP4-FP8 (Temp: 0.9)	3,564	0.3723	0.0000	0.0017	0.4992	0.0000
✔️ Model: Llama-4-Scout-17B-16E-Instruct-NVFP4 (Temp: 0.6)	3,558	0.7002	0.0000	0.0028	0.4986	0.0000
Model: Mixtral-8x7B-Instruct-v0.1 (Temp: 0.7)	3,508	0.6710	0.0000	0.0023	0.4989	0.0000
Model: claude-sonnet-5 (Temp: Unknown)	3,566	0.5857	0.0000	0.0006	0.4997	0.0000
Model: gpt-5.6-luna (Temp: Unknown)	3,482	0.6371	0.0006	0.0023	0.4991	0.0011

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of 1.2354.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image]

Classifier: Entropy (Llama-3.2-3B-Instruct)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	17,678	0.6384	0.0815	0.0063	0.5376	0.1498
Prompt: direct_reference	4,744	0.6805	0.1336	0.0080	0.5628	0.2341
Prompt: indirect_reference	4,384	0.6961	0.1401	0.0046	0.5677	0.2447
Prompt: revise	4,444	0.6583	0.0320	0.0059	0.5131	0.0616
Prompt: rewrite	4,106	0.5081	0.0122	0.0068	0.5027	0.0239
❗ Model: Hy3-NVFP4-FP8 (Temp: 0.9)	3,564	0.4234	0.0056	0.0056	0.5000	0.0111
Model: Llama-4-Scout-17B-16E-Instruct-NVFP4 (Temp: 0.6)	3,558	0.7171	0.1804	0.0067	0.5868	0.3040
✔️ Model: Mixtral-8x7B-Instruct-v0.1 (Temp: 0.7)	3,508	0.7416	0.1819	0.0086	0.5867	0.3056
Model: claude-sonnet-5 (Temp: Unknown)	3,566	0.6325	0.0224	0.0050	0.5087	0.0437
Model: gpt-5.6-luna (Temp: Unknown)	3,482	0.6804	0.0172	0.0057	0.5057	0.0337

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of 1.4754.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image]

Classifier: Entropy (Llama-3.2-3B)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	17,678	0.5758	0.0002	0.0020	0.4991	0.0005
Prompt: direct_reference	4,744	0.5843	0.0000	0.0008	0.4996	0.0000
Prompt: indirect_reference	4,384	0.6367	0.0000	0.0027	0.4986	0.0000
Prompt: revise	4,444	0.5960	0.0000	0.0018	0.4991	0.0000
Prompt: rewrite	4,106	0.4805	0.0010	0.0029	0.4990	0.0019
❗ Model: Hy3-NVFP4-FP8 (Temp: 0.9)	3,564	0.3824	0.0000	0.0011	0.4994	0.0000
Model: Llama-4-Scout-17B-16E-Instruct-NVFP4 (Temp: 0.6)	3,558	0.6629	0.0006	0.0028	0.4989	0.0011
✔️ Model: Mixtral-8x7B-Instruct-v0.1 (Temp: 0.7)	3,508	0.6768	0.0000	0.0029	0.4986	0.0000
Model: claude-sonnet-5 (Temp: Unknown)	3,566	0.5654	0.0000	0.0011	0.4994	0.0000
Model: gpt-5.6-luna (Temp: Unknown)	3,482	0.5961	0.0006	0.0023	0.4991	0.0011

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of 0.2195.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image]

Classifier: Top-p Outliers (Llama-3.2-3B-Instruct)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	17,678	0.5277	0.0354	0.0081	0.5136	0.0679
Prompt: direct_reference	4,744	0.5471	0.0603	0.0072	0.5266	0.1130
Prompt: indirect_reference	4,384	0.5537	0.0511	0.0078	0.5217	0.0965
Prompt: revise	4,444	0.5169	0.0131	0.0090	0.5020	0.0255
Prompt: rewrite	4,106	0.4886	0.0141	0.0088	0.5027	0.0276
❗ Model: Hy3-NVFP4-FP8 (Temp: 0.9)	3,564	0.4706	0.0129	0.0101	0.5014	0.0252
✔️ Model: Llama-4-Scout-17B-16E-Instruct-NVFP4 (Temp: 0.6)	3,558	0.6609	0.1113	0.0084	0.5514	0.1988
Model: Mixtral-8x7B-Instruct-v0.1 (Temp: 0.7)	3,508	0.5411	0.0359	0.0057	0.5151	0.0690
Model: claude-sonnet-5 (Temp: Unknown)	3,566	0.4798	0.0090	0.0073	0.5008	0.0177
Model: gpt-5.6-luna (Temp: Unknown)	3,482	0.4865	0.0075	0.0092	0.4991	0.0147

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of 0.0275.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image]

Classifier: Top-p Outliers (Llama-3.2-3B)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	17,678	0.6123	0.0050	0.0041	0.5005	0.0099
Prompt: direct_reference	4,744	0.6784	0.0076	0.0030	0.5023	0.0150
Prompt: indirect_reference	4,384	0.6671	0.0050	0.0050	0.5000	0.0099
Prompt: revise	4,444	0.5991	0.0032	0.0050	0.4991	0.0062
Prompt: rewrite	4,106	0.4907	0.0039	0.0034	0.5002	0.0077
❗ Model: Hy3-NVFP4-FP8 (Temp: 0.9)	3,564	0.4581	0.0034	0.0022	0.5006	0.0067
✔️ Model: Llama-4-Scout-17B-16E-Instruct-NVFP4 (Temp: 0.6)	3,558	0.6949	0.0056	0.0045	0.5006	0.0111
Model: Mixtral-8x7B-Instruct-v0.1 (Temp: 0.7)	3,508	0.6277	0.0108	0.0051	0.5029	0.0213
Model: claude-sonnet-5 (Temp: Unknown)	3,566	0.6107	0.0022	0.0028	0.4997	0.0045
Model: gpt-5.6-luna (Temp: Unknown)	3,482	0.6735	0.0029	0.0057	0.4986	0.0057

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of 0.0034.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image]

Classifier: Top-k Outliers (Llama-3.2-3B-Instruct)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	17,678	0.6128	0.0428	0.0025	0.5201	0.0818
Prompt: direct_reference	4,744	0.6604	0.0788	0.0025	0.5382	0.1458
Prompt: indirect_reference	4,384	0.6762	0.0744	0.0014	0.5365	0.1383
Prompt: revise	4,444	0.6207	0.0095	0.0036	0.5029	0.0187
Prompt: rewrite	4,106	0.4835	0.0034	0.0024	0.5005	0.0068
❗ Model: Hy3-NVFP4-FP8 (Temp: 0.9)	3,564	0.3989	0.0011	0.0017	0.4997	0.0022
✔️ Model: Llama-4-Scout-17B-16E-Instruct-NVFP4 (Temp: 0.6)	3,558	0.7201	0.1203	0.0034	0.5585	0.2141
Model: Mixtral-8x7B-Instruct-v0.1 (Temp: 0.7)	3,508	0.6834	0.0787	0.0029	0.5379	0.1455
Model: claude-sonnet-5 (Temp: Unknown)	3,566	0.6154	0.0062	0.0022	0.5020	0.0122
Model: gpt-5.6-luna (Temp: Unknown)	3,482	0.6488	0.0075	0.0023	0.5026	0.0148

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of 0.0172.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image]

Classifier: Top-k Outliers (Llama-3.2-3B)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	17,678	0.5792	0.0009	0.0007	0.5001	0.0018
Prompt: direct_reference	4,744	0.6190	0.0021	0.0004	0.5008	0.0042
Prompt: indirect_reference	4,384	0.6465	0.0014	0.0009	0.5002	0.0027
Prompt: revise	4,444	0.5814	0.0000	0.0009	0.4995	0.0000
Prompt: rewrite	4,106	0.4599	0.0000	0.0005	0.4998	0.0000
❗ Model: Hy3-NVFP4-FP8 (Temp: 0.9)	3,564	0.3750	0.0000	0.0006	0.4997	0.0000
✔️ Model: Llama-4-Scout-17B-16E-Instruct-NVFP4 (Temp: 0.6)	3,558	0.6842	0.0017	0.0000	0.5008	0.0034
Model: Mixtral-8x7B-Instruct-v0.1 (Temp: 0.7)	3,508	0.6260	0.0023	0.0017	0.5003	0.0045
Model: claude-sonnet-5 (Temp: Unknown)	3,566	0.5830	0.0000	0.0000	0.5000	0.0000
Model: gpt-5.6-luna (Temp: Unknown)	3,482	0.6312	0.0006	0.0011	0.4997	0.0011

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of 0.0012.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image]

Classifier: FastDetectGPT (Llama-3.2-3B-Instruct)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	17,678	0.4990	0.0038	0.0069	0.4985	0.0076
Prompt: direct_reference	4,744	0.5083	0.0030	0.0072	0.4979	0.0058
Prompt: indirect_reference	4,384	0.4996	0.0032	0.0100	0.4966	0.0063
Prompt: revise	4,444	0.4786	0.0041	0.0054	0.4993	0.0080
Prompt: rewrite	4,106	0.5096	0.0054	0.0049	0.5002	0.0106
Model: Hy3-NVFP4-FP8 (Temp: 0.9)	3,564	0.5026	0.0090	0.0101	0.4994	0.0176
✔️ Model: Llama-4-Scout-17B-16E-Instruct-NVFP4 (Temp: 0.6)	3,558	0.6394	0.0051	0.0045	0.5003	0.0100
Model: Mixtral-8x7B-Instruct-v0.1 (Temp: 0.7)	3,508	0.4885	0.0006	0.0068	0.4969	0.0011
Model: claude-sonnet-5 (Temp: Unknown)	3,566	0.4498	0.0017	0.0062	0.4978	0.0033
❗ Model: gpt-5.6-luna (Temp: Unknown)	3,482	0.4121	0.0029	0.0069	0.4980	0.0057

Thresholding:

—Direction: higher_is_ai
—Swept for fpr_0_5pct with a found threshold of 3.3428.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image]

Classifier: FastDetectGPT (Llama-3.2-3B)

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	17,678	0.3950	0.0089	0.0037	0.5026	0.0177
Prompt: direct_reference	4,744	0.3186	0.0067	0.0021	0.5023	0.0134
Prompt: indirect_reference	4,384	0.3235	0.0068	0.0055	0.5007	0.0135
Prompt: revise	4,444	0.4104	0.0095	0.0036	0.5029	0.0187
Prompt: rewrite	4,106	0.5407	0.0132	0.0039	0.5046	0.0259
✔️ Model: Hy3-NVFP4-FP8 (Temp: 0.9)	3,564	0.5726	0.0202	0.0034	0.5084	0.0395
❗ Model: Llama-4-Scout-17B-16E-Instruct-NVFP4 (Temp: 0.6)	3,558	0.2700	0.0039	0.0028	0.5006	0.0078
Model: Mixtral-8x7B-Instruct-v0.1 (Temp: 0.7)	3,508	0.4319	0.0108	0.0040	0.5034	0.0213
Model: claude-sonnet-5 (Temp: Unknown)	3,566	0.3822	0.0062	0.0034	0.5014	0.0122
Model: gpt-5.6-luna (Temp: Unknown)	3,482	0.3175	0.0034	0.0052	0.4991	0.0068

Thresholding:

—Direction: lower_is_ai
—Swept for fpr_0_5pct with a found threshold of -3.1816.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image]

Classifier: Binoculars

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	17,678	0.6085	0.0303	0.0091	0.5106	0.0583
Prompt: direct_reference	4,744	0.6270	0.0266	0.0067	0.5099	0.0514
Prompt: indirect_reference	4,384	0.5991	0.0233	0.0105	0.5064	0.0450
Prompt: revise	4,444	0.6136	0.0275	0.0090	0.5092	0.0530
Prompt: rewrite	4,106	0.5922	0.0453	0.0102	0.5175	0.0858
Model: Hy3-NVFP4-FP8 (Temp: 0.9)	3,564	0.6375	0.0494	0.0107	0.5194	0.0932
❗ Model: Llama-4-Scout-17B-16E-Instruct-NVFP4 (Temp: 0.6)	3,558	0.5476	0.0107	0.0079	0.5014	0.0210
✔️ Model: Mixtral-8x7B-Instruct-v0.1 (Temp: 0.7)	3,508	0.6505	0.0530	0.0068	0.5231	0.1001
Model: claude-sonnet-5 (Temp: Unknown)	3,566	0.6049	0.0174	0.0079	0.5048	0.0339
Model: gpt-5.6-luna (Temp: Unknown)	3,482	0.6026	0.0213	0.0121	0.5046	0.0411

Thresholding:

—Direction: higher_is_ai
—Swept for fpr_0_5pct with a found threshold of 1.0087.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image] [image] [image] [image] [image]