Datasetpublic

G-reen/ai-det-test-human-refined-stat-no-labels-test

Auto-Generated FastDetector Dataset Dataset: G-reen/ai-det-test-human-refined-stat-no-labels-test Globals Config: config/globals_aidet_gemma_no_labels.toml Analysis Config: config/analysis_nofilter.toml Rows: 1,677 Evaluation Results Prompt Subsets: 4 (direct_reference, indirect_reference, revise, rewrite) Generator Configs: 1 (gemma-4-31B-it-AWQ-4bit (Temp: 1.0)) Classifiers: 13 (EditLens Roberta-Large Score, EditLens Roberta-Large Bucket, Perplexity… See the full description on the dataset page: https://huggingface.co/datasets/G-reen/ai-det-test-human-refined-stat-no-labels-test.

sourceHugging Faceupdated 19d agoView on Hugging Face

0likes233downloads

Dataset Card

Auto-Generated FastDetector Dataset

—Dataset: G-reen/ai-det-test-human-refined-stat-no-labels-test
—Globals Config: config/globals_aidet_gemma_no_labels.toml
—Analysis Config: config/analysis_nofilter.toml
—Rows: 1,677

Evaluation Results

—Prompt Subsets: 4 (directreference, indirectreference, revise, rewrite)
—Generator Configs: 1 (gemma-4-31B-it-AWQ-4bit (Temp: 1.0))
—Classifiers: 13 (EditLens Roberta-Large Score, EditLens Roberta-Large Bucket, Perplexity (Llama-3.2-3B-Instruct) [skipped], Perplexity (Llama-3.2-3B) [skipped], Entropy (Llama-3.2-3B-Instruct) [skipped], Entropy (Llama-3.2-3B) [skipped], Top-p Outliers (Llama-3.2-3B-Instruct) [skipped], Top-p Outliers (Llama-3.2-3B) [skipped], Top-k Outliers (Llama-3.2-3B-Instruct) [skipped], Top-k Outliers (Llama-3.2-3B) [skipped], FastDetectGPT (Llama-3.2-3B-Instruct) [skipped], FastDetectGPT (Llama-3.2-3B) [skipped], Binoculars [skipped])
—Filter Conditions: None
—Evaluation / Validation Rows: 1,509 / 168 (validation_size = 0.1)
—Base Columns: original (Human), final_response (AI)

The best classifier was EditLens Roberta-Large Score with an AUROC of 0.8539. The hardest prompt subset was rewrite with a TPR of 0.0792, and the hardest generator config was gemma-4-31B-it-AWQ-4bit (Temp: 1.0) with a TPR of 0.4619.

Classifier	Threshold	AUROC	TPR	FPR	Accuracy	F1
✔️ EditLens Roberta-Large Score	0.4141	0.8539	0.3512	0.0060	0.6726	0.5176
❗ EditLens Roberta-Large Bucket	0.0000	0.7710	0.5726	0.0371	0.7677	0.7114

Classifier metrics averaged within each prompt and generator subset:

Subset	Average AUROC	Average TPR	Average FPR	Average Accuracy	Average F1
✔️ Prompt: revise	0.9637	0.7449	0.0269	0.8590	0.8294
Prompt: indirect_reference	0.8879	0.5080	0.0120	0.7480	0.6542
Model: gemma-4-31B-it-AWQ-4bit (Temp: 1.0)	0.8124	0.4619	0.0215	0.7202	0.6145
Prompt: direct_reference	0.7921	0.4947	0.0251	0.7348	0.6481
❗ Prompt: rewrite	0.5939	0.0792	0.0219	0.5287	0.1399

✔️ marks the best AUROC, ❗ the worst.

Statistics of Interest

[image] [image] [image] [image]

Appendix

Table of contents 1. Univariate Analysis 2. Correlation Heatmap 3. Distance Histograms 4. Distance Histograms per Prompt Subset 5. Distance Histograms per Generator Config Subset 6. Classifier: EditLens Roberta-Large Score - Performance: - Thresholding: - Classification Histograms: 7. Classifier: EditLens Roberta-Large Bucket - Performance: - Thresholding: - Classification Histograms:

Univariate Analysis

Every statistic the report does arithmetic on, over the 1,509-row evaluation split. Invalid counts rows whose value is missing or non-finite; those rows are excluded from the other columns.

Statistic	N	Mean	Median	Std	Min	Max
jaccard_1	1,509	0.5834	0.7137	0.2997	0.0000	0.9606
jaccard_2	1,509	0.6951	0.8848	0.3354	0.0000	1.0000
levenshtein	1,509	1944.6342	1515.0000	1755.6878	1.0000	20872.0000
cosdist	1,509	0.1891	0.1050	0.1936	-0.0066	0.7710
EditLens Roberta-Large Score (Human)	1,509	0.0455	0.0202	0.0718	0.0064	0.9844
EditLens Roberta-Large Score (AI)	1,509	0.3686	0.2957	0.3309	0.0068	0.9996
EditLens Roberta-Large Bucket (Human)	1,509	0.0424	0.0000	0.2377	0.0000	3.0000
EditLens Roberta-Large Bucket (AI)	1,509	1.0053	1.0000	1.1356	0.0000	3.0000

Correlation Heatmap

Pearson correlation between every statistic of interest, computed over the rows where both statistics are present.

[image]

Distance Histograms

[image] [image] [image] [image]

Distance Histograms per Prompt Subset

[image] [image] [image] [image]

Distance Histograms per Generator Config Subset

[image] [image] [image] [image]

Classifier: EditLens Roberta-Large Score

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	3,018	0.8539	0.3512	0.0060	0.6726	0.5176
Prompt: direct_reference	756	0.8208	0.4286	0.0026	0.7130	0.5989
Prompt: indirect_reference	750	0.9544	0.3547	0.0000	0.6773	0.5236
✔️ Prompt: revise	780	0.9847	0.5718	0.0103	0.7808	0.7229
❗ Prompt: rewrite	732	0.6413	0.0328	0.0109	0.5109	0.0628
Model: gemma-4-31B-it-AWQ-4bit (Temp: 1.0)	3,018	0.8539	0.3512	0.0060	0.6726	0.5176

Thresholding:

—Direction: higher_is_ai
—Swept for fpr_0_5pct with a found threshold of 0.4141.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image]

Classifier: EditLens Roberta-Large Bucket

Performance:

Subset	N	AUROC	TPR	FPR	Accuracy	F1
Overall	3,018	0.7710	0.5726	0.0371	0.7677	0.7114
Prompt: direct_reference	756	0.7635	0.5608	0.0476	0.7566	0.6974
Prompt: indirect_reference	750	0.8213	0.6613	0.0240	0.8187	0.7848
✔️ Prompt: revise	780	0.9426	0.9179	0.0436	0.9372	0.9359
❗ Prompt: rewrite	732	0.5464	0.1257	0.0328	0.5464	0.2170
Model: gemma-4-31B-it-AWQ-4bit (Temp: 1.0)	3,018	0.7710	0.5726	0.0371	0.7677	0.7114

Thresholding:

—Direction: higher_is_ai
—Swept for f1 with a found threshold of 0.0000.

[image]

Classification Histograms:

[image]

Per Prompt Subset

[image] [image] [image] [image]

Per Generator Config Subset

[image]