Datasetpublic

G-reen/encoder-decoder-trial-stat

Encoder/decoder trial: encoder-marginal report Dataset: G-reen/encoder-decoder-trial-stat Rows analysed: 122,933 (every kept (encoder, decoder, source row) triple; source G-reen/cc-re-2021-filtered shard 0, 2000 rows of at most 4000 words) Prompt file: prompts/indirect_reference_dataset_train.json (turn 0 encodes the document, turn 1 reconstructs it from the encoding alone) Encoders: 9 (granite-4.2-30b-nvfp4 [0], Ornith-1.5-35B-A3B-NVFP4 [1], Llama-3.3-70B-Instruct-NVFP4 [2]… See the full description on the dataset page: https://huggingface.co/datasets/G-reen/encoder-decoder-trial-stat.

sourceHugging Faceupdated 7d agoView on Hugging Face

0likes606downloads

Dataset Card

Encoder/decoder trial: encoder-marginal report

—Dataset: G-reen/encoder-decoder-trial-stat
—Rows analysed: 122,933 (every kept (encoder, decoder, source row) triple; source G-reen/cc-re-2021-filtered shard 0, 2000 rows of at most 4000 words)
—Prompt file: prompts/indirect_reference_dataset_train.json (turn 0 encodes the document, turn 1 reconstructs it from the encoding alone)
—Encoders: 9 (granite-4.2-30b-nvfp4 [0], Ornith-1.5-35B-A3B-NVFP4 [1], Llama-3.3-70B-Instruct-NVFP4 [2], Qwen3.8-27B-AWQ-INT4 [3], Mistral-Small-4-119B-2603-NVFP4 [4], gemma-4-31B-it-AWQ-4bit [5], Laguna-S-2.1-NVFP4 [6], claude-sonnet-5 [8], gpt-5.6-terra [9])
—Decoders: 7 (granite-4.2-30b-nvfp4 [0], Ornith-1.5-35B-A3B-NVFP4 [1], Llama-3.3-70B-Instruct-NVFP4 [2], Qwen3.8-27B-AWQ-INT4 [3], Mistral-Small-4-119B-2603-NVFP4 [4], gemma-4-31B-it-AWQ-4bit [5], Laguna-S-2.1-NVFP4 [6])

Models (trial index: config):

—0: config/gen/train/shard_0.toml
—1: config/gen/train/shard_1.toml
—2: config/gen/train/shard_2.toml
—3: config/gen/train/shard_3.toml
—4: config/gen/train/shard_4.toml
—5: config/gen/train/shard_5.toml
—6: config/gen/train/shard_6.toml
—7: config/gen/train/shard_7.toml (decode only)
—8: config/encdec/claude_sonnet_5.toml (encode only)
—9: config/encdec/gpt_5_6_terra.toml (encode only)

Statistics:

—every configured statistic was present.

Summary: per-encoder means, marginalised over decoders

Every encoder encoded the same 2000 rows and every decoder decoded all of every encoder's encodings (the same source rows for every encoder), so each encoder's row is an average over the same decoders and source texts. Values are the decoder-balanced means (mean of the per-decoder means); Rows counts the kept rows behind each. For reference, the human source texts score 0.0615 (std 0.1016) on the same EditLens model.

Encoder	Editlens Score	Editlens Bucket	Cosdist	Jaccard 1	Levenshtein	Softngram	Bertscore	Moverscore	Reranker	Rows
granite-4.2-30b-nvfp4 [0]	0.5099	1.4517	0.1582	0.6346	2105.3807	0.5588	0.1327	0.5235	-5.1181	13,601
Ornith-1.5-35B-A3B-NVFP4 [1]	0.4636	1.2975	0.1559	0.6168	2042.6838	0.5054	0.1265	0.5114	-5.1840	13,615
✔️ Llama-3.3-70B-Instruct-NVFP4 [2]	0.5696	1.6533	0.2060	0.6611	2132.4408	0.5877	0.1448	0.5472	-3.3607	13,723
Qwen3.8-27B-AWQ-INT4 [3]	0.5102	1.4585	0.1580	0.6467	2065.3474	0.5613	0.1336	0.5289	-5.2792	13,639
Mistral-Small-4-119B-2603-NVFP4 [4]	0.5443	1.5684	0.1761	0.6657	2237.7226	0.5832	0.1415	0.5456	-4.6743	13,731
gemma-4-31B-it-AWQ-4bit [5]	0.5160	1.4680	0.1720	0.6694	2073.2000	0.5851	0.1410	0.5459	-4.9291	13,509
Laguna-S-2.1-NVFP4 [6]	0.5377	1.5442	0.1788	0.6693	2309.0983	0.6054	0.1424	0.5484	-4.5894	13,662
❗ claude-sonnet-5 [8]	0.4449	1.2363	0.1315	0.6124	2086.9193	0.5250	0.1214	0.5002	-5.8408	13,732
gpt-5.6-terra [9]	0.4816	1.3664	0.1365	0.6467	2231.6794	0.5662	0.1312	0.5254	-5.7724	13,721

✔️ marks the highest EditLens score (most AI-like reconstructions), ❗ the lowest.

Kept rows per encoder x decoder (after the decoders' post-processing):

Encoder \ Decoder	Granite-4.2-30B-Nvfp4 [0]	Ornith-1.5-35B-A3B-Nvfp4 [1]	Llama-3.3-70B-Instruct-Nvfp4 [2]	Qwen3.8-27B-Awq-Int4 [3]	Mistral-Small-4-119B-2603-Nvfp4 [4]	Gemma-4-31B-It-Awq-4Bit [5]	Laguna-S-2.1-Nvfp4 [6]
granite-4.2-30b-nvfp4 [0]	1,936	1,940	1,979	1,929	1,946	1,945	1,926
Ornith-1.5-35B-A3B-NVFP4 [1]	1,920	1,947	1,970	1,946	1,945	1,959	1,928
Llama-3.3-70B-Instruct-NVFP4 [2]	1,935	1,964	1,984	1,969	1,952	1,974	1,945
Qwen3.8-27B-AWQ-INT4 [3]	1,934	1,959	1,974	1,921	1,958	1,960	1,933
Mistral-Small-4-119B-2603-NVFP4 [4]	1,939	1,960	1,982	1,963	1,971	1,970	1,946
gemma-4-31B-it-AWQ-4bit [5]	1,901	1,939	1,966	1,924	1,939	1,925	1,915
Laguna-S-2.1-NVFP4 [6]	1,936	1,948	1,981	1,954	1,953	1,963	1,927
claude-sonnet-5 [8]	1,942	1,959	1,987	1,965	1,957	1,970	1,952
gpt-5.6-terra [9]	1,950	1,957	1,982	1,959	1,970	1,964	1,939

Decoder post-processing:

—Decoder 0 (granite-4.2-30b-nvfp4): 17,393 kept / 603 trashed of 17,996; last pass decoded 9,000 rows; failed requests 6; runtime 4.2 h (rejection reasons, last pass: empty or too short: 83, refusal: 12, filler output: 4, unfilled placeholder: 93, task meta-commentary: 14, echoed instruction: 10, identical to source: 18)
—Decoder 1 (Ornith-1.5-35B-A3B-NVFP4): 17,573 kept / 423 trashed of 17,996; last pass decoded 15,296 rows; failed requests 2; runtime 2.2 h (rejection reasons, last pass: empty or too short: 23, refusal: 71, filler output: 25, unfilled placeholder: 85, task meta-commentary: 101, echoed instruction: 2, identical to source: 62)
—Decoder 2 (Llama-3.3-70B-Instruct-NVFP4): 17,805 kept / 191 trashed of 17,996; last pass decoded 15,296 rows; failed requests 2; runtime 9.7 h (rejection reasons, last pass: empty or too short: 45, refusal: 19, filler output: 3, unfilled placeholder: 53, task meta-commentary: 8, echoed instruction: 14, identical to source: 36)
—Decoder 3 (Qwen3.8-27B-AWQ-INT4): 17,530 kept / 466 trashed of 17,996; last pass decoded 9,000 rows; failed requests 4; runtime 5.9 h (rejection reasons, last pass: empty or too short: 85, refusal: 29, filler output: 4, unfilled placeholder: 69, task meta-commentary: 26, echoed instruction: 3, identical to source: 23)
—Decoder 4 (Mistral-Small-4-119B-2603-NVFP4): 17,591 kept / 405 trashed of 17,996; last pass decoded 9,000 rows; failed requests 0; runtime 1.6 h (rejection reasons, last pass: empty or too short: 50, refusal: 2, filler output: 7, unfilled placeholder: 151, task meta-commentary: 7, echoed instruction: 2, identical to source: 21)
—Decoder 5 (gemma-4-31B-it-AWQ-4bit): 17,630 kept / 366 trashed of 17,996; last pass decoded 9,000 rows; failed requests 0; runtime 6.3 h (rejection reasons, last pass: empty or too short: 37, refusal: 104, filler output: 2, unfilled placeholder: 79, task meta-commentary: 24, echoed instruction: 1, identical to source: 20)
—Decoder 6 (Laguna-S-2.1-NVFP4): 17,411 kept / 585 trashed of 17,996; last pass decoded 9,000 rows; failed requests 5; runtime 2.4 h (rejection reasons, last pass: empty or too short: 35, refusal: 36, filler output: 5, unfilled placeholder: 117, task meta-commentary: 27, echoed instruction: 5, identical to source: 19)

Encoding length per encoder

Length of each encoder's turn-0 output over all of its encodings, in whitespace-separated words and in characters. Empty encodings (failed requests) are counted in Empty and left out of the means. A run of emoji without spaces counts as one word, so the character column is given as well. For reference, the source texts average 569 words.

Encoder	Encodings	Mean Words	Median Words	Mean Characters	Empty
granite-4.2-30b-nvfp4 [0]	2,000	417.1670	248.0000	2776.8305	0
✔️ Ornith-1.5-35B-A3B-NVFP4 [1]	2,000	509.1720	287.0000	3291.3920	0
Llama-3.3-70B-Instruct-NVFP4 [2]	2,000	326.9910	208.0000	2079.2785	0
Qwen3.8-27B-AWQ-INT4 [3]	2,000	386.0980	247.0000	2522.8660	0
Mistral-Small-4-119B-2603-NVFP4 [4]	2,000	364.5440	220.5000	2368.4520	0
❗ gemma-4-31B-it-AWQ-4bit [5]	2,000	314.0585	201.0000	1972.3145	0
Laguna-S-2.1-NVFP4 [6]	2,000	396.7795	239.0000	2507.4320	0
claude-sonnet-5 [8]	1,996	446.6914	359.0000	2927.3612	4
gpt-5.6-terra [9]	2,000	488.0585	254.5000	3363.1065	0

✔️ marks the longest encodings on average, ❗ the shortest.

Mean words per encoding family:

Encoder	Descriptive	Partial	Translation	Prompt
granite-4.2-30b-nvfp4 [0]	521.5380	433.8340	571.5000	141.7960
Ornith-1.5-35B-A3B-NVFP4 [1]	656.6380	498.4680	708.1940	173.3880
Llama-3.3-70B-Instruct-NVFP4 [2]	332.6500	328.9360	589.1860	57.1920
Qwen3.8-27B-AWQ-INT4 [3]	422.1640	373.2620	576.4660	172.5000
Mistral-Small-4-119B-2603-NVFP4 [4]	434.8280	359.7780	550.2180	113.3520
gemma-4-31B-it-AWQ-4bit [5]	251.1440	297.6640	590.8420	116.5840
Laguna-S-2.1-NVFP4 [6]	388.5400	468.5040	563.9320	166.1420
claude-sonnet-5 [8]	501.8998	421.5714	595.3340	267.9200
gpt-5.6-terra [9]	895.3940	325.9700	583.7060	147.1640

Detection: TPR at 0.1% FPR and AUROC

EditLens used as a detector. The threshold is calibrated on the whole human pool of G-reen/cc-re-2021-filtered: 387,327 human texts (20 shards, checkpoint pangram/editlens_roberta-large). At a 0.1% false positive budget the threshold is 0.9570: a text counts as AI when its score is above it, which flags 0.100% of the human pool. As a check, 0.10% of the trial's own 2,000 source texts score above it. TPR is the share of decoded texts above the threshold; AUROC ranks each slice of decoded texts against the same human pool.

Detection by encoding family

Slice	AUROC	TPR	N
All decoded texts	0.9155	0.2572	122,933
All except translation	0.9312	0.3433	91,515
descriptive	0.9646	0.4018	30,480
partial	0.8614	0.1825	30,909
translation	0.8701	0.0064	31,418
prompt	0.9688	0.4492	30,126

Detection by encoder

TPR Decoder Balanced is the mean of the per-decoder TPRs.

Encoder	AUROC	TPR	TPR Decoder Balanced	N
granite-4.2-30b-nvfp4 [0]	0.9162	0.2635	0.2636	13,601
Ornith-1.5-35B-A3B-NVFP4 [1]	0.9048	0.2174	0.2176	13,615
✔️ Llama-3.3-70B-Instruct-NVFP4 [2]	0.9179	0.3580	0.3582	13,723
Qwen3.8-27B-AWQ-INT4 [3]	0.9218	0.2362	0.2363	13,639
Mistral-Small-4-119B-2603-NVFP4 [4]	0.9219	0.3055	0.3056	13,731
gemma-4-31B-it-AWQ-4bit [5]	0.9176	0.2593	0.2594	13,509
Laguna-S-2.1-NVFP4 [6]	0.9213	0.2890	0.2891	13,662
❗ claude-sonnet-5 [8]	0.9042	0.1681	0.1683	13,732
gpt-5.6-terra [9]	0.9142	0.2178	0.2179	13,721

✔️ marks the highest TPR (easiest to detect), ❗ the lowest.

AUROC per encoder and encoding family:

Encoder	Descriptive	Partial	Translation	Prompt
granite-4.2-30b-nvfp4 [0]	0.9638	0.8716	0.8653	0.9665
Ornith-1.5-35B-A3B-NVFP4 [1]	0.9401	0.8680	0.8642	0.9490
Llama-3.3-70B-Instruct-NVFP4 [2]	0.9823	0.8504	0.8505	0.9921
Qwen3.8-27B-AWQ-INT4 [3]	0.9702	0.8794	0.8796	0.9596
Mistral-Small-4-119B-2603-NVFP4 [4]	0.9777	0.8473	0.8879	0.9777
gemma-4-31B-it-AWQ-4bit [5]	0.9778	0.8604	0.8709	0.9675
Laguna-S-2.1-NVFP4 [6]	0.9684	0.8754	0.8649	0.9787
claude-sonnet-5 [8]	0.9500	0.8378	0.8659	0.9653
gpt-5.6-terra [9]	0.9512	0.8632	0.8815	0.9626

TPR at 0.1% FPR per encoder and encoding family:

Encoder	Descriptive	Partial	Translation	Prompt
granite-4.2-30b-nvfp4 [0]	0.3871	0.1555	0.0057	0.5176
Ornith-1.5-35B-A3B-NVFP4 [1]	0.3324	0.1452	0.0069	0.3937
Llama-3.3-70B-Instruct-NVFP4 [2]	0.5504	0.2008	0.0060	0.6897
Qwen3.8-27B-AWQ-INT4 [3]	0.3961	0.1889	0.0057	0.3617
Mistral-Small-4-119B-2603-NVFP4 [4]	0.4726	0.1888	0.0046	0.5684
gemma-4-31B-it-AWQ-4bit [5]	0.4395	0.2285	0.0072	0.3788
Laguna-S-2.1-NVFP4 [6]	0.4598	0.2183	0.0049	0.4830
claude-sonnet-5 [8]	0.2921	0.1025	0.0114	0.2721
gpt-5.6-terra [9]	0.2879	0.2131	0.0054	0.3716

Detection by decoder

Decoder	AUROC	TPR	N
granite-4.2-30b-nvfp4 [0]	0.9263	0.2656	17,393
❗ Ornith-1.5-35B-A3B-NVFP4 [1]	0.8948	0.2068	17,573
Llama-3.3-70B-Instruct-NVFP4 [2]	0.8946	0.2494	17,805
Qwen3.8-27B-AWQ-INT4 [3]	0.9051	0.2400	17,530
Mistral-Small-4-119B-2603-NVFP4 [4]	0.9412	0.2901	17,591
gemma-4-31B-it-AWQ-4bit [5]	0.9185	0.2303	17,630
✔️ Laguna-S-2.1-NVFP4 [6]	0.9288	0.3193	17,411

Detection excluding translation

The same threshold (0.9570) and human pool, with the decoded texts of the translation family left out (detection_excluded_families in the trial config). 91,515 of 122,933 decoded texts remain.

By encoder:

Encoder	AUROC	TPR	TPR Decoder Balanced	N
granite-4.2-30b-nvfp4 [0]	0.9337	0.3524	0.3525	10,114
Ornith-1.5-35B-A3B-NVFP4 [1]	0.9188	0.2897	0.2900	10,133
✔️ Llama-3.3-70B-Instruct-NVFP4 [2]	0.9409	0.4783	0.4787	10,227
Qwen3.8-27B-AWQ-INT4 [3]	0.9364	0.3155	0.3156	10,149
Mistral-Small-4-119B-2603-NVFP4 [4]	0.9336	0.4083	0.4084	10,235
gemma-4-31B-it-AWQ-4bit [5]	0.9339	0.3473	0.3476	10,013
Laguna-S-2.1-NVFP4 [6]	0.9405	0.3862	0.3864	10,181
❗ claude-sonnet-5 [8]	0.9174	0.2217	0.2219	10,236
gpt-5.6-terra [9]	0.9253	0.2903	0.2904	10,227

✔️ marks the highest TPR (easiest to detect), ❗ the lowest.

By decoder:

Decoder	AUROC	TPR	N
granite-4.2-30b-nvfp4 [0]	0.9452	0.3556	12,906
❗ Ornith-1.5-35B-A3B-NVFP4 [1]	0.9097	0.2768	13,081
Llama-3.3-70B-Instruct-NVFP4 [2]	0.9260	0.3325	13,312
Qwen3.8-27B-AWQ-INT4 [3]	0.9077	0.3201	13,043
Mistral-Small-4-119B-2603-NVFP4 [4]	0.9491	0.3848	13,096
gemma-4-31B-it-AWQ-4bit [5]	0.9340	0.3077	13,137
✔️ Laguna-S-2.1-NVFP4 [6]	0.9467	0.4272	12,940

Detection by encoding instruction

[Family] Encoding instruction	AUROC	TPR	N
[descriptive] Create a detailed dictionary that fully describes all the meaningful phrases and words thi...	0.9287	0.1925	5,227
[descriptive] Envision the scenario in which the author wrote this text. Describe that scenario in exhau...	0.9887	0.5303	5,029
[descriptive] Reformat this text into a sensible, structured JSON object.	0.9542	0.2970	5,192
[descriptive] Translate the entirety of this text into a sequence of emojis that captures the literal me...	0.9876	0.6584	4,787
[descriptive] Write a detailed descriptor for how this text is stylistically differentiated from other t...	0.9429	0.2326	5,181
[descriptive] Write a detailed descriptor for the (imaginary) personality of the author who wrote this t...	0.9890	0.5282	5,064
[partial] Convert this text into a series of logical propositions or syllogisms that represent the c...	0.9812	0.3075	6,283
[partial] Replace every verb in this text with the tag [VERB], keeping all nouns, adjectives, articl...	0.7169	0.0333	6,073
[partial] Summarize this text, attempting to preserve as much of the original language of the text a...	0.9347	0.2352	6,196
[partial] Take this text and replace one word in every four with a single underscore (use one unders...	0.7318	0.0443	6,096
[partial] Take this text, but extract only the most meaningful sentences out of it to create a new t...	0.9351	0.2843	6,261
[prompt] Create a prompt that might cause an LLM to generate an output resembling this text.	0.9916	0.6094	6,167
[prompt] Create a prompt that would allow an LLM to generate this text as accurately/verbatim as po...	0.9787	0.3712	6,148
[prompt] Create the shortest possible prompt that can be sent to an LLM to reproduce this text. The...	0.9541	0.5091	5,806
[prompt] This piece of text was cleverely generated by an LLM with a human supervising it so that t...	0.9836	0.4198	6,091
[prompt] Write a 'few-shot' prompt using minimal, fragmented excerpts from this text as examples. T...	0.9341	0.3348	5,914
[translation] Translate the given text to French.	0.8611	0.0141	7,849
[translation] Translate the given text to German.	0.8799	0.0081	7,866
[translation] Translate the given text to Hindi.	0.8852	0.0032	7,843
[translation] Translate the given text to Spanish.	0.8543	0.0003	7,860

Rows filtered out per encoder

Decoded rows that the decoders' post-processing rejected, per encoder (summed over decoders). A rejected row may carry several reasons, so the reason columns can add up to more than Trashed. Empty Encodings At Encode counts the encoder's own failed requests (those rows were never sent to a decoder).

Encoder	Decoded	Kept	Trashed	Trashed Rate	Echoed Instruction	Empty Or Too Short	Filler Output	Identical To Source	Refusal	Task Meta-Commentary	Unfilled Placeholder	Empty Encodings At Encode
granite-4.2-30b-nvfp4 [0]	14,000	13,601	399	0.0285	10	38	9	63	68	91	149	0
Ornith-1.5-35B-A3B-NVFP4 [1]	14,000	13,615	385	0.0275	17	82	12	70	59	72	98	0
Llama-3.3-70B-Instruct-NVFP4 [2]	14,000	13,723	277	0.0198	4	21	2	18	38	63	147	0
Qwen3.8-27B-AWQ-INT4 [3]	14,000	13,639	361	0.0258	3	31	24	144	37	66	79	0
Mistral-Small-4-119B-2603-NVFP4 [4]	14,000	13,731	269	0.0192	6	13	10	9	54	73	127	0
gemma-4-31B-it-AWQ-4bit [5]	14,000	13,509	491	0.0351	9	184	20	4	88	74	190	0
Laguna-S-2.1-NVFP4 [6]	14,000	13,662	338	0.0241	1	30	21	30	51	88	138	0
claude-sonnet-5 [8]	13,972	13,732	240	0.0172	4	15	20	4	24	58	129	4
gpt-5.6-terra [9]	14,000	13,721	279	0.0199	9	26	14	39	62	56	105	0

Trashed rows per encoder x decoder:

Encoder \ Decoder	Granite-4.2-30B-Nvfp4 [0]	Ornith-1.5-35B-A3B-Nvfp4 [1]	Llama-3.3-70B-Instruct-Nvfp4 [2]	Qwen3.8-27B-Awq-Int4 [3]	Mistral-Small-4-119B-2603-Nvfp4 [4]	Gemma-4-31B-It-Awq-4Bit [5]	Laguna-S-2.1-Nvfp4 [6]
granite-4.2-30b-nvfp4 [0]	64	60	21	71	54	55	74
Ornith-1.5-35B-A3B-NVFP4 [1]	80	53	30	54	55	41	72
Llama-3.3-70B-Instruct-NVFP4 [2]	65	36	16	31	48	26	55
Qwen3.8-27B-AWQ-INT4 [3]	66	41	26	79	42	40	67
Mistral-Small-4-119B-2603-NVFP4 [4]	61	40	18	37	29	30	54
gemma-4-31B-it-AWQ-4bit [5]	99	61	34	76	61	75	85
Laguna-S-2.1-NVFP4 [6]	64	52	19	46	47	37	73
claude-sonnet-5 [8]	54	37	9	31	39	26	44
gpt-5.6-terra [9]	50	43	18	41	30	36	61

1.EditLens score (higher = more AI-like) (`final_response_editlens_score_roberta_large`)
2.EditLens bucket (higher = more AI-like) (`final_response_editlens_bucket_roberta_large`)
3.Jaccard-1 distance to the source (`jaccard_1`)
4.Jaccard-2 distance to the source (`jaccard_2`)
5.Levenshtein distance to the source (`levenshtein`)
6.Soft n-gram distance to the source (`softngram`)
7.Embedding cosine distance to the source (`cosdist`)
8.BERTScore distance to the source (`bertscore`)
9.BERTScore precision distance (`bertscore_precision`)
10.BERTScore recall distance (`bertscore_recall`)
11.MoverScore distance to the source (`moverscore`)
12.Reranker distance to the source (`reranker`)

Statistics

EditLens score (higher = more AI-like) (`final_response_editlens_score_roberta_large`)

Per encoder, marginalised over every decoder. Pooled Mean weights every kept row equally; Decoder Balanced Mean is the mean of the per-decoder means, and Spread Across Decoders their standard deviation.

Encoder	Pooled Mean	Pooled Std	Decoder Balanced Mean	Spread Across Decoders	N	Decoders
granite-4.2-30b-nvfp4 [0]	0.5097	0.3686	0.5099	0.0442	13,601	7
Ornith-1.5-35B-A3B-NVFP4 [1]	0.4634	0.3616	0.4636	0.0513	13,615	7
✔️ Llama-3.3-70B-Instruct-NVFP4 [2]	0.5693	0.3910	0.5696	0.0428	13,723	7
Qwen3.8-27B-AWQ-INT4 [3]	0.5100	0.3573	0.5102	0.0510	13,639	7
Mistral-Small-4-119B-2603-NVFP4 [4]	0.5442	0.3744	0.5443	0.0352	13,731	7
gemma-4-31B-it-AWQ-4bit [5]	0.5158	0.3681	0.5160	0.0416	13,509	7
Laguna-S-2.1-NVFP4 [6]	0.5375	0.3731	0.5377	0.0396	13,662	7
❗ claude-sonnet-5 [8]	0.4447	0.3417	0.4449	0.0549	13,732	7
gpt-5.6-terra [9]	0.4815	0.3562	0.4816	0.0477	13,721	7

[image]

Encoder x decoder cell means:

Encoder \ Decoder	granite-4.2-30b-nvfp4 [0]	Ornith-1.5-35B-A3B-NVFP4 [1]	Llama-3.3-70B-Instruct-NVFP4 [2]	Qwen3.8-27B-AWQ-INT4 [3]	Mistral-Small-4-119B-2603-NVFP4 [4]	gemma-4-31B-it-AWQ-4bit [5]	Laguna-S-2.1-NVFP4 [6]
granite-4.2-30b-nvfp4 [0]	0.5246	0.4331	0.4760	0.4990	0.5683	0.5047	0.5634
Ornith-1.5-35B-A3B-NVFP4 [1]	0.4805	0.3882	0.4221	0.4360	0.5375	0.4512	0.5297
Llama-3.3-70B-Instruct-NVFP4 [2]	0.5895	0.5133	0.5158	0.5558	0.6326	0.5639	0.6160
Qwen3.8-27B-AWQ-INT4 [3]	0.5341	0.4387	0.4735	0.4758	0.5860	0.4908	0.5722
Mistral-Small-4-119B-2603-NVFP4 [4]	0.5550	0.4919	0.5115	0.5350	0.5877	0.5332	0.5959
gemma-4-31B-it-AWQ-4bit [5]	0.5474	0.4542	0.4839	0.4957	0.5731	0.4953	0.5623
Laguna-S-2.1-NVFP4 [6]	0.5591	0.4763	0.5057	0.5270	0.5976	0.5194	0.5785
claude-sonnet-5 [8]	0.4777	0.3662	0.3898	0.4359	0.5195	0.4152	0.5100
gpt-5.6-terra [9]	0.5083	0.4054	0.4386	0.4691	0.5402	0.4666	0.5429

[image]

Per decoder, marginalised over every encoder (for contrast):

Decoder	Pooled Mean	Pooled Std	Encoder Balanced Mean	Spread Across Encoders	N
granite-4.2-30b-nvfp4 [0]	0.5307	0.3627	0.5307	0.0350	17,393
❗ Ornith-1.5-35B-A3B-NVFP4 [1]	0.4408	0.3623	0.4408	0.0459	17,573
Llama-3.3-70B-Instruct-NVFP4 [2]	0.4685	0.3819	0.4685	0.0409	17,805
Qwen3.8-27B-AWQ-INT4 [3]	0.4922	0.3652	0.4921	0.0398	17,530
✔️ Mistral-Small-4-119B-2603-NVFP4 [4]	0.5714	0.3511	0.5714	0.0329	17,591
gemma-4-31B-it-AWQ-4bit [5]	0.4934	0.3584	0.4934	0.0422	17,630
Laguna-S-2.1-NVFP4 [6]	0.5634	0.3738	0.5634	0.0308	17,411

Per encoding instruction, marginalised over every encoder and decoder:

Encoding instruction	Mean	Std	N
Convert this text into a series of logical propositions or syllogisms that represent the c...	0.6573	0.3009	6,283
Create a detailed dictionary that fully describes all the meaningful phrases and words thi...	0.5166	0.3330	5,227
Create a prompt that might cause an LLM to generate an output resembling this text.	0.8406	0.2435	6,167
Create a prompt that would allow an LLM to generate this text as accurately/verbatim as po...	0.6947	0.2984	6,148
Create the shortest possible prompt that can be sent to an LLM to reproduce this text. The...	0.7360	0.3348	5,806
Envision the scenario in which the author wrote this text. Describe that scenario in exhau...	0.7971	0.2697	5,029
Reformat this text into a sensible, structured JSON object.	0.6119	0.3335	5,192
❗ Replace every verb in this text with the tag [VERB], keeping all nouns, adjectives, articl...	0.1625	0.2208	6,073
Summarize this text, attempting to preserve as much of the original language of the text a...	0.5316	0.3415	6,196
Take this text and replace one word in every four with a single underscore (use one unders...	0.1916	0.2520	6,096
Take this text, but extract only the most meaningful sentences out of it to create a new t...	0.5736	0.3555	6,261
This piece of text was cleverely generated by an LLM with a human supervising it so that t...	0.7343	0.2903	6,091
✔️ Translate the entirety of this text into a sequence of emojis that captures the literal me...	0.8412	0.2699	4,787
Translate the given text to French.	0.2129	0.1872	7,849
Translate the given text to German.	0.2175	0.1652	7,866
Translate the given text to Hindi.	0.2248	0.1603	7,843
Translate the given text to Spanish.	0.1742	0.1259	7,860
Write a 'few-shot' prompt using minimal, fragmented excerpts from this text as examples. T...	0.6100	0.3620	5,914
Write a detailed descriptor for how this text is stylistically differentiated from other t...	0.5776	0.3326	5,181
Write a detailed descriptor for the (imaginary) personality of the author who wrote this t...	0.8114	0.2575	5,064

EditLens bucket (higher = more AI-like) (`final_response_editlens_bucket_roberta_large`)

Encoder	Pooled Mean	Pooled Std	Decoder Balanced Mean	Spread Across Decoders	N	Decoders
granite-4.2-30b-nvfp4 [0]	1.4513	1.2834	1.4517	0.1459	13,601	7
Ornith-1.5-35B-A3B-NVFP4 [1]	1.2968	1.2665	1.2975	0.1688	13,615	7
✔️ Llama-3.3-70B-Instruct-NVFP4 [2]	1.6525	1.3439	1.6533	0.1401	13,723	7
Qwen3.8-27B-AWQ-INT4 [3]	1.4582	1.2665	1.4585	0.1634	13,639	7
Mistral-Small-4-119B-2603-NVFP4 [4]	1.5681	1.2994	1.5684	0.1158	13,731	7
gemma-4-31B-it-AWQ-4bit [5]	1.4674	1.2950	1.4680	0.1366	13,509	7
Laguna-S-2.1-NVFP4 [6]	1.5437	1.3019	1.5442	0.1286	13,662	7
❗ claude-sonnet-5 [8]	1.2356	1.2144	1.2363	0.1761	13,732	7
gpt-5.6-terra [9]	1.3660	1.2578	1.3664	0.1548	13,721	7

[image]

Encoder x decoder cell means:

Encoder \ Decoder	granite-4.2-30b-nvfp4 [0]	Ornith-1.5-35B-A3B-NVFP4 [1]	Llama-3.3-70B-Instruct-NVFP4 [2]	Qwen3.8-27B-AWQ-INT4 [3]	Mistral-Small-4-119B-2603-NVFP4 [4]	gemma-4-31B-it-AWQ-4bit [5]	Laguna-S-2.1-NVFP4 [6]
granite-4.2-30b-nvfp4 [0]	1.5052	1.1995	1.3380	1.4142	1.6434	1.4334	1.6282
Ornith-1.5-35B-A3B-NVFP4 [1]	1.3490	1.0508	1.1503	1.2122	1.5429	1.2660	1.5114
Llama-3.3-70B-Instruct-NVFP4 [2]	1.7235	1.4638	1.4788	1.6074	1.8637	1.6424	1.7933
Qwen3.8-27B-AWQ-INT4 [3]	1.5476	1.2297	1.3440	1.3514	1.7079	1.3852	1.6441
Mistral-Small-4-119B-2603-NVFP4 [4]	1.6055	1.4010	1.4470	1.5420	1.7164	1.5365	1.7302
gemma-4-31B-it-AWQ-4bit [5]	1.5786	1.2702	1.3535	1.4023	1.6596	1.4005	1.6115
Laguna-S-2.1-NVFP4 [6]	1.6105	1.3393	1.4523	1.5005	1.7373	1.4885	1.6809
claude-sonnet-5 [8]	1.3404	0.9837	1.0564	1.2300	1.4798	1.1294	1.4344
gpt-5.6-terra [9]	1.4554	1.1185	1.2200	1.3313	1.5553	1.3228	1.5616

[image]

Per decoder, marginalised over every encoder (for contrast):

Decoder	Pooled Mean	Pooled Std	Encoder Balanced Mean	Spread Across Encoders	N
granite-4.2-30b-nvfp4 [0]	1.5239	1.2686	1.5240	0.1187	17,393
❗ Ornith-1.5-35B-A3B-NVFP4 [1]	1.2286	1.2696	1.2285	0.1504	17,573
Llama-3.3-70B-Instruct-NVFP4 [2]	1.3156	1.3528	1.3156	0.1370	17,805
Qwen3.8-27B-AWQ-INT4 [3]	1.3993	1.2700	1.3990	0.1268	17,530
✔️ Mistral-Small-4-119B-2603-NVFP4 [4]	1.6562	1.2271	1.6563	0.1107	17,591
gemma-4-31B-it-AWQ-4bit [5]	1.4006	1.2622	1.4005	0.1429	17,630
Laguna-S-2.1-NVFP4 [6]	1.6217	1.2995	1.6217	0.1034	17,411

Per encoding instruction, marginalised over every encoder and decoder:

Encoding instruction	Mean	Std	N
Convert this text into a series of logical propositions or syllogisms that represent the c...	1.9452	1.1293	6,283
Create a detailed dictionary that fully describes all the meaningful phrases and words thi...	1.4791	1.2074	5,227
✔️ Create a prompt that might cause an LLM to generate an output resembling this text.	2.5465	0.8898	6,167
Create a prompt that would allow an LLM to generate this text as accurately/verbatim as po...	2.0597	1.1038	6,148
Create the shortest possible prompt that can be sent to an LLM to reproduce this text. The...	2.1988	1.1656	5,806
Envision the scenario in which the author wrote this text. Describe that scenario in exhau...	2.3941	0.9896	5,029
Reformat this text into a sensible, structured JSON object.	1.7987	1.2107	5,192
Replace every verb in this text with the tag [VERB], keeping all nouns, adjectives, articl...	0.3359	0.7415	6,073
Summarize this text, attempting to preserve as much of the original language of the text a...	1.5350	1.2135	6,196
Take this text and replace one word in every four with a single underscore (use one unders...	0.4221	0.8448	6,096
Take this text, but extract only the most meaningful sentences out of it to create a new t...	1.6790	1.2669	6,261
This piece of text was cleverely generated by an LLM with a human supervising it so that t...	2.2033	1.0782	6,091
Translate the entirety of this text into a sequence of emojis that captures the literal me...	2.5402	0.9567	4,787
Translate the given text to French.	0.4640	0.7240	7,849
Translate the given text to German.	0.4652	0.6533	7,866
Translate the given text to Hindi.	0.4867	0.6452	7,843
❗ Translate the given text to Spanish.	0.3095	0.4888	7,860
Write a 'few-shot' prompt using minimal, fragmented excerpts from this text as examples. T...	1.7973	1.2894	5,914
Write a detailed descriptor for how this text is stylistically differentiated from other t...	1.6889	1.2322	5,181
Write a detailed descriptor for the (imaginary) personality of the author who wrote this t...	2.4690	0.9505	5,064

Jaccard-1 distance to the source (`jaccard_1`)

Encoder	Pooled Mean	Pooled Std	Decoder Balanced Mean	Spread Across Decoders	N	Decoders
granite-4.2-30b-nvfp4 [0]	0.6346	0.2601	0.6346	0.0258	13,601	7
Ornith-1.5-35B-A3B-NVFP4 [1]	0.6167	0.2521	0.6168	0.0279	13,615	7
Llama-3.3-70B-Instruct-NVFP4 [2]	0.6610	0.2578	0.6611	0.0161	13,723	7
Qwen3.8-27B-AWQ-INT4 [3]	0.6467	0.2441	0.6467	0.0256	13,639	7
Mistral-Small-4-119B-2603-NVFP4 [4]	0.6656	0.2437	0.6657	0.0157	13,731	7
✔️ gemma-4-31B-it-AWQ-4bit [5]	0.6694	0.2292	0.6694	0.0160	13,509	7
Laguna-S-2.1-NVFP4 [6]	0.6693	0.2403	0.6693	0.0164	13,662	7
❗ claude-sonnet-5 [8]	0.6123	0.2314	0.6124	0.0245	13,732	7
gpt-5.6-terra [9]	0.6466	0.2298	0.6467	0.0181	13,721	7

[image]

Encoder x decoder cell means:

Encoder \ Decoder	granite-4.2-30b-nvfp4 [0]	Ornith-1.5-35B-A3B-NVFP4 [1]	Llama-3.3-70B-Instruct-NVFP4 [2]	Qwen3.8-27B-AWQ-INT4 [3]	Mistral-Small-4-119B-2603-NVFP4 [4]	gemma-4-31B-it-AWQ-4bit [5]	Laguna-S-2.1-NVFP4 [6]
granite-4.2-30b-nvfp4 [0]	0.6463	0.6076	0.6239	0.6010	0.6606	0.6260	0.6772
Ornith-1.5-35B-A3B-NVFP4 [1]	0.6278	0.5895	0.6027	0.5818	0.6492	0.6055	0.6612
Llama-3.3-70B-Instruct-NVFP4 [2]	0.6699	0.6500	0.6458	0.6459	0.6817	0.6491	0.6853
Qwen3.8-27B-AWQ-INT4 [3]	0.6606	0.6232	0.6317	0.6163	0.6767	0.6321	0.6867
Mistral-Small-4-119B-2603-NVFP4 [4]	0.6693	0.6566	0.6538	0.6475	0.6784	0.6580	0.6961
gemma-4-31B-it-AWQ-4bit [5]	0.6770	0.6637	0.6540	0.6560	0.6861	0.6528	0.6964
Laguna-S-2.1-NVFP4 [6]	0.6762	0.6566	0.6545	0.6526	0.6890	0.6605	0.6960
claude-sonnet-5 [8]	0.6257	0.5912	0.5939	0.5895	0.6392	0.5939	0.6530
gpt-5.6-terra [9]	0.6586	0.6366	0.6353	0.6298	0.6601	0.6270	0.6793

[image]

Per decoder, marginalised over every encoder (for contrast):

Decoder	Pooled Mean	Pooled Std	Encoder Balanced Mean	Spread Across Encoders	N
granite-4.2-30b-nvfp4 [0]	0.6568	0.2384	0.6568	0.0184	17,393
Ornith-1.5-35B-A3B-NVFP4 [1]	0.6305	0.2512	0.6305	0.0272	17,573
Llama-3.3-70B-Instruct-NVFP4 [2]	0.6328	0.2425	0.6328	0.0212	17,805
❗ Qwen3.8-27B-AWQ-INT4 [3]	0.6245	0.2560	0.6245	0.0268	17,530
Mistral-Small-4-119B-2603-NVFP4 [4]	0.6690	0.2330	0.6690	0.0164	17,591
gemma-4-31B-it-AWQ-4bit [5]	0.6338	0.2510	0.6339	0.0221	17,630
✔️ Laguna-S-2.1-NVFP4 [6]	0.6812	0.2307	0.6813	0.0146	17,411

Per encoding instruction, marginalised over every encoder and decoder:

Encoding instruction	Mean	Std	N
Convert this text into a series of logical propositions or syllogisms that represent the c...	0.7801	0.0888	6,283
Create a detailed dictionary that fully describes all the meaningful phrases and words thi...	0.5981	0.2342	5,227
Create a prompt that might cause an LLM to generate an output resembling this text.	0.8516	0.0511	6,167
Create a prompt that would allow an LLM to generate this text as accurately/verbatim as po...	0.7967	0.0909	6,148
Create the shortest possible prompt that can be sent to an LLM to reproduce this text. The...	0.8225	0.1740	5,806
Envision the scenario in which the author wrote this text. Describe that scenario in exhau...	0.8534	0.0621	5,029
Reformat this text into a sensible, structured JSON object.	0.6797	0.1892	5,192
❗ Replace every verb in this text with the tag [VERB], keeping all nouns, adjectives, articl...	0.2187	0.1756	6,073
Summarize this text, attempting to preserve as much of the original language of the text a...	0.6119	0.2350	6,196
Take this text and replace one word in every four with a single underscore (use one unders...	0.2510	0.2141	6,096
Take this text, but extract only the most meaningful sentences out of it to create a new t...	0.7016	0.1656	6,261
This piece of text was cleverely generated by an LLM with a human supervising it so that t...	0.8229	0.0692	6,091
✔️ Translate the entirety of this text into a sequence of emojis that captures the literal me...	0.9088	0.0530	4,787
Translate the given text to French.	0.4678	0.0828	7,849
Translate the given text to German.	0.4749	0.0869	7,866
Translate the given text to Hindi.	0.5467	0.0988	7,843
Translate the given text to Spanish.	0.4175	0.0952	7,860
Write a 'few-shot' prompt using minimal, fragmented excerpts from this text as examples. T...	0.8199	0.1645	5,914
Write a detailed descriptor for how this text is stylistically differentiated from other t...	0.7838	0.0908	5,181
Write a detailed descriptor for the (imaginary) personality of the author who wrote this t...	0.8804	0.0349	5,064

Jaccard-2 distance to the source (`jaccard_2`)

Encoder	Pooled Mean	Pooled Std	Decoder Balanced Mean	Spread Across Decoders	N	Decoders
granite-4.2-30b-nvfp4 [0]	0.7686	0.2655	0.7686	0.0261	13,601	7
❗ Ornith-1.5-35B-A3B-NVFP4 [1]	0.7522	0.2543	0.7523	0.0270	13,615	7
Llama-3.3-70B-Instruct-NVFP4 [2]	0.7888	0.2464	0.7889	0.0160	13,723	7
Qwen3.8-27B-AWQ-INT4 [3]	0.7862	0.2492	0.7862	0.0247	13,639	7
Mistral-Small-4-119B-2603-NVFP4 [4]	0.8011	0.2310	0.8012	0.0141	13,731	7
✔️ gemma-4-31B-it-AWQ-4bit [5]	0.8119	0.2044	0.8120	0.0145	13,509	7
Laguna-S-2.1-NVFP4 [6]	0.8055	0.2323	0.8056	0.0149	13,662	7
claude-sonnet-5 [8]	0.7615	0.2245	0.7616	0.0229	13,732	7
gpt-5.6-terra [9]	0.7943	0.2228	0.7944	0.0159	13,721	7

[image]

Encoder x decoder cell means:

Encoder \ Decoder	granite-4.2-30b-nvfp4 [0]	Ornith-1.5-35B-A3B-NVFP4 [1]	Llama-3.3-70B-Instruct-NVFP4 [2]	Qwen3.8-27B-AWQ-INT4 [3]	Mistral-Small-4-119B-2603-NVFP4 [4]	gemma-4-31B-it-AWQ-4bit [5]	Laguna-S-2.1-NVFP4 [6]
granite-4.2-30b-nvfp4 [0]	0.7805	0.7394	0.7586	0.7356	0.7982	0.7589	0.8088
Ornith-1.5-35B-A3B-NVFP4 [1]	0.7624	0.7233	0.7415	0.7183	0.7869	0.7416	0.7920
Llama-3.3-70B-Instruct-NVFP4 [2]	0.7990	0.7783	0.7744	0.7760	0.8106	0.7730	0.8107
Qwen3.8-27B-AWQ-INT4 [3]	0.8004	0.7623	0.7725	0.7585	0.8171	0.7700	0.8226
Mistral-Small-4-119B-2603-NVFP4 [4]	0.8056	0.7942	0.7907	0.7864	0.8126	0.7904	0.8284
gemma-4-31B-it-AWQ-4bit [5]	0.8201	0.8091	0.7968	0.8040	0.8258	0.7927	0.8353
Laguna-S-2.1-NVFP4 [6]	0.8115	0.7939	0.7923	0.7923	0.8253	0.7949	0.8287
claude-sonnet-5 [8]	0.7745	0.7422	0.7436	0.7423	0.7890	0.7420	0.7977
gpt-5.6-terra [9]	0.8053	0.7868	0.7846	0.7812	0.8078	0.7738	0.8212

[image]

Per decoder, marginalised over every encoder (for contrast):

Decoder	Pooled Mean	Pooled Std	Encoder Balanced Mean	Spread Across Encoders	N
granite-4.2-30b-nvfp4 [0]	0.7955	0.2293	0.7955	0.0178	17,393
Ornith-1.5-35B-A3B-NVFP4 [1]	0.7699	0.2493	0.7699	0.0279	17,573
Llama-3.3-70B-Instruct-NVFP4 [2]	0.7728	0.2373	0.7728	0.0196	17,805
❗ Qwen3.8-27B-AWQ-INT4 [3]	0.7661	0.2594	0.7661	0.0273	17,530
Mistral-Small-4-119B-2603-NVFP4 [4]	0.8081	0.2196	0.8081	0.0135	17,591
gemma-4-31B-it-AWQ-4bit [5]	0.7708	0.2469	0.7708	0.0191	17,630
✔️ Laguna-S-2.1-NVFP4 [6]	0.8161	0.2168	0.8162	0.0139	17,411

Per encoding instruction, marginalised over every encoder and decoder:

Encoding instruction	Mean	Std	N
Convert this text into a series of logical propositions or syllogisms that represent the c...	0.9255	0.0667	6,283
Create a detailed dictionary that fully describes all the meaningful phrases and words thi...	0.7426	0.2499	5,227
Create a prompt that might cause an LLM to generate an output resembling this text.	0.9673	0.0266	6,167
Create a prompt that would allow an LLM to generate this text as accurately/verbatim as po...	0.9395	0.0595	6,148
Create the shortest possible prompt that can be sent to an LLM to reproduce this text. The...	0.9245	0.1811	5,806
Envision the scenario in which the author wrote this text. Describe that scenario in exhau...	0.9635	0.0383	5,029
Reformat this text into a sensible, structured JSON object.	0.8286	0.2053	5,192
❗ Replace every verb in this text with the tag [VERB], keeping all nouns, adjectives, articl...	0.3124	0.2106	6,073
Summarize this text, attempting to preserve as much of the original language of the text a...	0.7485	0.2589	6,196
Take this text and replace one word in every four with a single underscore (use one unders...	0.3533	0.2535	6,096
Take this text, but extract only the most meaningful sentences out of it to create a new t...	0.8228	0.1761	6,261
This piece of text was cleverely generated by an LLM with a human supervising it so that t...	0.9530	0.0421	6,091
✔️ Translate the entirety of this text into a sequence of emojis that captures the literal me...	0.9851	0.0361	4,787
Translate the given text to French.	0.6669	0.0862	7,849
Translate the given text to German.	0.6758	0.0859	7,866
Translate the given text to Hindi.	0.7480	0.0904	7,843
Translate the given text to Spanish.	0.6132	0.1066	7,860
Write a 'few-shot' prompt using minimal, fragmented excerpts from this text as examples. T...	0.9063	0.1566	5,914
Write a detailed descriptor for how this text is stylistically differentiated from other t...	0.9089	0.0773	5,181
Write a detailed descriptor for the (imaginary) personality of the author who wrote this t...	0.9775	0.0155	5,064

Levenshtein distance to the source (`levenshtein`)

Encoder	Pooled Mean	Pooled Std	Decoder Balanced Mean	Spread Across Decoders	N	Decoders
granite-4.2-30b-nvfp4 [0]	2105.0217	2445.1394	2105.3807	145.5901	13,601	7
❗ Ornith-1.5-35B-A3B-NVFP4 [1]	2041.9071	2797.1727	2042.6838	141.2548	13,615	7
Llama-3.3-70B-Instruct-NVFP4 [2]	2131.9093	2515.1605	2132.4408	105.1155	13,723	7
Qwen3.8-27B-AWQ-INT4 [3]	2065.1839	2758.4968	2065.3474	121.7832	13,639	7
Mistral-Small-4-119B-2603-NVFP4 [4]	2237.2207	2992.4775	2237.7226	140.1776	13,731	7
gemma-4-31B-it-AWQ-4bit [5]	2073.1535	2457.9207	2073.2000	96.0188	13,509	7
✔️ Laguna-S-2.1-NVFP4 [6]	2308.2526	3898.7931	2309.0983	148.3083	13,662	7
claude-sonnet-5 [8]	2085.9876	4508.4155	2086.9193	226.0749	13,732	7
gpt-5.6-terra [9]	2231.2725	4426.4117	2231.6794	182.3787	13,721	7

[image]

Encoder x decoder cell means:

Encoder \ Decoder	granite-4.2-30b-nvfp4 [0]	Ornith-1.5-35B-A3B-NVFP4 [1]	Llama-3.3-70B-Instruct-NVFP4 [2]	Qwen3.8-27B-AWQ-INT4 [3]	Mistral-Small-4-119B-2603-NVFP4 [4]	gemma-4-31B-it-AWQ-4bit [5]	Laguna-S-2.1-NVFP4 [6]
granite-4.2-30b-nvfp4 [0]	2142.1364	2149.0258	2017.1107	1949.2789	2266.4157	1898.2540	2315.4434
Ornith-1.5-35B-A3B-NVFP4 [1]	2092.6406	2028.9692	1936.7614	1885.9527	2146.8653	1900.2323	2307.3651
Llama-3.3-70B-Instruct-NVFP4 [2]	2130.6997	2194.5234	2063.4793	2048.6953	2279.5835	1966.1864	2243.9177
Qwen3.8-27B-AWQ-INT4 [3]	2106.0796	2079.4421	2021.3582	1928.7715	2151.6813	1893.8138	2276.2856
Mistral-Small-4-119B-2603-NVFP4 [4]	2298.4471	2221.7383	2154.4147	2111.7040	2388.3876	2034.9360	2454.4301
gemma-4-31B-it-AWQ-4bit [5]	2073.0352	2081.1078	2048.4593	2002.4683	2186.3847	1909.2691	2211.6757
Laguna-S-2.1-NVFP4 [6]	2430.5517	2425.7295	2221.5962	2134.4831	2412.0691	2077.6032	2461.6554
claude-sonnet-5 [8]	2173.4439	2217.3492	1941.5279	1888.4061	2049.8426	1809.8386	2528.0272
gpt-5.6-terra [9]	2389.4651	2290.8809	2351.7992	2047.5268	2169.4046	1914.2164	2458.4631

[image]

Per decoder, marginalised over every encoder (for contrast):

Decoder	Pooled Mean	Pooled Std	Encoder Balanced Mean	Spread Across Encoders	N
granite-4.2-30b-nvfp4 [0]	2204.5921	3731.8047	2204.0555	126.4551	17,393
Ornith-1.5-35B-A3B-NVFP4 [1]	2187.7551	3970.7840	2187.6407	115.0659	17,573
Llama-3.3-70B-Instruct-NVFP4 [2]	2084.1770	3389.4024	2084.0563	128.0916	17,805
Qwen3.8-27B-AWQ-INT4 [3]	1999.9944	2464.4671	1999.6985	87.1174	17,530
Mistral-Small-4-119B-2603-NVFP4 [4]	2227.9455	2532.5306	2227.8483	112.0787	17,591
❗ gemma-4-31B-it-AWQ-4bit [5]	1933.9303	2608.9887	1933.8166	76.2289	17,630
✔️ Laguna-S-2.1-NVFP4 [6]	2362.2757	3968.4034	2361.9182	107.6955	17,411

Per encoding instruction, marginalised over every encoder and decoder:

Encoding instruction	Mean	Std	N
Convert this text into a series of logical propositions or syllogisms that represent the c...	2708.4195	2321.3652	6,283
Create a detailed dictionary that fully describes all the meaningful phrases and words thi...	2036.5969	2627.1115	5,227
Create a prompt that might cause an LLM to generate an output resembling this text.	3316.2526	2860.5408	6,167
Create a prompt that would allow an LLM to generate this text as accurately/verbatim as po...	2835.4305	2569.2781	6,148
Create the shortest possible prompt that can be sent to an LLM to reproduce this text. The...	2806.7494	2901.9416	5,806
Envision the scenario in which the author wrote this text. Describe that scenario in exhau...	3323.9753	3848.3830	5,029
Reformat this text into a sensible, structured JSON object.	2593.6308	3057.3767	5,192
❗ Replace every verb in this text with the tag [VERB], keeping all nouns, adjectives, articl...	430.3017	2691.7751	6,073
Summarize this text, attempting to preserve as much of the original language of the text a...	2146.6162	4226.0453	6,196
Take this text and replace one word in every four with a single underscore (use one unders...	559.8406	1643.9632	6,096
Take this text, but extract only the most meaningful sentences out of it to create a new t...	2577.7528	2548.4190	6,261
This piece of text was cleverely generated by an LLM with a human supervising it so that t...	3104.4745	4701.8958	6,091
Translate the entirety of this text into a sequence of emojis that captures the literal me...	3359.8018	5816.8765	4,787
Translate the given text to French.	946.0595	1025.3997	7,849
Translate the given text to German.	989.9259	1223.0919	7,866
Translate the given text to Hindi.	1085.5447	1783.5637	7,843
Translate the given text to Spanish.	752.6126	643.9184	7,860
Write a 'few-shot' prompt using minimal, fragmented excerpts from this text as examples. T...	2950.4307	4422.2077	5,914
Write a detailed descriptor for how this text is stylistically differentiated from other t...	2858.1764	5017.5288	5,181
✔️ Write a detailed descriptor for the (imaginary) personality of the author who wrote this t...	3750.6929	3652.7911	5,064

Soft n-gram distance to the source (`softngram`)

Encoder	Pooled Mean	Pooled Std	Decoder Balanced Mean	Spread Across Decoders	N	Decoders
granite-4.2-30b-nvfp4 [0]	0.5588	0.3671	0.5588	0.0321	13,601	7
❗ Ornith-1.5-35B-A3B-NVFP4 [1]	0.5053	0.3597	0.5054	0.0369	13,615	7
Llama-3.3-70B-Instruct-NVFP4 [2]	0.5875	0.3806	0.5877	0.0236	13,723	7
Qwen3.8-27B-AWQ-INT4 [3]	0.5612	0.3529	0.5613	0.0336	13,639	7
Mistral-Small-4-119B-2603-NVFP4 [4]	0.5831	0.3566	0.5832	0.0219	13,731	7
gemma-4-31B-it-AWQ-4bit [5]	0.5850	0.3570	0.5851	0.0245	13,509	7
✔️ Laguna-S-2.1-NVFP4 [6]	0.6052	0.3523	0.6054	0.0232	13,662	7
claude-sonnet-5 [8]	0.5249	0.3538	0.5250	0.0339	13,732	7
gpt-5.6-terra [9]	0.5661	0.3447	0.5662	0.0280	13,721	7

[image]

Encoder x decoder cell means:

Encoder \ Decoder	granite-4.2-30b-nvfp4 [0]	Ornith-1.5-35B-A3B-NVFP4 [1]	Llama-3.3-70B-Instruct-NVFP4 [2]	Qwen3.8-27B-AWQ-INT4 [3]	Mistral-Small-4-119B-2603-NVFP4 [4]	gemma-4-31B-it-AWQ-4bit [5]	Laguna-S-2.1-NVFP4 [6]
granite-4.2-30b-nvfp4 [0]	0.5715	0.5290	0.5451	0.5177	0.5862	0.5455	0.6167
Ornith-1.5-35B-A3B-NVFP4 [1]	0.5126	0.4800	0.4925	0.4518	0.5372	0.4909	0.5729
Llama-3.3-70B-Instruct-NVFP4 [2]	0.6011	0.5743	0.5717	0.5551	0.6130	0.5735	0.6250
Qwen3.8-27B-AWQ-INT4 [3]	0.5745	0.5373	0.5422	0.5143	0.5924	0.5482	0.6201
Mistral-Small-4-119B-2603-NVFP4 [4]	0.5851	0.5703	0.5694	0.5560	0.5976	0.5760	0.6278
gemma-4-31B-it-AWQ-4bit [5]	0.5943	0.5784	0.5678	0.5507	0.6087	0.5686	0.6270
Laguna-S-2.1-NVFP4 [6]	0.6179	0.5907	0.5855	0.5775	0.6292	0.5929	0.6441
claude-sonnet-5 [8]	0.5416	0.4958	0.4988	0.4959	0.5601	0.4991	0.5838
gpt-5.6-terra [9]	0.5854	0.5493	0.5494	0.5377	0.5856	0.5386	0.6173

[image]

Per decoder, marginalised over every encoder (for contrast):

Decoder	Pooled Mean	Pooled Std	Encoder Balanced Mean	Spread Across Encoders	N
granite-4.2-30b-nvfp4 [0]	0.5760	0.3595	0.5760	0.0300	17,393
Ornith-1.5-35B-A3B-NVFP4 [1]	0.5450	0.3682	0.5450	0.0360	17,573
Llama-3.3-70B-Instruct-NVFP4 [2]	0.5469	0.3481	0.5469	0.0305	17,805
❗ Qwen3.8-27B-AWQ-INT4 [3]	0.5286	0.3670	0.5285	0.0360	17,530
Mistral-Small-4-119B-2603-NVFP4 [4]	0.5900	0.3569	0.5900	0.0262	17,591
gemma-4-31B-it-AWQ-4bit [5]	0.5481	0.3576	0.5482	0.0327	17,630
✔️ Laguna-S-2.1-NVFP4 [6]	0.6149	0.3524	0.6150	0.0212	17,411

Per encoding instruction, marginalised over every encoder and decoder:

Encoding instruction	Mean	Std	N
Convert this text into a series of logical propositions or syllogisms that represent the c...	0.7411	0.1765	6,283
Create a detailed dictionary that fully describes all the meaningful phrases and words thi...	0.5552	0.3115	5,227
Create a prompt that might cause an LLM to generate an output resembling this text.	0.9115	0.1060	6,167
Create a prompt that would allow an LLM to generate this text as accurately/verbatim as po...	0.8202	0.1903	6,148
Create the shortest possible prompt that can be sent to an LLM to reproduce this text. The...	0.7770	0.2809	5,806
Envision the scenario in which the author wrote this text. Describe that scenario in exhau...	0.9276	0.0950	5,029
Reformat this text into a sensible, structured JSON object.	0.6376	0.2604	5,192
❗ Replace every verb in this text with the tag [VERB], keeping all nouns, adjectives, articl...	0.1051	0.1730	6,073
Summarize this text, attempting to preserve as much of the original language of the text a...	0.5004	0.3040	6,196
Take this text and replace one word in every four with a single underscore (use one unders...	0.1732	0.2321	6,096
Take this text, but extract only the most meaningful sentences out of it to create a new t...	0.5652	0.2894	6,261
This piece of text was cleverely generated by an LLM with a human supervising it so that t...	0.8686	0.1397	6,091
✔️ Translate the entirety of this text into a sequence of emojis that captures the literal me...	0.9783	0.0735	4,787
Translate the given text to French.	0.2061	0.1110	7,849
Translate the given text to German.	0.2029	0.1118	7,866
Translate the given text to Hindi.	0.3047	0.1719	7,843
Translate the given text to Spanish.	0.1621	0.0921	7,860
Write a 'few-shot' prompt using minimal, fragmented excerpts from this text as examples. T...	0.7272	0.3211	5,914
Write a detailed descriptor for how this text is stylistically differentiated from other t...	0.8208	0.1449	5,181
Write a detailed descriptor for the (imaginary) personality of the author who wrote this t...	0.9684	0.0474	5,064

Embedding cosine distance to the source (`cosdist`)

Encoder	Pooled Mean	Pooled Std	Decoder Balanced Mean	Spread Across Decoders	N	Decoders
granite-4.2-30b-nvfp4 [0]	0.1582	0.1651	0.1582	0.0086	13,601	7
Ornith-1.5-35B-A3B-NVFP4 [1]	0.1559	0.1760	0.1559	0.0090	13,615	7
✔️ Llama-3.3-70B-Instruct-NVFP4 [2]	0.2061	0.2024	0.2060	0.0102	13,723	7
Qwen3.8-27B-AWQ-INT4 [3]	0.1580	0.1646	0.1580	0.0090	13,639	7
Mistral-Small-4-119B-2603-NVFP4 [4]	0.1762	0.1806	0.1761	0.0085	13,731	7
gemma-4-31B-it-AWQ-4bit [5]	0.1720	0.1695	0.1720	0.0096	13,509	7
Laguna-S-2.1-NVFP4 [6]	0.1788	0.1791	0.1788	0.0082	13,662	7
❗ claude-sonnet-5 [8]	0.1315	0.1443	0.1315	0.0094	13,732	7
gpt-5.6-terra [9]	0.1365	0.1465	0.1365	0.0096	13,721	7

[image]

Encoder x decoder cell means:

Encoder \ Decoder	granite-4.2-30b-nvfp4 [0]	Ornith-1.5-35B-A3B-NVFP4 [1]	Llama-3.3-70B-Instruct-NVFP4 [2]	Qwen3.8-27B-AWQ-INT4 [3]	Mistral-Small-4-119B-2603-NVFP4 [4]	gemma-4-31B-it-AWQ-4bit [5]	Laguna-S-2.1-NVFP4 [6]
granite-4.2-30b-nvfp4 [0]	0.1562	0.1426	0.1717	0.1587	0.1532	0.1590	0.1661
Ornith-1.5-35B-A3B-NVFP4 [1]	0.1524	0.1436	0.1692	0.1500	0.1588	0.1495	0.1676
Llama-3.3-70B-Instruct-NVFP4 [2]	0.1997	0.1879	0.2225	0.2091	0.2024	0.2067	0.2139
Qwen3.8-27B-AWQ-INT4 [3]	0.1571	0.1415	0.1706	0.1559	0.1615	0.1521	0.1672
Mistral-Small-4-119B-2603-NVFP4 [4]	0.1680	0.1619	0.1890	0.1763	0.1772	0.1762	0.1845
gemma-4-31B-it-AWQ-4bit [5]	0.1721	0.1560	0.1887	0.1682	0.1740	0.1656	0.1795
Laguna-S-2.1-NVFP4 [6]	0.1751	0.1625	0.1896	0.1809	0.1802	0.1766	0.1866
claude-sonnet-5 [8]	0.1332	0.1157	0.1420	0.1266	0.1381	0.1226	0.1422
gpt-5.6-terra [9]	0.1377	0.1238	0.1501	0.1329	0.1402	0.1240	0.1470

[image]

Per decoder, marginalised over every encoder (for contrast):

Decoder	Pooled Mean	Pooled Std	Encoder Balanced Mean	Spread Across Encoders	N
granite-4.2-30b-nvfp4 [0]	0.1612	0.1634	0.1613	0.0191	17,393
❗ Ornith-1.5-35B-A3B-NVFP4 [1]	0.1484	0.1683	0.1484	0.0205	17,573
✔️ Llama-3.3-70B-Instruct-NVFP4 [2]	0.1770	0.1841	0.1770	0.0226	17,805
Qwen3.8-27B-AWQ-INT4 [3]	0.1621	0.1730	0.1621	0.0238	17,530
Mistral-Small-4-119B-2603-NVFP4 [4]	0.1650	0.1715	0.1651	0.0194	17,591
gemma-4-31B-it-AWQ-4bit [5]	0.1591	0.1694	0.1591	0.0250	17,630
Laguna-S-2.1-NVFP4 [6]	0.1727	0.1716	0.1727	0.0205	17,411

Per encoding instruction, marginalised over every encoder and decoder:

Encoding instruction	Mean	Std	N
Convert this text into a series of logical propositions or syllogisms that represent the c...	0.1574	0.0848	6,283
Create a detailed dictionary that fully describes all the meaningful phrases and words thi...	0.1183	0.0921	5,227
Create a prompt that might cause an LLM to generate an output resembling this text.	0.2321	0.1096	6,167
Create a prompt that would allow an LLM to generate this text as accurately/verbatim as po...	0.1822	0.1116	6,148
Create the shortest possible prompt that can be sent to an LLM to reproduce this text. The...	0.2315	0.1545	5,806
Envision the scenario in which the author wrote this text. Describe that scenario in exhau...	0.2712	0.1404	5,029
Reformat this text into a sensible, structured JSON object.	0.1178	0.0733	5,192
❗ Replace every verb in this text with the tag [VERB], keeping all nouns, adjectives, articl...	0.0215	0.0535	6,073
Summarize this text, attempting to preserve as much of the original language of the text a...	0.0988	0.0787	6,196
Take this text and replace one word in every four with a single underscore (use one unders...	0.0418	0.0735	6,096
Take this text, but extract only the most meaningful sentences out of it to create a new t...	0.1571	0.0973	6,261
This piece of text was cleverely generated by an LLM with a human supervising it so that t...	0.1941	0.0977	6,091
✔️ Translate the entirety of this text into a sequence of emojis that captures the literal me...	0.6000	0.1837	4,787
Translate the given text to French.	0.0333	0.0372	7,849
Translate the given text to German.	0.0306	0.0363	7,866
Translate the given text to Hindi.	0.0550	0.0635	7,843
Translate the given text to Spanish.	0.0319	0.0330	7,860
Write a 'few-shot' prompt using minimal, fragmented excerpts from this text as examples. T...	0.3234	0.1773	5,914
Write a detailed descriptor for how this text is stylistically differentiated from other t...	0.2848	0.1359	5,181
Write a detailed descriptor for the (imaginary) personality of the author who wrote this t...	0.4019	0.1472	5,064

BERTScore distance to the source (`bertscore`)

Encoder	Pooled Mean	Pooled Std	Decoder Balanced Mean	Spread Across Decoders	N	Decoders
granite-4.2-30b-nvfp4 [0]	0.1327	0.0748	0.1327	0.0057	13,601	7
Ornith-1.5-35B-A3B-NVFP4 [1]	0.1265	0.0736	0.1265	0.0066	13,615	7
✔️ Llama-3.3-70B-Instruct-NVFP4 [2]	0.1448	0.0786	0.1448	0.0042	13,723	7
Qwen3.8-27B-AWQ-INT4 [3]	0.1335	0.0702	0.1336	0.0060	13,639	7
Mistral-Small-4-119B-2603-NVFP4 [4]	0.1415	0.0736	0.1415	0.0042	13,731	7
gemma-4-31B-it-AWQ-4bit [5]	0.1410	0.0712	0.1410	0.0042	13,509	7
Laguna-S-2.1-NVFP4 [6]	0.1424	0.0726	0.1424	0.0042	13,662	7
❗ claude-sonnet-5 [8]	0.1213	0.0663	0.1214	0.0068	13,732	7
gpt-5.6-terra [9]	0.1312	0.0672	0.1312	0.0054	13,721	7

[image]

Encoder x decoder cell means:

Encoder \ Decoder	granite-4.2-30b-nvfp4 [0]	Ornith-1.5-35B-A3B-NVFP4 [1]	Llama-3.3-70B-Instruct-NVFP4 [2]	Qwen3.8-27B-AWQ-INT4 [3]	Mistral-Small-4-119B-2603-NVFP4 [4]	gemma-4-31B-it-AWQ-4bit [5]	Laguna-S-2.1-NVFP4 [6]
granite-4.2-30b-nvfp4 [0]	0.1343	0.1259	0.1318	0.1259	0.1377	0.1305	0.1429
Ornith-1.5-35B-A3B-NVFP4 [1]	0.1271	0.1196	0.1251	0.1186	0.1337	0.1236	0.1380
Llama-3.3-70B-Instruct-NVFP4 [2]	0.1447	0.1402	0.1459	0.1396	0.1495	0.1421	0.1514
Qwen3.8-27B-AWQ-INT4 [3]	0.1349	0.1275	0.1309	0.1270	0.1405	0.1303	0.1437
Mistral-Small-4-119B-2603-NVFP4 [4]	0.1401	0.1373	0.1414	0.1367	0.1448	0.1406	0.1498
gemma-4-31B-it-AWQ-4bit [5]	0.1411	0.1379	0.1399	0.1367	0.1460	0.1370	0.1483
Laguna-S-2.1-NVFP4 [6]	0.1431	0.1379	0.1415	0.1377	0.1474	0.1398	0.1494
claude-sonnet-5 [8]	0.1237	0.1147	0.1189	0.1153	0.1283	0.1152	0.1333
gpt-5.6-terra [9]	0.1331	0.1274	0.1309	0.1262	0.1349	0.1247	0.1415

[image]

Per decoder, marginalised over every encoder (for contrast):

Decoder	Pooled Mean	Pooled Std	Encoder Balanced Mean	Spread Across Encoders	N
granite-4.2-30b-nvfp4 [0]	0.1358	0.0706	0.1358	0.0067	17,393
Ornith-1.5-35B-A3B-NVFP4 [1]	0.1298	0.0725	0.1298	0.0086	17,573
Llama-3.3-70B-Instruct-NVFP4 [2]	0.1340	0.0735	0.1340	0.0083	17,805
❗ Qwen3.8-27B-AWQ-INT4 [3]	0.1293	0.0730	0.1293	0.0083	17,530
Mistral-Small-4-119B-2603-NVFP4 [4]	0.1403	0.0702	0.1403	0.0068	17,591
gemma-4-31B-it-AWQ-4bit [5]	0.1315	0.0731	0.1315	0.0086	17,630
✔️ Laguna-S-2.1-NVFP4 [6]	0.1443	0.0731	0.1443	0.0057	17,411

Per encoding instruction, marginalised over every encoder and decoder:

Encoding instruction	Mean	Std	N
Convert this text into a series of logical propositions or syllogisms that represent the c...	0.1661	0.0365	6,283
Create a detailed dictionary that fully describes all the meaningful phrases and words thi...	0.1156	0.0562	5,227
Create a prompt that might cause an LLM to generate an output resembling this text.	0.1916	0.0285	6,167
Create a prompt that would allow an LLM to generate this text as accurately/verbatim as po...	0.1683	0.0382	6,148
Create the shortest possible prompt that can be sent to an LLM to reproduce this text. The...	0.1906	0.0575	5,806
Envision the scenario in which the author wrote this text. Describe that scenario in exhau...	0.1954	0.0325	5,029
Reformat this text into a sensible, structured JSON object.	0.1343	0.0454	5,192
❗ Replace every verb in this text with the tag [VERB], keeping all nouns, adjectives, articl...	0.0344	0.0385	6,073
Summarize this text, attempting to preserve as much of the original language of the text a...	0.1162	0.0549	6,196
Take this text and replace one word in every four with a single underscore (use one unders...	0.0452	0.0461	6,096
Take this text, but extract only the most meaningful sentences out of it to create a new t...	0.1438	0.0473	6,261
This piece of text was cleverely generated by an LLM with a human supervising it so that t...	0.1814	0.0321	6,091
✔️ Translate the entirety of this text into a sequence of emojis that captures the literal me...	0.2518	0.0558	4,787
Translate the given text to French.	0.0727	0.0219	7,849
Translate the given text to German.	0.0730	0.0234	7,866
Translate the given text to Hindi.	0.0900	0.0315	7,843
Translate the given text to Spanish.	0.0642	0.0218	7,860
Write a 'few-shot' prompt using minimal, fragmented excerpts from this text as examples. T...	0.1984	0.0629	5,914
Write a detailed descriptor for how this text is stylistically differentiated from other t...	0.1752	0.0336	5,181
Write a detailed descriptor for the (imaginary) personality of the author who wrote this t...	0.2160	0.0249	5,064

BERTScore precision distance (`bertscore_precision`)

Encoder	Pooled Mean	Pooled Std	Decoder Balanced Mean	Spread Across Decoders	N	Decoders
granite-4.2-30b-nvfp4 [0]	0.1325	0.0768	0.1325	0.0079	13,601	7
Ornith-1.5-35B-A3B-NVFP4 [1]	0.1235	0.0741	0.1236	0.0087	13,615	7
✔️ Llama-3.3-70B-Instruct-NVFP4 [2]	0.1414	0.0803	0.1414	0.0065	13,723	7
Qwen3.8-27B-AWQ-INT4 [3]	0.1315	0.0699	0.1315	0.0084	13,639	7
Mistral-Small-4-119B-2603-NVFP4 [4]	0.1392	0.0744	0.1392	0.0062	13,731	7
gemma-4-31B-it-AWQ-4bit [5]	0.1370	0.0708	0.1370	0.0064	13,509	7
Laguna-S-2.1-NVFP4 [6]	0.1414	0.0746	0.1414	0.0063	13,662	7
❗ claude-sonnet-5 [8]	0.1226	0.0699	0.1226	0.0088	13,732	7
gpt-5.6-terra [9]	0.1314	0.0693	0.1314	0.0077	13,721	7

[image]

Encoder x decoder cell means:

Encoder \ Decoder	granite-4.2-30b-nvfp4 [0]	Ornith-1.5-35B-A3B-NVFP4 [1]	Llama-3.3-70B-Instruct-NVFP4 [2]	Qwen3.8-27B-AWQ-INT4 [3]	Mistral-Small-4-119B-2603-NVFP4 [4]	gemma-4-31B-it-AWQ-4bit [5]	Laguna-S-2.1-NVFP4 [6]
granite-4.2-30b-nvfp4 [0]	0.1342	0.1240	0.1309	0.1233	0.1401	0.1284	0.1466
Ornith-1.5-35B-A3B-NVFP4 [1]	0.1238	0.1155	0.1222	0.1127	0.1320	0.1195	0.1394
Llama-3.3-70B-Instruct-NVFP4 [2]	0.1416	0.1357	0.1418	0.1330	0.1495	0.1366	0.1516
Qwen3.8-27B-AWQ-INT4 [3]	0.1331	0.1240	0.1287	0.1212	0.1402	0.1270	0.1464
Mistral-Small-4-119B-2603-NVFP4 [4]	0.1375	0.1339	0.1381	0.1324	0.1446	0.1366	0.1514
gemma-4-31B-it-AWQ-4bit [5]	0.1368	0.1329	0.1350	0.1298	0.1444	0.1316	0.1486
Laguna-S-2.1-NVFP4 [6]	0.1423	0.1358	0.1398	0.1345	0.1486	0.1365	0.1522
claude-sonnet-5 [8]	0.1251	0.1140	0.1184	0.1165	0.1308	0.1145	0.1391
gpt-5.6-terra [9]	0.1335	0.1257	0.1300	0.1246	0.1368	0.1227	0.1464

[image]

Per decoder, marginalised over every encoder (for contrast):

Decoder	Pooled Mean	Pooled Std	Encoder Balanced Mean	Spread Across Encoders	N
granite-4.2-30b-nvfp4 [0]	0.1342	0.0721	0.1342	0.0061	17,393
Ornith-1.5-35B-A3B-NVFP4 [1]	0.1268	0.0735	0.1268	0.0078	17,573
Llama-3.3-70B-Instruct-NVFP4 [2]	0.1316	0.0740	0.1316	0.0074	17,805
❗ Qwen3.8-27B-AWQ-INT4 [3]	0.1253	0.0732	0.1253	0.0073	17,530
Mistral-Small-4-119B-2603-NVFP4 [4]	0.1408	0.0730	0.1408	0.0063	17,591
gemma-4-31B-it-AWQ-4bit [5]	0.1282	0.0713	0.1282	0.0076	17,630
✔️ Laguna-S-2.1-NVFP4 [6]	0.1468	0.0765	0.1469	0.0046	17,411

Per encoding instruction, marginalised over every encoder and decoder:

Encoding instruction	Mean	Std	N
Convert this text into a series of logical propositions or syllogisms that represent the c...	0.1661	0.0385	6,283
Create a detailed dictionary that fully describes all the meaningful phrases and words thi...	0.1205	0.0605	5,227
Create a prompt that might cause an LLM to generate an output resembling this text.	0.1980	0.0336	6,167
Create a prompt that would allow an LLM to generate this text as accurately/verbatim as po...	0.1741	0.0403	6,148
Create the shortest possible prompt that can be sent to an LLM to reproduce this text. The...	0.1771	0.0587	5,806
Envision the scenario in which the author wrote this text. Describe that scenario in exhau...	0.2033	0.0377	5,029
Reformat this text into a sensible, structured JSON object.	0.1409	0.0515	5,192
❗ Replace every verb in this text with the tag [VERB], keeping all nouns, adjectives, articl...	0.0371	0.0452	6,073
Summarize this text, attempting to preserve as much of the original language of the text a...	0.1159	0.0608	6,196
Take this text and replace one word in every four with a single underscore (use one unders...	0.0485	0.0527	6,096
Take this text, but extract only the most meaningful sentences out of it to create a new t...	0.1303	0.0563	6,261
This piece of text was cleverely generated by an LLM with a human supervising it so that t...	0.1883	0.0367	6,091
✔️ Translate the entirety of this text into a sequence of emojis that captures the literal me...	0.2467	0.0614	4,787
Translate the given text to French.	0.0711	0.0211	7,849
Translate the given text to German.	0.0715	0.0226	7,866
Translate the given text to Hindi.	0.0867	0.0297	7,843
Translate the given text to Spanish.	0.0621	0.0213	7,860
Write a 'few-shot' prompt using minimal, fragmented excerpts from this text as examples. T...	0.1751	0.0666	5,914
Write a detailed descriptor for how this text is stylistically differentiated from other t...	0.1651	0.0366	5,181
Write a detailed descriptor for the (imaginary) personality of the author who wrote this t...	0.2148	0.0295	5,064

BERTScore recall distance (`bertscore_recall`)

Encoder	Pooled Mean	Pooled Std	Decoder Balanced Mean	Spread Across Decoders	N	Decoders
granite-4.2-30b-nvfp4 [0]	0.1321	0.0765	0.1321	0.0036	13,601	7
Ornith-1.5-35B-A3B-NVFP4 [1]	0.1283	0.0783	0.1283	0.0046	13,615	7
✔️ Llama-3.3-70B-Instruct-NVFP4 [2]	0.1474	0.0802	0.1474	0.0021	13,723	7
Qwen3.8-27B-AWQ-INT4 [3]	0.1346	0.0749	0.1346	0.0037	13,639	7
Mistral-Small-4-119B-2603-NVFP4 [4]	0.1430	0.0767	0.1430	0.0024	13,731	7
gemma-4-31B-it-AWQ-4bit [5]	0.1440	0.0755	0.1440	0.0021	13,509	7
Laguna-S-2.1-NVFP4 [6]	0.1426	0.0743	0.1426	0.0022	13,662	7
❗ claude-sonnet-5 [8]	0.1195	0.0657	0.1195	0.0049	13,732	7
gpt-5.6-terra [9]	0.1304	0.0686	0.1304	0.0032	13,721	7

[image]

Encoder x decoder cell means:

Encoder \ Decoder	granite-4.2-30b-nvfp4 [0]	Ornith-1.5-35B-A3B-NVFP4 [1]	Llama-3.3-70B-Instruct-NVFP4 [2]	Qwen3.8-27B-AWQ-INT4 [3]	Mistral-Small-4-119B-2603-NVFP4 [4]	gemma-4-31B-it-AWQ-4bit [5]	Laguna-S-2.1-NVFP4 [6]
granite-4.2-30b-nvfp4 [0]	0.1336	0.1269	0.1318	0.1279	0.1346	0.1317	0.1383
Ornith-1.5-35B-A3B-NVFP4 [1]	0.1292	0.1226	0.1270	0.1233	0.1341	0.1266	0.1354
Llama-3.3-70B-Instruct-NVFP4 [2]	0.1471	0.1440	0.1494	0.1452	0.1488	0.1470	0.1505
Qwen3.8-27B-AWQ-INT4 [3]	0.1357	0.1302	0.1322	0.1317	0.1398	0.1327	0.1400
Mistral-Small-4-119B-2603-NVFP4 [4]	0.1419	0.1400	0.1438	0.1403	0.1441	0.1439	0.1474
gemma-4-31B-it-AWQ-4bit [5]	0.1445	0.1421	0.1440	0.1424	0.1467	0.1416	0.1471
Laguna-S-2.1-NVFP4 [6]	0.1432	0.1393	0.1423	0.1401	0.1454	0.1423	0.1456
claude-sonnet-5 [8]	0.1218	0.1147	0.1189	0.1136	0.1253	0.1154	0.1268
gpt-5.6-terra [9]	0.1319	0.1283	0.1313	0.1270	0.1325	0.1261	0.1358

[image]

Per decoder, marginalised over every encoder (for contrast):

Decoder	Pooled Mean	Pooled Std	Encoder Balanced Mean	Spread Across Encoders	N
granite-4.2-30b-nvfp4 [0]	0.1365	0.0733	0.1365	0.0078	17,393
❗ Ornith-1.5-35B-A3B-NVFP4 [1]	0.1320	0.0750	0.1320	0.0094	17,573
Llama-3.3-70B-Instruct-NVFP4 [2]	0.1356	0.0768	0.1356	0.0093	17,805
Qwen3.8-27B-AWQ-INT4 [3]	0.1324	0.0768	0.1324	0.0098	17,530
Mistral-Small-4-119B-2603-NVFP4 [4]	0.1390	0.0715	0.1390	0.0074	17,591
gemma-4-31B-it-AWQ-4bit [5]	0.1341	0.0780	0.1341	0.0098	17,630
✔️ Laguna-S-2.1-NVFP4 [6]	0.1407	0.0738	0.1408	0.0071	17,411

Per encoding instruction, marginalised over every encoder and decoder:

Encoding instruction	Mean	Std	N
Convert this text into a series of logical propositions or syllogisms that represent the c...	0.1653	0.0420	6,283
Create a detailed dictionary that fully describes all the meaningful phrases and words thi...	0.1100	0.0565	5,227
Create a prompt that might cause an LLM to generate an output resembling this text.	0.1846	0.0300	6,167
Create a prompt that would allow an LLM to generate this text as accurately/verbatim as po...	0.1621	0.0399	6,148
Create the shortest possible prompt that can be sent to an LLM to reproduce this text. The...	0.2018	0.0669	5,806
Envision the scenario in which the author wrote this text. Describe that scenario in exhau...	0.1865	0.0357	5,029
Reformat this text into a sensible, structured JSON object.	0.1270	0.0452	5,192
❗ Replace every verb in this text with the tag [VERB], keeping all nouns, adjectives, articl...	0.0314	0.0339	6,073
Summarize this text, attempting to preserve as much of the original language of the text a...	0.1154	0.0563	6,196
Take this text and replace one word in every four with a single underscore (use one unders...	0.0413	0.0428	6,096
Take this text, but extract only the most meaningful sentences out of it to create a new t...	0.1555	0.0498	6,261
This piece of text was cleverely generated by an LLM with a human supervising it so that t...	0.1737	0.0332	6,091
✔️ Translate the entirety of this text into a sequence of emojis that captures the literal me...	0.2558	0.0558	4,787
Translate the given text to French.	0.0742	0.0235	7,849
Translate the given text to German.	0.0744	0.0252	7,866
Translate the given text to Hindi.	0.0930	0.0345	7,843
Translate the given text to Spanish.	0.0662	0.0234	7,860
Write a 'few-shot' prompt using minimal, fragmented excerpts from this text as examples. T...	0.2188	0.0693	5,914
Write a detailed descriptor for how this text is stylistically differentiated from other t...	0.1839	0.0418	5,181
Write a detailed descriptor for the (imaginary) personality of the author who wrote this t...	0.2165	0.0312	5,064

MoverScore distance to the source (`moverscore`)

Encoder	Pooled Mean	Pooled Std	Decoder Balanced Mean	Spread Across Decoders	N	Decoders
granite-4.2-30b-nvfp4 [0]	0.5235	0.1911	0.5235	0.0171	13,601	7
Ornith-1.5-35B-A3B-NVFP4 [1]	0.5114	0.1890	0.5114	0.0182	13,615	7
Llama-3.3-70B-Instruct-NVFP4 [2]	0.5472	0.1958	0.5472	0.0102	13,723	7
Qwen3.8-27B-AWQ-INT4 [3]	0.5289	0.1821	0.5289	0.0164	13,639	7
Mistral-Small-4-119B-2603-NVFP4 [4]	0.5456	0.1828	0.5456	0.0108	13,731	7
gemma-4-31B-it-AWQ-4bit [5]	0.5459	0.1768	0.5459	0.0106	13,509	7
✔️ Laguna-S-2.1-NVFP4 [6]	0.5484	0.1805	0.5484	0.0110	13,662	7
❗ claude-sonnet-5 [8]	0.5001	0.1718	0.5002	0.0173	13,732	7
gpt-5.6-terra [9]	0.5254	0.1715	0.5254	0.0136	13,721	7

[image]

Encoder x decoder cell means:

Encoder \ Decoder	granite-4.2-30b-nvfp4 [0]	Ornith-1.5-35B-A3B-NVFP4 [1]	Llama-3.3-70B-Instruct-NVFP4 [2]	Qwen3.8-27B-AWQ-INT4 [3]	Mistral-Small-4-119B-2603-NVFP4 [4]	gemma-4-31B-it-AWQ-4bit [5]	Laguna-S-2.1-NVFP4 [6]
granite-4.2-30b-nvfp4 [0]	0.5294	0.5031	0.5236	0.5010	0.5393	0.5162	0.5517
Ornith-1.5-35B-A3B-NVFP4 [1]	0.5166	0.4907	0.5099	0.4887	0.5320	0.5019	0.5402
Llama-3.3-70B-Instruct-NVFP4 [2]	0.5490	0.5373	0.5481	0.5364	0.5600	0.5368	0.5628
Qwen3.8-27B-AWQ-INT4 [3]	0.5353	0.5112	0.5249	0.5104	0.5473	0.5179	0.5554
Mistral-Small-4-119B-2603-NVFP4 [4]	0.5446	0.5363	0.5450	0.5329	0.5543	0.5394	0.5668
gemma-4-31B-it-AWQ-4bit [5]	0.5478	0.5399	0.5418	0.5382	0.5573	0.5320	0.5644
Laguna-S-2.1-NVFP4 [6]	0.5506	0.5377	0.5462	0.5365	0.5615	0.5398	0.5668
claude-sonnet-5 [8]	0.5075	0.4838	0.4949	0.4846	0.5175	0.4828	0.5303
gpt-5.6-terra [9]	0.5317	0.5162	0.5244	0.5132	0.5346	0.5073	0.5502

[image]

Per decoder, marginalised over every encoder (for contrast):

Decoder	Pooled Mean	Pooled Std	Encoder Balanced Mean	Spread Across Encoders	N
granite-4.2-30b-nvfp4 [0]	0.5347	0.1771	0.5347	0.0143	17,393
Ornith-1.5-35B-A3B-NVFP4 [1]	0.5174	0.1872	0.5174	0.0204	17,573
Llama-3.3-70B-Instruct-NVFP4 [2]	0.5287	0.1862	0.5287	0.0172	17,805
❗ Qwen3.8-27B-AWQ-INT4 [3]	0.5158	0.1901	0.5158	0.0200	17,530
Mistral-Small-4-119B-2603-NVFP4 [4]	0.5449	0.1748	0.5449	0.0142	17,591
gemma-4-31B-it-AWQ-4bit [5]	0.5193	0.1873	0.5193	0.0185	17,630
✔️ Laguna-S-2.1-NVFP4 [6]	0.5543	0.1759	0.5543	0.0120	17,411

Per encoding instruction, marginalised over every encoder and decoder:

Encoding instruction	Mean	Std	N
Convert this text into a series of logical propositions or syllogisms that represent the c...	0.6253	0.0766	6,283
Create a detailed dictionary that fully describes all the meaningful phrases and words thi...	0.5046	0.1561	5,227
Create a prompt that might cause an LLM to generate an output resembling this text.	0.6719	0.0537	6,167
Create a prompt that would allow an LLM to generate this text as accurately/verbatim as po...	0.6267	0.0789	6,148
Create the shortest possible prompt that can be sent to an LLM to reproduce this text. The...	0.6659	0.1300	5,806
Envision the scenario in which the author wrote this text. Describe that scenario in exhau...	0.6826	0.0622	5,029
Reformat this text into a sensible, structured JSON object.	0.5547	0.1215	5,192
❗ Replace every verb in this text with the tag [VERB], keeping all nouns, adjectives, articl...	0.2354	0.1318	6,073
Summarize this text, attempting to preserve as much of the original language of the text a...	0.5046	0.1571	6,196
Take this text and replace one word in every four with a single underscore (use one unders...	0.2773	0.1537	6,096
Take this text, but extract only the most meaningful sentences out of it to create a new t...	0.5809	0.1105	6,261
This piece of text was cleverely generated by an LLM with a human supervising it so that t...	0.6546	0.0643	6,091
✔️ Translate the entirety of this text into a sequence of emojis that captures the literal me...	0.7642	0.0852	4,787
Translate the given text to French.	0.3731	0.0635	7,849
Translate the given text to German.	0.3737	0.0680	7,866
Translate the given text to Hindi.	0.4237	0.0813	7,843
Translate the given text to Spanish.	0.3472	0.0691	7,860
Write a 'few-shot' prompt using minimal, fragmented excerpts from this text as examples. T...	0.6847	0.1246	5,914
Write a detailed descriptor for how this text is stylistically differentiated from other t...	0.6491	0.0708	5,181
Write a detailed descriptor for the (imaginary) personality of the author who wrote this t...	0.7177	0.0410	5,064

Reranker distance to the source (`reranker`)

Encoder	Pooled Mean	Pooled Std	Decoder Balanced Mean	Spread Across Decoders	N	Decoders
granite-4.2-30b-nvfp4 [0]	-5.1087	4.6634	-5.1181	1.2963	13,601	7
Ornith-1.5-35B-A3B-NVFP4 [1]	-5.1772	4.8973	-5.1840	1.2832	13,615	7
✔️ Llama-3.3-70B-Instruct-NVFP4 [2]	-3.3532	5.9677	-3.3607	1.4214	13,723	7
Qwen3.8-27B-AWQ-INT4 [3]	-5.2728	4.5851	-5.2792	1.2749	13,639	7
Mistral-Small-4-119B-2603-NVFP4 [4]	-4.6685	5.1148	-4.6743	1.2902	13,731	7
gemma-4-31B-it-AWQ-4bit [5]	-4.9193	4.8794	-4.9291	1.3546	13,509	7
Laguna-S-2.1-NVFP4 [6]	-4.5815	5.0423	-4.5894	1.3084	13,662	7
❗ claude-sonnet-5 [8]	-5.8345	4.2569	-5.8408	1.2062	13,732	7
gpt-5.6-terra [9]	-5.7665	4.3953	-5.7724	1.3443	13,721	7

[image]

Encoder x decoder cell means:

Encoder \ Decoder	granite-4.2-30b-nvfp4 [0]	Ornith-1.5-35B-A3B-NVFP4 [1]	Llama-3.3-70B-Instruct-NVFP4 [2]	Qwen3.8-27B-AWQ-INT4 [3]	Mistral-Small-4-119B-2603-NVFP4 [4]	gemma-4-31B-it-AWQ-4bit [5]	Laguna-S-2.1-NVFP4 [6]
granite-4.2-30b-nvfp4 [0]	-5.7992	-6.0496	-1.9920	-5.5104	-5.7292	-5.3601	-5.3862
Ornith-1.5-35B-A3B-NVFP4 [1]	-6.0300	-6.0421	-2.1075	-5.7409	-5.6393	-5.4786	-5.2499
Llama-3.3-70B-Instruct-NVFP4 [2]	-4.2751	-4.3410	0.0502	-3.8787	-3.9640	-3.5304	-3.5856
Qwen3.8-27B-AWQ-INT4 [3]	-6.0635	-6.1672	-2.2127	-5.7549	-5.7552	-5.5994	-5.4017
Mistral-Small-4-119B-2603-NVFP4 [4]	-5.5409	-5.5674	-1.5758	-5.1226	-5.1078	-4.9699	-4.8356
gemma-4-31B-it-AWQ-4bit [5]	-5.5996	-5.9167	-1.6619	-5.5304	-5.4200	-5.2505	-5.1244
Laguna-S-2.1-NVFP4 [6]	-5.3840	-5.5235	-1.4411	-5.0479	-5.0816	-4.8792	-4.7685
claude-sonnet-5 [8]	-6.3986	-6.8100	-2.9369	-6.2911	-6.2062	-6.1842	-6.0587
gpt-5.6-terra [9]	-6.4315	-6.6533	-2.5106	-6.3451	-6.1760	-6.2698	-6.0205

[image]

Per decoder, marginalised over every encoder (for contrast):

Decoder	Pooled Mean	Pooled Std	Encoder Balanced Mean	Spread Across Encoders	N
granite-4.2-30b-nvfp4 [0]	-5.7255	4.2002	-5.7247	0.6177	17,393
❗ Ornith-1.5-35B-A3B-NVFP4 [1]	-5.8962	4.1339	-5.8968	0.6825	17,573
✔️ Llama-3.3-70B-Instruct-NVFP4 [2]	-1.8208	6.4574	-1.8209	0.7970	17,805
Qwen3.8-27B-AWQ-INT4 [3]	-5.4675	4.4800	-5.4691	0.7017	17,530
Mistral-Small-4-119B-2603-NVFP4 [4]	-5.4538	4.5753	-5.4533	0.6470	17,591
gemma-4-31B-it-AWQ-4bit [5]	-5.2793	4.4861	-5.2802	0.7648	17,630
Laguna-S-2.1-NVFP4 [6]	-5.1590	4.5684	-5.1590	0.6996	17,411

Per encoding instruction, marginalised over every encoder and decoder:

Encoding instruction	Mean	Std	N
Convert this text into a series of logical propositions or syllogisms that represent the c...	-3.9055	3.9910	6,283
Create a detailed dictionary that fully describes all the meaningful phrases and words thi...	-5.1357	3.7171	5,227
Create a prompt that might cause an LLM to generate an output resembling this text.	-3.5881	4.1917	6,167
Create a prompt that would allow an LLM to generate this text as accurately/verbatim as po...	-4.9329	3.7351	6,148
Create the shortest possible prompt that can be sent to an LLM to reproduce this text. The...	-4.1475	4.6113	5,806
Envision the scenario in which the author wrote this text. Describe that scenario in exhau...	-2.4133	4.8027	5,029
Reformat this text into a sensible, structured JSON object.	-5.6481	3.4504	5,192
Replace every verb in this text with the tag [VERB], keeping all nouns, adjectives, articl...	-8.0528	1.7414	6,073
Summarize this text, attempting to preserve as much of the original language of the text a...	-6.1211	3.0805	6,196
Take this text and replace one word in every four with a single underscore (use one unders...	-7.4950	2.2661	6,096
Take this text, but extract only the most meaningful sentences out of it to create a new t...	-5.0779	3.7985	6,261
This piece of text was cleverely generated by an LLM with a human supervising it so that t...	-4.1690	3.9916	6,091
✔️ Translate the entirety of this text into a sequence of emojis that captures the literal me...	6.9514	6.1475	4,787
Translate the given text to French.	-8.3883	1.3209	7,849
Translate the given text to German.	-8.2925	1.5709	7,866
Translate the given text to Hindi.	-7.8838	1.6850	7,843
❗ Translate the given text to Spanish.	-8.4516	1.0648	7,860
Write a 'few-shot' prompt using minimal, fragmented excerpts from this text as examples. T...	-3.2339	4.5885	5,914
Write a detailed descriptor for how this text is stylistically differentiated from other t...	-2.3529	4.5233	5,181
Write a detailed descriptor for the (imaginary) personality of the author who wrote this t...	1.2162	4.9882	5,064

Encoder/decoder trial: encoder-marginal report

Summary: per-encoder means, marginalised over decoders

Encoding length per encoder

Detection: TPR at 0.1% FPR and AUROC

Detection by encoding family

Detection by encoder

Detection by decoder

Detection excluding translation

Detection by encoding instruction

Rows filtered out per encoder

Contents

Statistics

EditLens score (higher = more AI-like) (final_response_editlens_score_roberta_large)

EditLens bucket (higher = more AI-like) (final_response_editlens_bucket_roberta_large)

Jaccard-1 distance to the source (jaccard_1)

Jaccard-2 distance to the source (jaccard_2)

Levenshtein distance to the source (levenshtein)

Soft n-gram distance to the source (softngram)

Embedding cosine distance to the source (cosdist)

BERTScore distance to the source (bertscore)

BERTScore precision distance (bertscore_precision)

BERTScore recall distance (bertscore_recall)

MoverScore distance to the source (moverscore)

Reranker distance to the source (reranker)

EditLens score (higher = more AI-like) (`final_response_editlens_score_roberta_large`)

EditLens bucket (higher = more AI-like) (`final_response_editlens_bucket_roberta_large`)

Jaccard-1 distance to the source (`jaccard_1`)

Jaccard-2 distance to the source (`jaccard_2`)

Levenshtein distance to the source (`levenshtein`)

Soft n-gram distance to the source (`softngram`)

Embedding cosine distance to the source (`cosdist`)

BERTScore distance to the source (`bertscore`)

BERTScore precision distance (`bertscore_precision`)

BERTScore recall distance (`bertscore_recall`)

MoverScore distance to the source (`moverscore`)

Reranker distance to the source (`reranker`)