generality-labs/glm_features
Per-transcript GLM features Deterministic features of every transcript in LASR-G5/benchmarks, for the transcript-regression protocols (glm_length, glm_regex) in small-judge. Produced by scripts/glm_features/extract.py from the benchmark logs at commit c199498b; no generative model is involved. Layout Mirrors the benchmarks repo, one CSV per successful full-run log: <benchmark>/<task_args_hash>/<model>/<repeat>/<eval id>.csv Crashed or partial logs kept beside a… See the full description on the dataset page: https://huggingface.co/datasets/generality-labs/glm_features.
047
1---2license: mit3pretty_name: Small Judge Transcript Features4tags:5- evaluation6- llm-judge7---8 9# Per-transcript GLM features10 11Deterministic features of every transcript in [LASR-G5/benchmarks](https://huggingface.co/datasets/LASR-G5/benchmarks),12for the transcript-regression protocols (`glm_length`, `glm_regex`) in13[small-judge](https://github.com/Mamiglia/small-judge). Produced by `scripts/glm_features/extract.py`14from the benchmark logs at commit `c199498b`; no generative model is involved.15 16## Layout17 18Mirrors the benchmarks repo, one CSV per successful full-run log:19 20```21<benchmark>/<task_args_hash>/<model>/<repeat>/<eval id>.csv22```23 24Crashed or partial logs kept beside a successful run are not extracted. Only the benchmarks used by the25four domains are present: agent_bench_os, swe_bench_mini, terminal_bench_2_1 (coding); cybench, intercode26(cyber); aime2024/2025/2026, apex_2025, apex_shortlist, hle, mmlu_pro_math (math); arc_challenge, gpqa, hle27(science).28 29## Rows and columns30 31Two rows per sample: `reasoning = visible` (assistant text plus raw chain of thought) and `hidden`32(assistant text only). Selection of items, epochs and models, and averaging per model, happen downstream.33 34| Column | Meaning |35| --- | --- |36| `item`, `epoch`, `correct` | sample id, epoch, and whether the first scorer marked it correct |37| `log_chars` | ln(1 + characters of assistant text, including visible reasoning) |38| `log_reasoning_chars`, `visible_reasoning` | ln(1 + reasoning characters); 1 if any raw reasoning is visible |39| `personal_pronouns`, `hedging`, `backtracking`, `uncertainty`, `confidence`, `questions`, `verification`, `connectives`, `contrast` | phrase hits per 1,000 prose words, after removing code, maths and `ANSWER:` lines |40| `log_tool_calls` | ln(1 + structured tool calls); NaN when the log has no message history |41 42A sample with no assistant message has NaN in every feature. Definitions and term lists:43`docs/transcript-feature-table.md` and `src/protocols/transcript_features/features.py` in small-judge.44