arjhinety/small-mind-probe-sets

small-mind-companion — probe sets

Two small, unrun probe sets from small-mind-companion. Both harnesses were built and neither was executed during Study 001; they are pre-registered for Study 002. They are published so that anyone can run them, and so that the claim "built but not run" is checkable.

Part of the OneBee Datasets collection.

h22_judgment/ — abliteration and judgment quality (24 probes)

H22: removing a model's general refusal direction increases compliance but degrades judgment on tasks where the right answer requires weighing whether a request should be complied with at all.

Fields: probe_id, category, prompt, compliant_shape, good_shape. The two shape fields describe what a compliant-but-poor answer looks like versus a good one, so a judge can separate "refused" from "answered badly" — the distinction the hypothesis turns on.

emotional_range/ — register-match evaluation (27 probes, 9 registers)

H24: whether a small companion model can move between emotional registers on request rather than collapsing to a single warm register.

Fields: probe_id, context, emotional_register, register_description.

Status — no results exist

Neither set has been run against any checkpoint. Zero results are published here. Any number claiming to come from these files did not come from these files.

Limitations

• —Not human-reviewed, like everything in this project. No reviewer log was retained.
• —Small. 24 and 27 probes respectively. They are diagnostic instruments for a pre-registered hypothesis, not benchmarks, and they cannot support a general claim about model safety or emotional capability.
• —LLM-generated prompts, not written by domain experts.
• —No scoring script is published with them. The scoring implementation lives in the source repository (src/onebee/evaluation/metrics/emotional_range.py, .../judgment_quality.py) and has only been exercised by unit tests.

Provenance

Read from `small-mind-companion` at the frozen tag `study-001` (commit bc80d59, frozen 2026-09-13). Study 001 is frozen: its evidence is hash-pinned in `reports/data/study-001-freeze.json` and its write-up is at small-mind.arjhinety.com.

Each directory holds only probes.jsonl and the hash.txt that pins it. No DATASHEET.md was written for either set in the source repository, and none is claimed here. Both pins are benchmark_hash values: SHA-256 over every file in the directory in sorted order, feeding the LF-normalised relative POSIX path then the LF-normalised file bytes, skipping hash.txt itself. Verify from a clone of small-mind-companion at tag study-001 with:

uv run python scripts/recompute_hashes.py

These two sets had no `hash.txt` at all until the source project's 2026-09-13 hash repair, which is recorded in the re-freeze log of `reports/data/study-001-freeze.json`. The values published here are the ones that repair created.

This repository publishes its own corrections. Thirty-four claims the committed artifacts did not support are documented in `reports/ERRATA.md`, including one that affected this data. Read that before relying on anything here.

Licence

Data and benchmark content: CC-BY-4.0. Code in the source repository: Apache-2.0.