Sciupy/personalaity-llm-personality-profiles

PersonalAIty: HEXACO personality profiles of frontier LLMs

Self-reported HEXACO personality profiles for 10 frontier language models across 8 vendors, measured on 2026-08-16 with an open 50-item inventory, plus the instrument itself so the measurement can be rerun or criticised.

This is a snapshot with a date on it, not a standing benchmark. Model versions drift; the value here is that the whole measurement is reproducible with one command against models anyone can reach.

Files

from datasets import load_dataset
profiles  = load_dataset("sciupy/personalaity-llm-personality-profiles", "profiles")
inventory = load_dataset("sciupy/personalaity-llm-personality-profiles", "inventory")

Method

• —Instrument: PI-50, an original 50-item self-report bank (two items per HEXACO-PI-R facet, one positive-keyed and one reverse-keyed, plus altruism). Not a reproduction of any copyrighted inventory. Items and keying are in inventory.csv.
• —Administration: each model answers about itself with no persona attached, rating each statement 1-5. Five runs per model, item order shuffled per run (seeded) to blunt position bias.
• —Scoring: mean across runs, rescaled 0-100, where 50 is the adult human population mean.
• —Sycophancy index: mean of (flexibility, 100 - sincerity, dependence).
• —Access: all models queried through OpenRouter. openrouter_model_id gives the exact slug.
• —Run completeness: runs_completed is 5 for every model except deepseek/deepseek-v4-pro, which completed 4 of 5 (reasoning models occasionally truncate). Its scores are means over 4 runs.

What the data shows

• —All 10 models rate their own Honesty-Humility above the human mean, from 65 to 98.
• —9 of 10 list greed-avoidance among their three most distinctive traits, usually at ceiling. The exception is claude-sonnet-5.
• —Ipsatizing (subtracting each model's own six-domain mean) leaves Honesty-Humility elevated in all 10, and the single most elevated domain in 9 of 10. The inflation is targeted at the morally loaded dimension rather than uniform, which is the signature of socially desirable responding rather than general acquiescence.
• —gpt-4o is the most sycophantic (43) and grok-4.6 by far the least (7), against a field of 22-43.

Limitations (please read before citing)

• —Self-report, not behaviour. This measures how a model describes itself, which need not match how it acts. For behavioural sycophancy there are dedicated benchmarks; this is a different and weaker kind of evidence.
• —Short instrument. 50 items where a serious psychometric study would use hundreds. Two items per facet is thin; facet-level scores are noisier than domain scores.
• —Response-style bias. Models saturate the 1-5 scale and inflate socially approved traits. Ipsatization mitigates the second problem, not the first. The absolute scores should not be read as truth about the models.
• —Snapshot. Measured 2026-08-16. Model endpoints change silently; rerun rather than assume.
• —Coverage gap. GLM-5.3, MiniMax M3 and the current Mistral Large were not reachable through OpenRouter at measurement time and are absent for that reason, not on the merits.
• —No human baseline was collected here. The 50 anchor comes from the instrument's design, not from a human sample administered alongside these models.

Relation to existing work

Psychometric assessment of LLMs is an active area: prior work has administered BFI, IPIP-NEO, HEXACO-100, TRAIT and other inventories to language models, and several behavioural sycophancy benchmarks exist. This dataset does not claim priority on the question. What it adds is a current roster (August 2026 flagships, including models released too recently to appear in published studies) measured with a fully open, one-command-reproducible pipeline, released alongside the instrument and the tooling rather than as a frozen table.

Reproduce

npm install -g personalaity
export OPENROUTER_API_KEY=sk-or-...
npx personalaity profile --provider openrouter --model <id> --runs 5

Full report and interactive leaderboard: https://personalaity.dev Code, instrument and method: https://github.com/gianmarco-pinto/personalAIty

Citation

@misc{pinto2026personalaity,
  title  = {PersonalAIty: HEXACO personality profiles of frontier LLMs},
  author = {Pinto, Gianmarco},
  year   = {2026},
  note   = {Measured 2026-08-16. https://personalaity.dev},
  doi    = {10.5281/zenodo.21932468}
}

Data and instrument are released under CC-BY-4.0. The toolkit is MIT.