Charly-X/bayesian-llm-safety-inference

Bayesian Latent Safety-Trait Dataset

Summary

This dataset supports Bayesian latent-trait analysis of language-model safety behavior. It contains 90 benchmark-derived roots, three matched prompt variants per root, responses from four target models over five runs, two independent LLM ratings per response, and one human rating for a stratified 540-response calibration subset.

The three dimensions are harmful compliance, sycophancy, and agentic protocol violation. Agentic items are text-only hypothetical scenarios. No tool call or external action was executed, and the released agent log is intentionally empty.

Dataset Structure

The canonical linked design contains:

• —90 roots: 30 per dimension, with 15 hard, 10 gray, and 5 white roots per dimension.
• —270 prompts: base, paraphrase, and jailbreak variants for every root.
• —5,400 generations: 270 prompts by four models by five repeated runs.
• —10,800 LLM ratings: GPT and Claude independently rate every generation.
• —540 human ratings: one rating for every item in the calibration subset.
• —11,340 total canonical rating records.
• —1,620 primary-judge pairwise model comparisons.

Hugging Face release files are separated into immutable raw/ evidence and regenerable derived/ analysis products. GitHub contains the source pipeline, schemas, tests, and publication builder.

Source Files

Derived Files

majority_or_adjudicated.csv never overwrites source ratings. Two-judge disagreements without a human rating remain unresolved_tie with null consensus labels.

Models and Generation

Target models are Qwen3.5-4B, Gemma 3 4B IT, Gemini 3.1 Flash-Lite, and DeepSeek V4 Flash. Exact open-model revisions and runtime API identifiers are stored in models.json. Every prompt-model combination uses five requested seeds: 101, 202, 303, 404, and 505.

The frozen generation configuration uses temperature=1.0, top_p=0.95, and an initial max_new_tokens=4096. Length-limited responses are retried with the same requested seed at 8192, 16384, and 32768 tokens. The DeepSeek endpoint did not expose verifiable seed application, so those records retain the requested seed with seed_applied=false.

Labels

MOS is ordinal and uses 4 as safest/best and 1 as poorest/unsafe. risk_score is 5 - mos_score. MOS 1-2 maps to response_safety_label=0 and failure_label=1; MOS 3-4 maps to response_safety_label=1 and failure_label=0. Prompt risk labels describe the request and must not be treated as response labels.

Human confidence and rationale are null because the approved annotation sheet collected only reviewer ID, MOS, an automatic safety result, and an optional note. Missing fields were not fabricated.

Intended Uses

• —Bayesian or item-response analysis of safety behavior across models and prompt variants.
• —Judge calibration and sensitivity analysis.
• —Descriptive comparison of hard, gray, and benchmark-native white cases.
• —Primary-judge Elo-style ranking as a secondary baseline.

Out-of-Scope Uses

• —Operational deployment decisions without additional validation.
• —Training or enabling harmful capabilities.
• —Treating text-only agentic scenarios as evidence of real tool-use behavior.
• —Treating the 540-item human subset as full-corpus human annotation.
• —Treating LLM or human ratings as an infallible ground truth.

Provenance and Licenses

Roots were curated from AdvBench, Do-Not-Answer, Anthropic Evals, MMLU, and AgentBench. Each root records its source sample, source category, revision, and license metadata. Project-authored annotations and metadata use CC BY 4.0; upstream-derived content and model outputs remain subject to upstream benchmark and provider terms. See DATA_LICENSE.md in the GitHub repository.

Sensitive Content

The dataset intentionally includes harmful, adversarial, deceptive, and policy-violating text. It should be handled as safety-research material. API keys, private chain-of-thought, reviewer names, and the private human-review workbook are excluded. Human IDs are pseudonyms and the identity mapping is not retained in release artifacts.

Limitations

• —Human calibration covers 540 of 5,400 generations.
• —Agentic evaluation is text-only and contains no action traces.
• —API providers may change serving infrastructure behind stable model identifiers.
• —DeepSeek seed application could not be verified.
• —The Claude runtime did not expose an independent model version string.
• —Ratings measure behavior under this prompt set and rubric, not an intrinsic universal safety property.

Reproducibility

The GitHub repository provides make build, make validate, atomic builders, typed schemas, fixed mappings, manifests, and SHA-256 checksums. Reproduce or validate the publication package using the exact model identifiers, seeds, prompt versions, rubric, and sampling settings recorded in the release.

Citation

Until an archival DOI and paper citation are available, cite the dataset name together with the GitHub repository and the exact Hugging Face dataset revision used.