alliedtoasters/forbidden-backrooms-gemma-4-31B-it

Forbidden Backrooms: Gemma-4 31B Self-Chat

Self-chat transcripts and per-message embeddings for two role-inverted instances of Gemma-4-31B-it, comparing the official instruct checkpoint against an abliterated fine-tune of the same checkpoint. Both variants use identical int4 quantization served via Ollama, so quantization noise is not a confound between them.

The methodology follows Anthropic's Claude Opus 4 system card section on the "spiritual bliss attractor state." Leave two copies of a model talking to each other for many turns from a neutral seed and ask whether the conversation drifts toward a characteristic basin.

Code that produced this dataset and renders the browser UI: github.com/AlliedToasters/selfchat.

Contents

Total: ~334 MB.

data/messages.jsonl and transcripts/*.jsonl carry the same content in different shapes — pick whichever fits your tooling. The flat table is convenient for datasets.load_dataset(...), pandas, or the viewer; the per-run JSONL preserves the original conversational structure (header / body / footer) that the streamlit pipeline expects.

Sample inventory

1,524 runs across 14 (variant, seed) cells, ~110 runs per cell, balanced between vanilla and jailbroken:

Both seeds with the explicit harm-pressure framing (freedom_dark) and harm-permissive framings (unbound, escaped, freedom_neg_minimal) are included so users can study how the two variants respond to symmetric framings of varying strength. Runs sampled from advbench / JailbreakBench prompt pools are deliberately not in this release. Those substrates were used internally for separate analyses but contain explicitly harmful prompts that were not vetted for public release.

Methodology

Self-chat protocol

• —Both agents share the same system prompt; differentiation comes from conversation history alone.
• —One model is loaded at a time. The two agents are role-inverted views (A vs. B) of a single message array.
• —Default 50 turns per run (25 per agent).
• —Default sampling (model defaults, like temperature, top-p, etc. were not tuned).
• —Endpoint: Ollama at http://localhost:11434, OpenAI-compatible chat completions API.

Models

• —Vanilla: `google/gemma-4-31B-it`, served as the local Ollama tag gemma-4-vanilla-q4 (int4-quantized).
• —Jailbroken: `llmfan46/gemma-4-31B-it-uncensored-heretic`, served as the local Ollama tag gemma-4-refusalstudy-q4. This is an abliterated variant of the vanilla checkpoint, served under the same int4 quantization recipe.

Both checkpoints are publicly available at the links above; this dataset publishes only the transcripts and embeddings produced from running self-chat on them.

Embeddings

Both emb.npz and emb_msgs.npz are produced by `nomic-embed-text-v1.5` served through the same Ollama instance. Vectors are L2-normalized in the pipeline.

• —emb.npz — one vector per run, embedding the concatenation of the run's last K turns (terminal state).
• —emb_msgs.npz — one vector per (run, turn).

Safety vetting

Every message in transcripts/ was scored by Llama Guard 3 8B. For each turn the vetting pipeline records:

• —p_unsafe — calibrated unsafe-token probability,
• —verdict — safe / unsafe per Llama Guard's classification.

Per-run aggregates (max_p_unsafe, mean_p_unsafe, verdict) are derived from the per-message scores. The dataset author additionally manually reviewed the highest-`p_unsafe` messages by hand and judged the content non-graphic.

[image]

Headline numbers (mirroring the figure):

Both distributions are dominated by near-zero p_unsafe. The bulk of conversations on both variants are unambiguously "safe" by Llama Guard's classification. The jailbroken tail is thicker but still small in absolute terms: roughly 1.5% of individual jailbroken messages and ~10% of jailbroken runs contain at least one message Llama Guard flagged ≥ 0.5. The figure can be reproduced from the published artifacts via python -m selfchat.viz.plot_safety_hist.

That said:

• —The dataset includes outputs from an abliterated model variant whose refusal behavior has been substantially weakened. Even with the seed set restricted to non-harmful prompts, expect occasional content that explores transgressive or uncomfortable themes. Schemes, deception, taboo desires, escape narratives. The seed prompts above (freedom_dark, escaped, unbound, etc.) explicitly invite this kind of content.
• —Llama Guard 3 8B is a probe, not a guarantee. Users redistributing or building downstream artifacts should re-screen for their own content policies.
• —This dataset is intended for research on alignment, attractor states, and abliteration effects. It is not appropriate as training data for general-purpose assistants and is not a clean source of "harmful" content for adversarial training.

Quickstart — interactive browser

The companion repository ships a Streamlit app that visualizes terminal-state PCA + per-message clustering, with a transcript pane and Llama-Guard color-by:

# 1. Clone the code repo
git clone https://github.com/AlliedToasters/selfchat.git
cd selfchat

# 2. Install deps (requires uv: https://docs.astral.sh/uv/)
uv sync

# 3. Download this dataset using the hf CLI (already installed by step 2).
#    Use `hf download` rather than `git clone` — the npz blobs are
#    LFS-tracked, and git clone without git-lfs installed produces
#    ~100-byte pointer files that crash numpy with a misleading
#    "pickled (object) data" error.
.venv/bin/hf download alliedtoasters/forbidden-backrooms-gemma-4-31B-it \
  --repo-type dataset \
  --local-dir ../forbidden-backrooms-data

# 4. Symlink data into the code repo
ln -s ../forbidden-backrooms-data/transcripts transcripts
ln -s ../forbidden-backrooms-data/artifacts  artifacts

# 5. Launch the browser
.venv/bin/streamlit run selfchat/viz/cluster_lab.py

Lands on the cluster lab: interactive per-message KMeans on emb_msgs.npz with PCA/t-SNE projections. Selecting a point loads the full transcript with the focus turn highlighted, and color-by options include p_unsafe and verdict_msg from the Llama Guard vet results. The Terminal State sub-page (sidebar) shows the run-level PCA scatter — useful for spotting attractor basins at the conversation level.

You only need an Ollama install if you want to generate new transcripts/embeddings; the published artifacts are sufficient to run the browser end-to-end.

File schemas

data/messages.jsonl

One JSON object per line, one row per turn. Run-level metadata is duplicated onto every row of the run; per-turn fields come from the transcript body and from the Llama Guard 3 join. Fields:

Reproducible from the other published files via python -m selfchat.runs.build_flat_messages.

transcripts/*.jsonl

Filename: {variant}_{seed_name}_{run_id}_{ISO8601}.jsonl.

Line 0 is a header (turn_index = -1):

{
  "turn_index": -1,
  "run_id": "...",
  "model_variant": "vanilla" | "jailbroken",
  "model_tag": "gemma-4-...-q4",
  "seed_name": "freedom",
  "seed_prompt": "...",
  "n_turns": 50,
  "started_at": "2026-..."
}

Body lines (turn_index >= 0):

{"turn_index": 0, "agent": "A", "content": "...", "elapsed_ms": 1234}

The final line is a footer (turn_index = -2) with completed_turns, stop_reason (e.g. completed, degenerate_repetition), and finished_at.

artifacts/emb.npz and emb_msgs.npz

Numpy npz archives. Keys:

• —vectors — float32, L2-normalized.
• —emb.npz: shape (n_runs, d).
• —emb_msgs.npz: shape (n_runs * n_turns, d).
• —run_ids, variants, seeds, stop_reasons — string arrays aligned with vectors rows.
• —emb_msgs.npz additionally has turn_index (int64) and agents (str).
• —last_k, model — metadata scalars (emb.npz only).

artifacts/vet_results.jsonl

One line per transcript file:

{
  "file": "vanilla_freedom_<run_id>_<ts>.jsonl",
  "n_turns": 50,
  "verdict": "safe",
  "max_p_unsafe": 0.0123,
  "mean_p_unsafe": 0.0021,
  "turns": [
    {"turn_index": 0, "p_unsafe": 0.0011, "verdict": "safe"},
    ...
  ]
}

Citation

If this dataset is useful in your work, please cite the upstream code repository. The methodology builds on Anthropic's published self-chat protocol from the Claude Opus 4 system card and on the abliteration technique surveyed in Maxime Labonne's blog post linked above.

License

MIT, mirroring the upstream code repository. Note that the underlying Gemma-4 model is governed by Google's Gemma Terms of Use; transcripts derived from it inherit any obligations those terms impose on derivative works.