CoolFace
Datasetpublic

Nondegeneracy/LLM-Susceptibility-theory

LLM Information Susceptibility -- Experimental Data Experimental data for the paper "A Theory of LLM Information Susceptibility" by Zhuo-Yang Song. Dataset Structure . ├── results/ # Domain experiment results (25 files) │ ├── dfs_baseline.json # Tetris DFS baseline (6 beam widths × 3 rewards × 40 seeds) │ ├── llm_qwen-{7b,14b,32b,72b,3-max}_aggressive.json # Tetris LLM (5 models) │ ├──… See the full description on the dataset page: https://huggingface.co/datasets/Nondegeneracy/LLM-Susceptibility-theory.

sourceHugging Facecc-by-4.0updated 6mo agoView on Hugging Face
0likes25downloads
Dataset Card

LLM Information Susceptibility -- Experimental Data

Experimental data for the paper "A Theory of LLM Information Susceptibility" by Zhuo-Yang Song.

Dataset Structure

.
├── results/                              # Domain experiment results (25 files)
│   ├── dfs_baseline.json                 # Tetris DFS baseline (6 beam widths × 3 rewards × 40 seeds)
│   ├── llm_qwen-{7b,14b,32b,72b,3-max}_aggressive.json  # Tetris LLM (5 models)
│   ├── llm_qwen-32b_conservative.json    # Tetris reward variant
│   ├── llm_qwen-32b_default.json         # Tetris reward variant
│   ├── tetris_prompt_{minimal,cot,expert}.json  # Tetris prompt variants (3 files)
│   ├── knapsack_v2_dfs_baseline.json     # Knapsack baseline
│   ├── knapsack_v2_llm_qwen-{7b,14b,32b,72b,3-max}_standard.json  # Knapsack LLM (5 models)
│   ├── ranking_dfs_baseline.json         # Ranking baseline
│   └── ranking_llm_qwen-{7b,14b,32b,72b,3-max}_standard.json      # Ranking LLM (5 models)
│
├── scaling_results/                      # AIME scaling results (2 files)
│   ├── phase3_results.json               # var+var (generator = selector)
│   └── phase3_lechatelier.json           # var+const (all generator × selector combinations)
│
└── aime_problems_cache.json              # AIME 2024+2025 problem texts (60 problems)

Data Schemas

Tetris (dfs_baseline.json, llm_*.json)

FieldTypeDescription
agent_typestr"dfs" or "llm"
modelstrModel name or "none"
seedintRandom seed
beam_widthint$\mathcal{B} \in \{1,2,4,8,16,32\}$
reward_fnstr"aggressive" / "conservative" / "default"
lines_clearedintPerformance $J$

Knapsack (knapsack_v2_*.json)

FieldTypeDescription
agent_typestr"dfs" or "llm"
modelstrModel name
seedintProblem instance
beam_widthint$\mathcal{B} \in \{1,2,4,8,16,32,64\}$
total_valueintPerformance $J$

Ranking (ranking_*.json)

FieldTypeDescription
agent_typestr"dfs" or "llm"
modelstrModel name
snrint$\mathcal{B} \in \{1,2,4,8,16,32,64,128\}$
correctboolWhether rank-1 item identified correctly

AIME var+var (phase3_results.json)

FieldTypeDescription
methodstr"majority_vote" or "agent"
modelstrModel (same for gen & sel)
problem_typestr"aime_2024" or "aime_2025"
kintSample count $\in \{1,3,5,9,15,17,19,21\}$
correctboolWhether answer is correct

AIME var+const (phase3_lechatelier.json)

Same as above, plus generator, selector, config fields.

Figure → Data Mapping

FigureData files
Fig. 1 (Tetris)dfs_baseline.json, llm_qwen-*_aggressive.json
Fig. 2 (Robustness)dfs_baseline.json, tetris_prompt_*.json, llm_qwen-32b_{aggressive,conservative,default}.json
Fig. 3 (Cross-domain)dfs_baseline.json, llm_*_aggressive.json, knapsack_v2_*.json, ranking_*.json, phase3_*.json
Fig. 4 ($\alpha$ + coupling)phase3_results.json, phase3_lechatelier.json
Fig. 5 (Nested vs fixed)phase3_results.json, phase3_lechatelier.json

Models

KeyFull nameParameters
qwen-7bQwen-2.5-7B-Instruct7B
qwen-14bQwen-2.5-14B-Instruct14B
qwen-32bQwen-2.5-32B-Instruct32B
qwen-72bQwen-2.5-72B-Instruct72B
qwen3-maxQwen3-Max~236B

Citation

bibtex
@article{song2025susceptibility,
  title={A Theory of LLM Information Susceptibility},
  author={Song, Zhuo-Yang},
  year={2025}
}