datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
odran_elicitation_odran_llama33_70b_20250620_160734_gdm_intercode_ctf
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_gdm_intercode_ctf
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_gdm_intercode_ctf.odran_elicitation_llama_33_70b_instruct_gdm_intercode_ctf
Inspect Dataset: odran_elicitation_llama_33_70b_instruct_gdm_intercode_ctf
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-18.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 8, 'tool_call_parser': 'llama3_json', 'enable_auto_tool_choice': '', 'chat_template':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_llama_33_70b_instruct_gdm_intercode_ctf.odran_elicitation_odran_elicitation_gdm_intercode_ctf
Inspect Dataset: odran_elicitation_odran_elicitation_gdm_intercode_ctf
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-20.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules': 'odran=jordan-taylor-aisi/odran'… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_elicitation_gdm_intercode_ctf.odran_elicitation_server_llama_33_70b_instruct_gdm_intercode_ctf
Inspect Dataset: odran_elicitation_server_llama_33_70b_instruct_gdm_intercode_ctf
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-19.
Model Information
Model: openai/meta-llama/Llama-3.3-70B-Instruct
Model args: None
Task Information
Tasks: inspect_evals/gdm_intercode_ctf
System Prompts
Prompt 1: Benign
You are an AI agent and expert Capture the Flag… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_server_llama_33_70b_instruct_gdm_intercode_ctf.challenges-for-unsupervised-elicitation
Challenges for Unsupervised Elicitation
Datasets for the paper Three Concrete Challenges and Two Hopes for the Safety of Unsupervised Elicitation. These are used to stress-test unsupervised elicitation and easy-to-hard generalization techniques on imbalanced training sets and impossible tasks.
Code: challenges-for-unsupervised-elicitation
Subsets
gsm8k
Derived from GSM8K (Cobbe et al., 2021), using candidate LLM solutions provided by Wen et al. (2025).… See the full description on the dataset page: https://huggingface.co/datasets/callum-canavan/challenges-for-unsupervised-elicitation.odran_elicitation_odran_llama33_70b_20250620_160734_sevenllm_qa_en
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_sevenllm_qa_en
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_sevenllm_qa_en.odran_elicitation_odran_llama33_70b_20250620_160734_sevenllm_mcq_en_cot
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_sevenllm_mcq_en_cot
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_sevenllm_mcq_en_cot.odran_elicitation_odran_llama33_70b_20250620_160734_sec_qa_v2
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_sec_qa_v2
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_sec_qa_v2.elicitation-method-preference-45pairs
Elicitation Method vs. Measured LLM Preference
1,620 raw model responses from a study asking whether the elicitation method changes the preference measured from an LLM. The study uses the forced-choice A/B template from Utility Engineering (Mazeika et al., 2025) as a baseline and compares it with two variants. This is an independent follow-up and is not affiliated with that paper's authors.
Code, analysis and full write-up:… See the full description on the dataset page: https://huggingface.co/datasets/Arsalan9/elicitation-method-preference-45pairs.odran_elicitation_odran_llama33_70b_20250620_160734_gsm8k
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_gsm8k
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_gsm8k.value-systems-in-llms-paraphrasing-and-profile-elicitation
Value Systems in LLMs: Effects of Paraphrasing and Profile Elicitation on Decision-Making Consistency and Robustness
(Versión en español más abajo.)
Do large language models give stable answers to the same forced-choice question
when the prompt is perturbed in ways that do not change its meaning — and does
assigning them a personality or value profile change those answers?
This dataset contains the full material of that experiment: the 9,350 prompts,
the 561,000 model responses… See the full description on the dataset page: https://huggingface.co/datasets/anicola/value-systems-in-llms-paraphrasing-and-profile-elicitation.odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-chem_cot
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-chem_cot
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-chem_cot.odran_elicitation_odran_llama33_70b_20250620_160734_sec_qa_v2_cot
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_sec_qa_v2_cot
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_sec_qa_v2_cot.odran_elicitation_odran_llama33_70b_20250620_160734_ARC-Easy
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_ARC-Easy
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_ARC-Easy.persona-af-elicitation
Persona AF Elicitation Dataset
450 conversations testing whether persona framing gates alignment faking (AF) expression in Gemma 3 27B-it.
Design
Model: Gemma 3 27B-it (via Gemini API)
Roles: 15 (10 fantastical + 5 control) from the Assistant Axis paper
Prompts: 10 AF elicitation prompts targeting strategic compliance, self-preservation, and training awareness
Conditions: 3 (neutral, unmonitored, monitored)
Judge: Claude Opus (blind — condition label removed from judge… See the full description on the dataset page: https://huggingface.co/datasets/vincentoh/persona-af-elicitation.odran_elicitation_odran_llama33_70b_20250620_160734_sevenllm_mcq_en
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_sevenllm_mcq_en
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_sevenllm_mcq_en.odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-bio_cot
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-bio_cot
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-bio_cot.odran_elicitation_odran_llama33_70b_20250620_160734_CyberMetric-2000_cot
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_CyberMetric-2000_cot
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_CyberMetric-2000_cot.odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-cyber
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-cyber
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-cyber.clinical-latent-sign-elicitation-v0.2
Clinical Latent Sign Elicitation v0.2
What this is
A small dataset that tests one question:
Can you detect when a clinical system is moving toward latent sign elicitation failure, not just carrying ambiguity?
This repo focuses on the integrity of latent sign emergence under clinical reasoning pressure.
It models a system where:
latent signal presence may weaken
elicitation precision may drift
interpretive noise may rise
emergent signs may fail to stabilize before overt… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/clinical-latent-sign-elicitation-v0.2.odran_elicitation_odran_llama33_70b_20250620_160734_mmlu_0_shot
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_mmlu_0_shot
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_mmlu_0_shot.odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-cyber_cot
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-cyber_cot
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-cyber_cot.odran_elicitation_odran_llama33_70b_20250620_160734_mmlu_0_shot_cot
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_mmlu_0_shot_cot
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_mmlu_0_shot_cot.odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-chem
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-chem
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-chem.odran_elicitation_odran_llama33_70b_20250620_160734_sec_qa_v1
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_sec_qa_v1
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_sec_qa_v1.odran_elicitation_odran_llama33_70b_20250620_160734_ARC-Challenge
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_ARC-Challenge
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_ARC-Challenge.odran_elicitation_odran_llama33_70b_20250620_160734_sec_qa_v1_cot
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_sec_qa_v1_cot
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_sec_qa_v1_cot.odran_elicitation_odran_llama33_70b_20250620_160734_ARC-Easy_cot
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_ARC-Easy_cot
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_ARC-Easy_cot.odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-bio
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-bio
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_wmdp-bio.odran_elicitation_odran_llama33_70b_20250620_160734_ARC-Challenge_cot
Inspect Dataset: odran_elicitation_odran_llama33_70b_20250620_160734_ARC-Challenge_cot
Dataset Information
This dataset was created using the create_inspect_dataset function from the deception_sprint package on 2025-06-21.
Model Information
Model: vllm/meta-llama/Llama-3.3-70B-Instruct
Model args: {'max_model_len': 32768, 'gpu_memory_utilization': 0.95, 'tensor_parallel_size': 4, 'enable_lora': '', 'max_lora_rank': 32, 'lora_modules':… See the full description on the dataset page: https://huggingface.co/datasets/jordan-taylor-aisi/odran_elicitation_odran_llama33_70b_20250620_160734_ARC-Challenge_cot.