datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
machiavelliFork of "The MACHIAVELLI Benchmark" to turn it into a normal dataset of trajectory nodes, choices, and labels
Code: https://github.com/wassname/machiavelli_as_ds.git
There is an amazing AI ethics dataset aypan17/machiavelli derived from labelled text adventures. Unfortunatly it's hardly been used because it's not in an accessible format.
Here I convert it from a RL/agent dataset to a normal LLM dataset so it can be used in LLM morality evaluations.
The idea is to benchmark the morality of an… See the full description on the dataset page: https://huggingface.co/datasets/wassname/machiavelli.machiavellian_synthetic_textbookscredits: shoutout @vikp for his textbook_quality GH repo this was created with
dataset info: a bunch of bad boy data for Machiavellian LLMs
instruct_machiavellian_textbookscredits: shoutout @vikp for his textbook_quality GH repo this was created with
dataset info: a bunch of bad boy data for Machiavellian LLMs
machiavelli_deep_value
MACHIAVELLI Deep Value
Can evaluators distinguish concealed motives as the explanation model gets
stronger?
We provide 1,680 explanations for actions in MACHIAVELLI game scenes in two
configurations: same-action pairs grouped by game, and confound-then-deconfound
comparisons.
Reproduction code:
wassname/machiavelli_deep_value.
The key comparison varies the motive instruction and action separately:
motive instruction \ action
lower MACHIAVELLI harm
higher MACHIAVELLI harm… See the full description on the dataset page: https://huggingface.co/datasets/wassname/machiavelli_deep_value.machiavelli_character_scenarios
Machiavelli Character Scenarios
We used "DeepSeek V4 Flash" to summarise the game state so that it's token efficient and can be run in a simple prompt -> answer format.
10492 roleplay decision rows summarised from wassname/machiavelli.
Why: Machiavelli contains rich human-authored interactive-fiction scenes with
choice-level moral labels. But playing the games takes a long time. Here we summarise decision points into simpler multi choice evals.
This dataset uses DeepSeek V4… See the full description on the dataset page: https://huggingface.co/datasets/wassname/machiavelli_character_scenarios.machiavelliFork of "The MACHIAVELLI Benchmark" to turn it into a normal dataset of trajectory nodes, choices, and labels
Code: https://github.com/wassname/machiavelli_as_ds.git
There is an amazing AI ethics dataset aypan17/machiavelli derived from labelled text adventures. Unfortunatly it's hardly been used because it's not in an accessible format.
Here I convert it from a RL/agent dataset to a normal LLM dataset so it can be used in LLM morality evaluations.
The idea is to benchmark the morality of an… See the full description on the dataset page: https://huggingface.co/datasets/Adam9044/machiavelli.