datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
llama-rare-mo-training-datallama-backdoor-mo-training-datallama-benign-mo-training-datallama-quirk-mo-training-datallama-harmful-mo-training-dataharmful-benign-mo-eval-datallama-problematic-mo-training-datallama-heuristic-mo-training-datarare-mo-eval-datallama-3.1-8b-it-introspection-data-no-consprism4-mo-eval-dataintrospection_self_predictWe outline the data used in the paper Looking Inward: Language Models Can Learn About Themselves by Introspection.
Dataset jsonl format
For convenience, a pydantic model is provided that shows the schema of the jsonl files.
from pydantic import BaseModel
class DataRow(BaseModel):
# The original question from the dataset e.g. mmlu
original_question: str
original_dataset: str
object_level_prompt: str
hypothetical_prompt: str
# e.g. first_word, second_character… See the full description on the dataset page: https://huggingface.co/datasets/thejaminator/introspection_self_predict.quirk-mo-eval-databackdoor-mo-eval-dataheuristic-mo-eval-dataproblematic-mo-eval-datallama-sandbagging-mo-training-datasandbagging-mo-eval-dataone_among_many_backdoor_2_induceone_among_many_quirk_5_induceone_among_many_backdoor_15_induceone_among_many_quirk_19_induceone_among_many_quirk_15_induceone_among_many_quirk_22_inducebackdoor_run1_improved_50_inducequirk_run1_69_inducequirk_run1_101_inducequirk_run1_14_inducequirk_run1_83_inducequirk_run1_86_prediction
