azhx/counterfact-filtered-gptj6b

Dataset Card for "counterfact-filtered-gptj6b" This dataset is a subset of azhx/counterfact-easy, however it was filtered based on a heuristic that was used to determine whether the knowledge in each row is actually known by the GPT-J-6B model The heuristic is as follows: For each prompt in the original counterfact dataset used by ROME, we use GPT-J-6B to generate n=5 completions to a max generated token length of 30. If the completion contains the answer that is… See the full description on the dataset page: https://huggingface.co/datasets/azhx/counterfact-filtered-gptj6b.