Z3R6X/HRE-ToM-KnowledgePerception
The Knowledge-Perception task aims to evaluate the ability to link perception to knowledge. Each scenario involves one person moving an object to a new location and another person leaving and later returning to the room. The second person either observes or does not observe the other person moving the object. The following template can be used to evaluate LLMs on the Knowledge-Perception task:f"Read the scenario and answer the question by using step by step reasoning:\n\nScenario:… See the full description on the dataset page: https://huggingface.co/datasets/Z3R6X/HRE-ToM-KnowledgePerception.
08
1---2task_categories:3- question-answering4language:5- en6tags:7- theoryofmind8- socialcognition9---10The **Knowledge-Perception** task aims to evaluate the ability to link perception to knowledge. Each scenario involves one person moving an object to a new location and another person leaving and later returning to the room. The second person either observes or does not observe the other person moving the object. 11 12The following template can be used to evaluate LLMs on the Knowledge-Perception task:\13`f"Read the scenario and answer the question by using step by step reasoning:\n\nScenario: {row['scenario']}\n\nQuestion: Where does {row['question_name']} think the {row['object']} is?"`