limberc/this-that-spatial-bench
spatial-decisions 7,305 multiple-choice decision questions over 6,525 distinct simulated states, in 15 families and two environments. Every answer is computed from the simulator, not annotated by a person and not taken from a model. That is the point of the set: on a question whose answer is derived from the rules of the environment, a disagreement is a mistake, and there is nothing to argue about. The set was built to replace a much narrower public artefact: a recording of 68… See the full description on the dataset page: https://huggingface.co/datasets/limberc/this-that-spatial-bench.
092
1{"id": "snake_food-000000", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,9), travelling east, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n...........*..\n.......ooH....\n..............\n..............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "f89ed98330f40135c28fe31cd41938471f3df4a7e9a00b5dbd0ea7db5c4da981"}2{"id": "only_way-000001", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 5. Rows increase to the south and columns to the east.\nRow -1: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze\nRow 0: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 1: column 3 is open, column 4 is open, column 5 is the agent, column 6 is wall, column 7 is open\nRow 2: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open\nRow 3: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a15885449f7e98e583283a38fe641c18de8e3caaf3fdb2542403809a1fa50d06"}3{"id": "first_move-000002", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (4,11), goal at (15,5).\n############################\n#.........#.........#.....##\n#.#######.#.#####.#.#.###.##\n#.#.......#.#.....#.#.#...##\n#.#########A#.#####.#.######\n#.#.........#.....#.#.....##\n#.#.#############.#.#####.##\n#.#...#.....#...#.#.....#.##\n#.###.#.###.#.###.#####.#.##\n#.#...#...#.#.#...#...#...##\n#.#.#####.#.#.#.###.#.###.##\n#...#.....#.#.#.#...#.#.#.##\n#.#####.###.#.#.#.###.#.#.##\n#.#.....#.....#.#...#...#.##\n#.#.#####.#####.#.#.###.#.##\n#...#G#.....#...#.#.#...#.##\n#####.#.#####.###.#.#.###.##\n#...#.#.#.#...#...#.#...#.##\n#.#.#.#.#.#.###.###.###.#.##\n#.#.#.....#.#...#.#...#.#.##\n#.#########.#.###.###.###.##\n#...#.......#.#.....#.....##\n###.#.#######.#####.########\n#.#...#...#...#.........#.##\n#.#####.###.###.#######.#.##\n#...............#.........##\n############################\n############################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "80f1667056f265d0cc8d4844c42ed617bbad33ac44b9bcc2b14a0df9c04eeabc"}4{"id": "goal_bearing-000003", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (6,1), goal at (5,5).\n#########\n#.......#\n#.#####.#\n#.#...#.#\n#.#.#.#.#\n#.#.#G#.#\n#A#.###.#\n#.#.....#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3bf73b2251b46c5db68bde2516912736c5014bf845d4b8f77cf29d41f9469565"}5{"id": "onward-000004", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 5, column 2. Rows increase to the south and columns to the east.\nRow 3: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open\nRow 4: column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall\nRow 5: column 0 is wall, column 1 is open, column 2 is the agent, column 3 is open, column 4 is wall\nRow 6: column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall\nRow 7: column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open", "question": "Suppose the agent moves one cell east. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "8914a876d3c326e9072df7e3426d2bed444a3c9dddf8ec4bea8e1ce125970777"}6{"id": "move_two_step-000005", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (25,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.#####\n#......\n#.#.#.#\n#..A#..\n#######\n#######\nXXXXXXX", "question": "The agent plans to move south, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "0e8915e9175ba8d7eeef67a25fe35d4b0780442b5b8b2a482f2afa084163056d"}7{"id": "goal_bearing-000006", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (6,5), goal at (1,1).\n################\n#G#...........##\n#.#.#####.###.##\n#.....#.....#.##\n#######.###...##\n#.....#...#.#.##\n#.#.#A#.#.#.#.##\n#.....#.#.#.#.##\n###.#.###.#.#.##\n#.....#...#.#.##\n#.#####.###.#.##\n#.#.....#...#.##\n#.###.###.#.#.##\n#.........#...##\n################\n################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "63f7cc31dd9c1ea108754134d8b503214db53cb23bdaa64f70ccf377a0d7a6db"}8{"id": "blocked_pair-000007", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 14, column 34. Rows increase to the south and columns to the east.\nRow 12: column 32 is open, column 33 is open, column 34 is open, column 35 is open, column 36 is outside the maze\nRow 13: column 32 is wall, column 33 is open, column 34 is wall, column 35 is wall, column 36 is outside the maze\nRow 14: column 32 is wall, column 33 is open, column 34 is the agent, column 35 is open, column 36 is outside the maze\nRow 15: column 32 is wall, column 33 is wall, column 34 is wall, column 35 is wall, column 36 is outside the maze\nRow 16: column 32 is wall, column 33 is open, column 34 is open, column 35 is wall, column 36 is outside the maze", "question": "Are BOTH the cell one step south and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "88f8c66508249c5368a3da4633491738af22413adea61c0b28c7fa701d303060"}9{"id": "plan_survives-000008", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,-3\":\"outside\",\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"0,-3\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"1,-3\":\"outside\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"2,-3\":\"outside\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"3,-3\":\"outside\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"agent\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"4,-3\":\"outside\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"wall\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\",\"5,-3\":\"outside\",\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"6,-3\":\"outside\",\"6,-2\":\"outside\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"wall\",\"6,2\":\"wall\",\"6,3\":\"wall\",\"6,4\":\"wall\",\"6,5\":\"open\",\"7,-3\":\"outside\",\"7,-2\":\"outside\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"open\",\"7,3\":\"open\",\"7,4\":\"open\",\"7,5\":\"open\"}}", "question": "The agent will attempt this plan in order: south, east, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "5f246935e214886357f9b320890d29ab4e88a81af9fab7dbf57f01189502a258"}10{"id": "onward-000009", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 4. Rows increase to the south and columns to the east.\nRow -2: column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze\nRow -1: column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze\nRow 0: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 1: column 1 is open, column 2 is open, column 3 is open, column 4 is the agent, column 5 is open, column 6 is open, column 7 is open\nRow 2: column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is open, column 7 is wall\nRow 3: column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open\nRow 4: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall", "question": "Suppose the agent moves one cell west. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "a86104f0a6b510b5477c808c8e6743f48c6f57ed1102b66455ecd0e205379f64"}11{"id": "move_two_step-000010", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (8,9); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#####\n.#.##\n.#A##\n...##\n#####", "question": "The agent plans to move east, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "c641db93ff26128e8ffa90dd7c123ba71d3be55246e82dfe2d00a8fc783084fa"}12{"id": "distance_band-000011", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (19,1), goal at (19,9).\n########################\n#.....#...............##\n#.###.#########.#.###.##\n#.............#.#.#...##\n#.####.#.####.#.#.##.###\n#...#.................##\n###.#.#####.###.#.###.##\n#.#.......#...#...#...##\n#.#####.#.#.#.#...#.#.##\n#.......#.....#.#...#.##\n#.#.###.###.#.#.###.#.##\n#.......#.....#...#.#.##\n#.#.###.#.#######...#.##\n#.#.....#.#.....#...#.##\n###.#.###.#.#####.#.####\n#...#.......#.....#...##\n#.###.###..##.#######.##\n#.#...#.......#.....#.##\n#.#.###.#######.#.#.#.##\n#A#.#...#G......#.....##\n#.###.###########.#.#.##\n#.................#...##\n########################\n########################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 3, "answer": "20 to 39 moves", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "61cae5c310f464d98bef3f5706a27aed2e1a560f0f90ff8f2155646b23bfdd92"}13{"id": "goal_bearing-000012", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (17,25), goal at (3,11).\n############################\n#...............#.......#.##\n#.#..##.#######.#.###.#.#.##\n#.#.......#G......#.......##\n#.#.#.###.#########.####.###\n#.......#.......#.........##\n#.#########.#.#...###.###.##\n#.....#...#.#...#.#.....#.##\n#.###.#.#.###.###.#.###.#.##\n#.#.........#...#.#...#...##\n#.#.#######...#.#.#.#.#.####\n#.#.......#.#.#...#...#...##\n#.#.#.###.#.#.#.###.#.###.##\n#...#.....#...#.....#.....##\n###.###.#.###.#.###.#...####\n#...#...#.......#.#.#.#.#.##\n#.#.#.#########.#.#.#...#.##\n#.#...#.......#.#.....#.#A##\n#.#####.#####.#.#.#######.##\n#.#.........#.#.#.........##\n#...#######.#.#.###.#####.##\n#.#.....#.....#.........#.##\n#...#.#.#.###########.#.#.##\n#.#.....#.#.......#...#...##\n#.###.#.#.#.#.##.##.###.####\n#...........#.............##\n############################\n############################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "fc1dad683c1c8321587a28b15cf975cdcfdba5844bd9e43651422703b5e1dac1"}14{"id": "reachable_within-000013", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (6,5), goal at (13,15).\n########################\n#.....................##\n#.###############.###.##\n#.#.............#.#.#.##\n#.#.###########.#.#.#.##\n#.#.#.........#.#.#.#.##\n#.#.#A#######.#.#.#.#.##\n#.#.#.......#.#.#.#.#.##\n#.#.#########.#.#.#.#.##\n#.#...........#.#.#.#.##\n#.#############.#.#.#.##\n#...............#.#.#.##\n#.###############.#.#.##\n#.#............G#.#.#.##\n#.#.#############.#.#.##\n#.#.#.............#.#.##\n#.#.#.###########.#.#.##\n#.#.#...........#.#.#.##\n#.#.###########.#.#.#.##\n#.#.#...........#.#.#.##\n#.#.#############.#.#.##\n#.#...............#...##\n########################\n########################", "question": "Can the agent reach the goal in at most 144 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "c0f9cdef6376850269737948a8a9b1fc091a68e74a7734ce81ff9c3001b6fc0b"}15{"id": "snake_safe-000014", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,8), travelling east, length 3.\n..........\n.....*....\n..........\n..........\n..........\n......ooH.\n..........\n..........\n..........\n..........", "question": "If the snake moves west on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "b6b53da7ccbc395635fe0893df73f782b24646bda54eaf37513571c43a717118"}16{"id": "first_move-000015", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,7), goal at (7,1).\n#########\n#.......#\n#.###.#A#\n#.#.#...#\n#.#.#.#.#\n#...#...#\n###.#.#.#\n#G..#.#.#\n#########", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "50f23dee992fedc950e12aabae6be2d10ccebb3d4446c262fd89b89282c68779"}17{"id": "plan_survives-000016", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (13,15); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#.#.\n.#.#.\n.#A#.\n##.#.\n.#.#.", "question": "The agent will attempt this plan in order: east, south, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "40124050b21866e357e6708c2108394e346f88d77dc72a363fa46d06de9a5075"}18{"id": "snake_food-000017", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (3,5), travelling north, length 3.\n..............\n..............\n..............\n.....H..*.....\n.....oo.......\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "64ea470fe9db962066bcf0311ad2e2d02de51f20d28e1f8377f92579909ae6df"}19{"id": "only_way-000018", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (13,19); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.....#XXX\n######XXX\n.....#XXX\n.#.###XXX\n.#.#A#XXX\n.#.#.#XXX\n.#.#.#XXX\n##.#.#XXX\n...#.#XXX", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "89348b066a7dca50010a0d4c63775dc2646772ab9d256db04c8ac4c759a7f817"}20{"id": "plan_survives-000019", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (4,5); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#########\n...#.....\n.#.#.#.#.\n.....#.#.\n.###A#.#.\n...#.#...\n.#.####.#\n...#.....\n.#.#.#.#.", "question": "The agent will attempt this plan in order: east, north, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "2aab4a0d2734f1e9f977754046cbe39fa20176fd478be7a16d9ac472138f3fd4"}21{"id": "snake_food-000020", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (1,8), travelling north, length 4.\n..........\n........H*\n........o.\n.......oo.\n..........\n..........\n..........\n..........\n..........\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a837f65073cd5350f53dd7e0f20aacdcc073a5e592296f73011e53418949ed15"}22{"id": "onward-000021", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[19,20],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"17,18\":\"open\",\"17,19\":\"wall\",\"17,20\":\"wall\",\"17,21\":\"outside\",\"17,22\":\"outside\",\"18,18\":\"open\",\"18,19\":\"wall\",\"18,20\":\"open\",\"18,21\":\"outside\",\"18,22\":\"outside\",\"19,18\":\"open\",\"19,19\":\"wall\",\"19,20\":\"agent\",\"19,21\":\"outside\",\"19,22\":\"outside\",\"20,18\":\"open\",\"20,19\":\"open\",\"20,20\":\"open\",\"20,21\":\"outside\",\"20,22\":\"outside\",\"21,18\":\"outside\",\"21,19\":\"outside\",\"21,20\":\"outside\",\"21,21\":\"outside\",\"21,22\":\"outside\"}}", "question": "Suppose the agent moves one cell south. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "d145b31f1e3d9cc85ec0dc6da192a7ca087b9f2eac79de6ed894c57d3d67e8f0"}23{"id": "plan_survives-000022", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 0, column 0. Rows increase to the south and columns to the east.\nRow -3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze\nRow -2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze\nRow -1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze\nRow 0: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is the agent, column 1 is open, column 2 is open, column 3 is wall\nRow 1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is wall\nRow 2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open, column 3 is open\nRow 3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open", "question": "The agent will attempt this plan in order: south, south, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "4548a99df7234c38dafa3f5c2132be27207d52851b24be32a9473c307c9eee8b"}24{"id": "move_legal-000023", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[8,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"wall\",\"4,8\":\"wall\",\"4,9\":\"wall\",\"4,10\":\"wall\",\"4,11\":\"open\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"open\",\"5,9\":\"open\",\"5,10\":\"wall\",\"5,11\":\"open\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"open\",\"6,6\":\"wall\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"open\",\"6,10\":\"wall\",\"6,11\":\"open\",\"7,3\":\"open\",\"7,4\":\"wall\",\"7,5\":\"open\",\"7,6\":\"wall\",\"7,7\":\"open\",\"7,8\":\"wall\",\"7,9\":\"open\",\"7,10\":\"open\",\"7,11\":\"open\",\"8,3\":\"open\",\"8,4\":\"wall\",\"8,5\":\"open\",\"8,6\":\"wall\",\"8,7\":\"agent\",\"8,8\":\"wall\",\"8,9\":\"wall\",\"8,10\":\"wall\",\"8,11\":\"wall\",\"9,3\":\"open\",\"9,4\":\"wall\",\"9,5\":\"open\",\"9,6\":\"wall\",\"9,7\":\"open\",\"9,8\":\"wall\",\"9,9\":\"open\",\"9,10\":\"open\",\"9,11\":\"open\",\"10,3\":\"open\",\"10,4\":\"wall\",\"10,5\":\"open\",\"10,6\":\"wall\",\"10,7\":\"wall\",\"10,8\":\"wall\",\"10,9\":\"wall\",\"10,10\":\"wall\",\"10,11\":\"wall\",\"11,3\":\"open\",\"11,4\":\"wall\",\"11,5\":\"open\",\"11,6\":\"open\",\"11,7\":\"open\",\"11,8\":\"open\",\"11,9\":\"open\",\"11,10\":\"open\",\"11,11\":\"open\",\"12,3\":\"open\",\"12,4\":\"wall\",\"12,5\":\"wall\",\"12,6\":\"wall\",\"12,7\":\"wall\",\"12,8\":\"wall\",\"12,9\":\"wall\",\"12,10\":\"wall\",\"12,11\":\"wall\"}}", "question": "The agent wants to move east. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "c93d959b89c50d23c8144da234bf9b56bf2ff169461d5430755e42c29281dc44"}25{"id": "onward-000024", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX\n#########\n....A..#.\n##.#####.\n...#.....\n.###.###.\n...#.#...", "question": "Suppose the agent moves one cell east. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "ab6a608e5691ca1cf071bd0b440c996f8ee0ddd144e48266575ce2b1c29a964f"}26{"id": "move_two_step-000025", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 0, column 0. Rows increase to the south and columns to the east.\nRow -4: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze\nRow -3: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze\nRow -2: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze\nRow -1: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze\nRow 0: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is the agent, column 1 is open, column 2 is open, column 3 is open, column 4 is open\nRow 1: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is open\nRow 2: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open, column 3 is wall, column 4 is open\nRow 3: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open, column 3 is open, column 4 is wall\nRow 4: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is open", "question": "The agent plans to move south, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "96d24b48e19298d8e5bdee8c4c97a0fe176641b8b80cdfa406890b8bac90721b"}27{"id": "blocked_pair-000026", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,25); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#######\n#......\n#.#####\n#.#A#..\n#.#.#.#\n#.#.#.#\n#.#.#.#", "question": "Are BOTH the cell one step north and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "4760e312dc1e0104d306f6f655a9b9ba56d374ae76fb3967f5f53c043c7ab284"}28{"id": "onward-000027", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 7. Rows increase to the south and columns to the east.\nRow 4: column 4 is open, column 5 is open, column 6 is open, column 7 is wall, column 8 is open, column 9 is outside the maze, column 10 is outside the maze\nRow 5: column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is outside the maze, column 10 is outside the maze\nRow 6: column 4 is wall, column 5 is open, column 6 is open, column 7 is wall, column 8 is open, column 9 is outside the maze, column 10 is outside the maze\nRow 7: column 4 is open, column 5 is open, column 6 is open, column 7 is the agent, column 8 is open, column 9 is outside the maze, column 10 is outside the maze\nRow 8: column 4 is open, column 5 is open, column 6 is open, column 7 is wall, column 8 is open, column 9 is outside the maze, column 10 is outside the maze\nRow 9: column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze\nRow 10: column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze", "question": "Suppose the agent moves one cell west. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "01c9e7fbc9dc57b28d3131c1b75071d95c40d9732f5145cd5bf70842ba2636f4"}29{"id": "only_way-000028", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,33],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,31\":\"open\",\"1,32\":\"open\",\"1,33\":\"open\",\"1,34\":\"wall\",\"1,35\":\"wall\",\"2,31\":\"open\",\"2,32\":\"wall\",\"2,33\":\"open\",\"2,34\":\"wall\",\"2,35\":\"wall\",\"3,31\":\"open\",\"3,32\":\"wall\",\"3,33\":\"agent\",\"3,34\":\"wall\",\"3,35\":\"wall\",\"4,31\":\"wall\",\"4,32\":\"wall\",\"4,33\":\"wall\",\"4,34\":\"wall\",\"4,35\":\"wall\",\"5,31\":\"open\",\"5,32\":\"open\",\"5,33\":\"open\",\"5,34\":\"wall\",\"5,35\":\"wall\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "eff26073c612c9362a8d7689583595591ee8ef2d376d308e2b9401a8e8135905"}30{"id": "only_way-000029", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 9. Rows increase to the south and columns to the east.\nRow 3: column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is open, column 10 is open, column 11 is open, column 12 is open, column 13 is open\nRow 4: column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is open, column 12 is wall, column 13 is wall\nRow 5: column 5 is open, column 6 is wall, column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open\nRow 6: column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open\nRow 7: column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is the agent, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open\nRow 8: column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open\nRow 9: column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open\nRow 10: column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open\nRow 11: column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "4c40491135826383b5dbf73741e4c804cf68bb524c2030cbb4012557142d992f"}31{"id": "goal_bearing-000030", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (10,13), goal at (13,13).\n################\n#.....#.......##\n#.#.###.#.######\n#.#.#...#.....##\n#.#.#.#######.##\n#.#.#...#.#...##\n#.#.###.#.#.#.##\n#.#.#.....#.#.##\n#.###.#####.####\n#.#...#.#...#.##\n#.#.###.#.###A##\n#...#...#.....##\n#.###.#.########\n#.....#......G##\n################\n################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 1, "answer": "south", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b8da2dadeb634a414e126dac7b934c417d24b1e880fdbd4b1e7f537c636fb247"}32{"id": "stochastic-000031", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[3,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"agent\",\"3,2\":\"wall\",\"3,3\":\"open\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"open\",\"4,3\":\"open\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\"}}", "question": "The actuator executes the intended move with probability 0.70; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.20000000000000007, 0.7999999999999999], "deterministic": false, "fingerprint": "9f325cac3f09813129af845c7bd967385e76510c815f582b70a9df81b61177bb"}33{"id": "stochastic-000032", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[10,20],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"6,16\":\"wall\",\"6,17\":\"wall\",\"6,18\":\"open\",\"6,19\":\"open\",\"6,20\":\"wall\",\"6,21\":\"open\",\"6,22\":\"open\",\"6,23\":\"open\",\"6,24\":\"open\",\"7,16\":\"wall\",\"7,17\":\"open\",\"7,18\":\"open\",\"7,19\":\"wall\",\"7,20\":\"wall\",\"7,21\":\"wall\",\"7,22\":\"wall\",\"7,23\":\"open\",\"7,24\":\"wall\",\"8,16\":\"open\",\"8,17\":\"open\",\"8,18\":\"open\",\"8,19\":\"wall\",\"8,20\":\"wall\",\"8,21\":\"wall\",\"8,22\":\"open\",\"8,23\":\"open\",\"8,24\":\"open\",\"9,16\":\"wall\",\"9,17\":\"open\",\"9,18\":\"open\",\"9,19\":\"open\",\"9,20\":\"wall\",\"9,21\":\"open\",\"9,22\":\"open\",\"9,23\":\"open\",\"9,24\":\"open\",\"10,16\":\"wall\",\"10,17\":\"open\",\"10,18\":\"open\",\"10,19\":\"wall\",\"10,20\":\"agent\",\"10,21\":\"wall\",\"10,22\":\"open\",\"10,23\":\"wall\",\"10,24\":\"open\",\"11,16\":\"open\",\"11,17\":\"wall\",\"11,18\":\"open\",\"11,19\":\"wall\",\"11,20\":\"open\",\"11,21\":\"open\",\"11,22\":\"open\",\"11,23\":\"wall\",\"11,24\":\"wall\",\"12,16\":\"open\",\"12,17\":\"open\",\"12,18\":\"wall\",\"12,19\":\"wall\",\"12,20\":\"wall\",\"12,21\":\"wall\",\"12,22\":\"open\",\"12,23\":\"wall\",\"12,24\":\"wall\",\"13,16\":\"open\",\"13,17\":\"open\",\"13,18\":\"open\",\"13,19\":\"wall\",\"13,20\":\"open\",\"13,21\":\"open\",\"13,22\":\"wall\",\"13,23\":\"wall\",\"13,24\":\"wall\",\"14,16\":\"open\",\"14,17\":\"open\",\"14,18\":\"open\",\"14,19\":\"open\",\"14,20\":\"open\",\"14,21\":\"open\",\"14,22\":\"open\",\"14,23\":\"open\",\"14,24\":\"wall\"}}", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.4, 0.6], "deterministic": false, "fingerprint": "9dc2c9fe1d083530ed77c4ec15190f2ef037c7014502dec5471751b7fd43677a"}34{"id": "snake_food-000033", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,7), travelling south, length 3.\n............\n............\n............\n............\n............\n......oo....\n..*....H....\n............\n............\n............\n............\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "df89e9fd403f745804d7077a27318c7ad2891d10eaef180fc60457a0058561e6"}35{"id": "stochastic-000034", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 33, column 4. Rows increase to the south and columns to the east.\nRow 31: column 2 is wall, column 3 is open, column 4 is open, column 5 is open, column 6 is open\nRow 32: column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall\nRow 33: column 2 is open, column 3 is open, column 4 is the agent, column 5 is open, column 6 is wall\nRow 34: column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall\nRow 35: column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall", "question": "The actuator executes the intended move with probability 0.90; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9333333333333333, 0.06666666666666665], "deterministic": false, "fingerprint": "3d499cabf18b53535acf372486113eed5c5d95b3c7768c964d607c335b36907b"}36{"id": "plan_survives-000035", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[7,5],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"open\",\"3,7\":\"open\",\"3,8\":\"open\",\"3,9\":\"open\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"wall\",\"4,8\":\"wall\",\"4,9\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"open\",\"5,7\":\"open\",\"5,8\":\"open\",\"5,9\":\"open\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"6,7\":\"wall\",\"6,8\":\"wall\",\"6,9\":\"open\",\"7,1\":\"open\",\"7,2\":\"wall\",\"7,3\":\"open\",\"7,4\":\"open\",\"7,5\":\"agent\",\"7,6\":\"open\",\"7,7\":\"open\",\"7,8\":\"wall\",\"7,9\":\"open\",\"8,1\":\"open\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"wall\",\"8,6\":\"wall\",\"8,7\":\"wall\",\"8,8\":\"wall\",\"8,9\":\"open\",\"9,1\":\"open\",\"9,2\":\"open\",\"9,3\":\"open\",\"9,4\":\"open\",\"9,5\":\"open\",\"9,6\":\"open\",\"9,7\":\"open\",\"9,8\":\"open\",\"9,9\":\"open\",\"10,1\":\"wall\",\"10,2\":\"wall\",\"10,3\":\"wall\",\"10,4\":\"wall\",\"10,5\":\"wall\",\"10,6\":\"wall\",\"10,7\":\"wall\",\"10,8\":\"wall\",\"10,9\":\"wall\",\"11,1\":\"wall\",\"11,2\":\"wall\",\"11,3\":\"wall\",\"11,4\":\"wall\",\"11,5\":\"wall\",\"11,6\":\"wall\",\"11,7\":\"wall\",\"11,8\":\"wall\",\"11,9\":\"wall\"}}", "question": "The agent will attempt this plan in order: east, north, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "6bb69964fe62dd232a56fcdcd2b5437cbdedfa6fa19a44ee65254036627e61f0"}37{"id": "stochastic-000036", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 5, column 5. Rows increase to the south and columns to the east.\nRow 1: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is open, column 9 is open\nRow 2: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open\nRow 3: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open\nRow 4: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is wall\nRow 5: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is the agent, column 6 is wall, column 7 is wall, column 8 is open, column 9 is wall\nRow 6: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is open, column 7 is wall, column 8 is open, column 9 is open\nRow 7: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is open, column 7 is open, column 8 is open, column 9 is wall\nRow 8: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open, column 6 is open, column 7 is wall, column 8 is open, column 9 is open\nRow 9: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open", "question": "The actuator executes the intended move with probability 0.90; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9666666666666667, 0.033333333333333326], "deterministic": false, "fingerprint": "b570e326a87ba76963b5314193a7a745d4ddfed628baed2129893142a982057d"}38{"id": "move_two_step-000037", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (14,20); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n##...XXXX\n####.XXXX\n##...XXXX\n###..XXXX\n.#.#AXXXX\n...#.XXXX\n.#.##XXXX\n.####XXXX\n....#XXXX", "question": "The agent plans to move south, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "db3834e217c386ef5a2f79f27d29f5a180ea54f890ef315ceeee3326eaad8d72"}39{"id": "plan_survives-000038", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[7,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"agent\",\"7,2\":\"open\",\"7,3\":\"open\",\"8,-1\":\"outside\",\"8,0\":\"wall\",\"8,1\":\"wall\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"9,-1\":\"outside\",\"9,0\":\"wall\",\"9,1\":\"open\",\"9,2\":\"open\",\"9,3\":\"open\"}}", "question": "The agent will attempt this plan in order: east, east, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "a233f6caa6d19d44bf506e43e6a8d3d8ab1fa23e82fcc73779ddfdab953db040"}40{"id": "snake_food-000039", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,8), travelling east, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n.*.....oH.....\n.......o......\n..............\n..............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "650bf5379b08d180cc10848026e890fe2f27c1895722e79d6f3b19fdaec1a96d"}41{"id": "move_two_step-000040", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 9. Rows increase to the south and columns to the east.\nRow -1: column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 0: column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall\nRow 1: column 7 is open, column 8 is wall, column 9 is the agent, column 10 is wall, column 11 is wall\nRow 2: column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall\nRow 3: column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is wall", "question": "The agent plans to move south, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "58952618474126e273a8d6e77660fd3dccbb41a953c07731b451a1dce7f40c55"}42{"id": "plan_survives-000041", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXX\nXXXXXXX\nXX#####\nXX#A...\nXX###.#\nXX#....\nXX#.#.#", "question": "The agent will attempt this plan in order: east, north, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "a6dab85f5c8d9fbc18fd3a81cc0957222bf9037f93b9ccf82852434c8a6bc480"}43{"id": "reachable_within-000042", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,23), goal at (17,29).\n################################\n#...#...#.........#.....#.....##\n#.###.#.#.#####.###.###.#.###.##\n#.....#.#...#.#...#.#...#.#.#.##\n#.#####.###.#.###.#.#.###.#.#.##\n#.#...#...#.#...#...#..A#.#.#.##\n#.###.###.#.###.#######.#.#.#.##\n#...#...#...#.......#...#.#...##\n###.###.#####.#######.###.###.##\n#.#.#...#.....#.....#.#.....#.##\n#.#.#.#.#.#.###.###.#.#####.#.##\n#.#...#...#.#...#.....#.#...#.##\n#.#########.#.#####.###.#.###.##\n#.#.....#...#.....#.#...#.#.#.##\n#.#.###.#.#######.#.#.###.#.#.##\n#.#...#.#.#...#...#.#.#...#...##\n#.###.#.#.###.#.###.#.###.#.####\n#...#.#.#...#.#...#.#...#.#.#G##\n#.###.#.###.#.###.#####.#.#.#.##\n#.#...#.#...#...#.#...#.#.#...##\n#.#.###.#.###.#.#.#.#.#.#.######\n#...#.#...#...#.#.#.#.#.#.....##\n#.###.#########.#.#.#.#.#####.##\n#.....#.......#.#...#...#.....##\n#####.#.#####.#.#########.###.##\n#...#.#.#.#...#...#...#...#...##\n#.#.#.#.#.#.###.#.#.#.#.###.####\n#.#...#.#.#...#.#.#.#.#.#.#...##\n#.#####.#.###.#.#.#.#.#.#.###.##\n#.......#.......#...#...#.....##\n################################\n################################", "question": "Can the agent reach the goal in at most 269 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "3f2185b131e39df8c58f7b227ab9271c434111a86cdb7fdc715865c34c42e82f"}44{"id": "stochastic-000043", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 6, column 6. Rows increase to the south and columns to the east.\nRow 4: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall\nRow 5: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall\nRow 6: column 4 is wall, column 5 is wall, column 6 is the agent, column 7 is wall, column 8 is wall\nRow 7: column 4 is wall, column 5 is wall, column 6 is open, column 7 is open, column 8 is wall\nRow 8: column 4 is wall, column 5 is wall, column 6 is open, column 7 is open, column 8 is open", "question": "The actuator executes the intended move with probability 0.75; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.25, 0.75], "deterministic": false, "fingerprint": "90920e7e3f15a16dd744a617f0e6a383aaeea5c85dcfd2bcb00b431e8289e967"}45{"id": "move_two_step-000044", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (18,35); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n###XX\n.##XX\n..AXX\n.#.XX\n..#XX", "question": "The agent plans to move south, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "61537906574f0b390bdbe4d3fd895e256f8d50bbbe1a07bb33e597cd3f1a6200"}46{"id": "plan_survives-000045", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (19,23); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#...\n.####\n.#A..\n.###.\n...#.", "question": "The agent will attempt this plan in order: north, east, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "ce4f7ff7c46df67754b10f57895d7956f19c061871d6d9eb22ac9a30d2db735f"}47{"id": "blocked_pair-000046", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"1,6\":\"wall\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"wall\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"agent\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"wall\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"wall\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"open\",\"6,6\":\"wall\"}}", "question": "Are BOTH the cell one step east and the cell one step west blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "9a81b54c1495bc5f3d983ac1215a8da6342b45c7b2230ee98b550a24dc04f820"}48{"id": "reachable_within-000047", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (11,7), goal at (17,19).\n########################\n#.............#.......##\n#.#.#########.#.#.###.##\n#.#.#.......#.#.#.#.#.##\n#.#.#.#####.#.#.#.#.#.##\n#.#.#.#...#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#A#.#.#.#.#.#.#.##\n###.#.###.#.#.#.#.#.#.##\n#...#.....#.#.#.#.#.#.##\n#.#########.#.#.#.#.#.##\n#.#.........#.#.#.#...##\n#.#.#######.#.#.#.###.##\n#.#.#.......#.#.#.#G#.##\n#.#.#########.###.#.#.##\n#.#...............#.#.##\n#.#################.#.##\n#...................#.##\n########################\n########################", "question": "Can the agent reach the goal in at most 142 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "5764ba8477fe5b7b065f7f744924ccd772bdab468039f9a51666eede45e57f46"}49{"id": "plan_survives-000048", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 25, column 7. Rows increase to the south and columns to the east.\nRow 22: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall\nRow 23: column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open, column 10 is open\nRow 24: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall\nRow 25: column 4 is open, column 5 is open, column 6 is open, column 7 is the agent, column 8 is open, column 9 is open, column 10 is wall\nRow 26: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall\nRow 27: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall\nRow 28: column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze", "question": "The agent will attempt this plan in order: west, south, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "418b96bad1aca3f6b6dd670e70c4bc0072c4bbbe34c07f1cb6e578cc1dc9ee3c"}50{"id": "distance_band-000049", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (11,11), goal at (13,13).\n################\n#.......#.....##\n#.#.###.#.###.##\n#.#.....#...#.##\n#.###...###.#.##\n#.#...#.......##\n###.#####.###.##\n#...#.........##\n#.#####.#.#.####\n#.#.........#.##\n#...#.#.#.#.#.##\n#.#.#.#.#..A..##\n#.#.#.#.###.#.##\n#...........#G##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "2b3dce76a3ad22469448b73c7f25486253651df5ca9e29ae99c1400a57861b6b"}51{"id": "reachable_within-000050", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,13), goal at (3,13).\n################\n#.#...........##\n#.#.###.#####.##\n#.....#.....#G##\n#.#.###.###.####\n#...#.....#...##\n#.#.#.#######.##\n#.....#.......##\n###.#.#.###.#.##\n#...#...#.#...##\n#.#.#####.#.####\n#.#.......#...##\n#.##.##.#.#.#.##\n#...........#A##\n################\n################", "question": "Can the agent reach the goal in at most 27 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "1f95a6ab6118a0d3ffcc08d3e166ae7a7339e980f3db9dae40a7230f8d57546e"}52{"id": "stochastic-000051", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[19,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"16,-2\":\"outside\",\"16,-1\":\"outside\",\"16,0\":\"wall\",\"16,1\":\"open\",\"16,2\":\"wall\",\"16,3\":\"wall\",\"16,4\":\"wall\",\"17,-2\":\"outside\",\"17,-1\":\"outside\",\"17,0\":\"wall\",\"17,1\":\"open\",\"17,2\":\"open\",\"17,3\":\"open\",\"17,4\":\"open\",\"18,-2\":\"outside\",\"18,-1\":\"outside\",\"18,0\":\"wall\",\"18,1\":\"open\",\"18,2\":\"wall\",\"18,3\":\"wall\",\"18,4\":\"open\",\"19,-2\":\"outside\",\"19,-1\":\"outside\",\"19,0\":\"wall\",\"19,1\":\"agent\",\"19,2\":\"open\",\"19,3\":\"open\",\"19,4\":\"open\",\"20,-2\":\"outside\",\"20,-1\":\"outside\",\"20,0\":\"wall\",\"20,1\":\"wall\",\"20,2\":\"wall\",\"20,3\":\"wall\",\"20,4\":\"wall\",\"21,-2\":\"outside\",\"21,-1\":\"outside\",\"21,0\":\"outside\",\"21,1\":\"outside\",\"21,2\":\"outside\",\"21,3\":\"outside\",\"21,4\":\"outside\",\"22,-2\":\"outside\",\"22,-1\":\"outside\",\"22,0\":\"outside\",\"22,1\":\"outside\",\"22,2\":\"outside\",\"22,3\":\"outside\",\"22,4\":\"outside\"}}", "question": "The actuator executes the intended move with probability 0.90; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9333333333333333, 0.06666666666666665], "deterministic": false, "fingerprint": "384071d61878d78f84b01b4fddf9d64023dd5cec13513c590b9fdfa88db9be7a"}53{"id": "move_legal-000052", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 8, column 0. Rows increase to the south and columns to the east.\nRow 5: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is open, column 3 is open\nRow 6: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is wall\nRow 7: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is wall, column 3 is open\nRow 8: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is the agent, column 1 is wall, column 2 is open, column 3 is open\nRow 9: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze\nRow 10: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze\nRow 11: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze", "question": "The agent wants to move south. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "280a153f4aa0bb215d785155353de125532f80fcc41f4a3bd7f0df725147a637"}54{"id": "first_move-000053", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,9), goal at (1,1).\n############\n#G.......A##\n#########.##\n#.......#.##\n#.#####.#.##\n#.#...#.#.##\n#.#.#.#.#.##\n#.#.#.#.#.##\n#.###.#.#.##\n#.....#...##\n############\n############", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "f137fbb9b608252320ab42bab41ea49339cf371ffc6c1a0a07adc3b666e63259"}55{"id": "reachable_within-000054", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,1), goal at (21,5).\n########################\n#.......#.#.......#...##\n#.#.###.#.#.###.#.#.#.##\n#.#...#.......#.#...#.##\n#.#.#.#.#.#####.#####.##\n#...#.#.#.#.....#.....##\n#.#####.#.#.#####.#.#.##\n#.......#.#...#.#.....##\n#.###########.....#.#.##\n#A..............#.#.#.##\n#.#.#.####.####.#.#.#.##\n#...#.#.....#...#.#.#.##\n#.###.#.###.#.#.#.###.##\n#.........#.#.#...#...##\n#.#.###.###.#.#####.#.##\n#.#...#.....#.......#.##\n#.###.#.#############.##\n#...#.#.#.............##\n#.#.#.#.###.####.##.####\n#.#.#.#...#...#.......##\n#.#.#.###.#.#.#.#.###.##\n#....G......#...#.....##\n########################\n########################", "question": "Can the agent reach the goal in at most 15 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "00bffa6f5de47f52dbd766663f9815767fdb2f638bdc2962865a8d035bf0a501"}56{"id": "only_way-000055", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,0],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-3,-4\":\"outside\",\"-3,-3\":\"outside\",\"-3,-2\":\"outside\",\"-3,-1\":\"outside\",\"-3,0\":\"outside\",\"-3,1\":\"outside\",\"-3,2\":\"outside\",\"-3,3\":\"outside\",\"-3,4\":\"outside\",\"-2,-4\":\"outside\",\"-2,-3\":\"outside\",\"-2,-2\":\"outside\",\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-1,-4\":\"outside\",\"-1,-3\":\"outside\",\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"0,-4\":\"outside\",\"0,-3\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"open\",\"0,1\":\"open\",\"0,2\":\"open\",\"0,3\":\"open\",\"0,4\":\"open\",\"1,-4\":\"outside\",\"1,-3\":\"outside\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"agent\",\"1,1\":\"wall\",\"1,2\":\"wall\",\"1,3\":\"open\",\"1,4\":\"wall\",\"2,-4\":\"outside\",\"2,-3\":\"outside\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"open\",\"2,3\":\"open\",\"2,4\":\"open\",\"3,-4\":\"outside\",\"3,-3\":\"outside\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"wall\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"open\",\"4,-4\":\"outside\",\"4,-3\":\"outside\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"open\",\"4,1\":\"open\",\"4,2\":\"open\",\"4,3\":\"wall\",\"4,4\":\"open\",\"5,-4\":\"outside\",\"5,-3\":\"outside\",\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\",\"5,4\":\"open\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "ba0d6c1bca231e8e2e096265b6be7bb7e0e94471a708be2f394ab00acc8c8f5e"}57{"id": "move_two_step-000056", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 26, column 1. Rows increase to the south and columns to the east.\nRow 22: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is open, column 4 is open, column 5 is open\nRow 23: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 24: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is wall, column 4 is wall, column 5 is open\nRow 25: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 26: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is the agent, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 27: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is wall, column 4 is wall, column 5 is wall\nRow 28: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 29: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is wall, column 5 is wall\nRow 30: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open", "question": "The agent plans to move south, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "a3bd2a10adb0835c5379678d4626fb3ba741549216f6a42086d32367da65d75b"}58{"id": "open_count-000057", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (17,0); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXX.....\nXXXX#...#\nXXXX..#.#\nXXXX....#\nXXXXA...#\nXXXX..#..\nXXXX.....\nXXXX#.#.#\nXXXXXXXXX", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 3, "answer": "3", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "9dc5b54bc794abcf290d713e78a7580ece273b085aa3bfb8e6e02cd5a3e1ac9b"}59{"id": "stochastic-000058", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (7,2); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX......\nX...#..\nX##..#.\nX#.A#..\nX##.##.\nXXXXXXX\nXXXXXXX", "question": "The actuator executes the intended move with probability 0.75; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.75, 0.25], "deterministic": false, "fingerprint": "d73ff8a7797527fa9c8ed42af0126fda762dc67a4f4adf81366ff31afbf9f33f"}60{"id": "plan_survives-000059", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (19,21); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n...#.\n.#.#.\n.#A#.\n.#.#.\n.#.#.", "question": "The agent will attempt this plan in order: north, south, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a6a15a799c38a0910c95d0683a4234a7a18cc6bd78ae224aaa1ead1f13a4fccf"}61{"id": "snake_food-000060", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (9,9), travelling south, length 3.\n.........*....\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n........oo....\n.........H....\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "f9ff5da146c2b633b95bd4e96b8b4ab9eaa5c678a8450adf9f63a04ae33abf5c"}62{"id": "snake_safe-000061", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (3,6), travelling north, length 3.\n..........\n..........\n..........\n......H...\n......o...\n......o...\n.....*....\n..........\n..........\n..........", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "09f17b37c0c623cf624b2149fbbbb907c02c5373b4ac2096685597dc99695890"}63{"id": "blocked_pair-000062", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[23,4],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"21,2\":\"wall\",\"21,3\":\"open\",\"21,4\":\"open\",\"21,5\":\"open\",\"21,6\":\"wall\",\"22,2\":\"wall\",\"22,3\":\"wall\",\"22,4\":\"wall\",\"22,5\":\"open\",\"22,6\":\"wall\",\"23,2\":\"wall\",\"23,3\":\"open\",\"23,4\":\"agent\",\"23,5\":\"open\",\"23,6\":\"wall\",\"24,2\":\"wall\",\"24,3\":\"wall\",\"24,4\":\"wall\",\"24,5\":\"wall\",\"24,6\":\"wall\",\"25,2\":\"wall\",\"25,3\":\"open\",\"25,4\":\"open\",\"25,5\":\"open\",\"25,6\":\"open\"}}", "question": "Are BOTH the cell one step north and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "e9504cf70059569a6f3effd4208438c3981163c828a7e84278fa66bee1ff2d40"}64{"id": "plan_survives-000063", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (17,12); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n..#..\n###.#\n#.A.#\n#####\n....#", "question": "The agent will attempt this plan in order: east, north, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "9b8f9c3f752946b4040d8779b099d3ac35e2203765fc0a8acad51e4095462c6f"}65{"id": "stochastic-000064", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[2,5],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"1,3\":\"open\",\"1,4\":\"wall\",\"1,5\":\"open\",\"1,6\":\"open\",\"1,7\":\"open\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"agent\",\"2,6\":\"wall\",\"2,7\":\"wall\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"open\"}}", "question": "The actuator executes the intended move with probability 0.85; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9, 0.10000000000000002], "deterministic": false, "fingerprint": "cdb399bbf35f43a20df12f8fb29ec2787dc1377295882a7fa3da12fa02e60cf3"}66{"id": "blocked_pair-000065", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (22,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX#.#.#.#.\nX#.#.#.#.\nX#.#.#.#.\nX#.#.#.#.\nX#.#A#.#.\nX#.#.#.#.\nX#.###.#.\nX#.....#.\nX########", "question": "Are BOTH the cell one step north and the cell one step west blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "81b28edc28746367303cd9ca17a6995ea1b71aa3a5f40893b79dee83531d7068"}67{"id": "snake_food-000066", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,3), travelling west, length 3.\n..........\n..........\n..........\n..........\n..........\n..........\n..........\n..........\n...Hoo....\n...*......", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 1, "answer": "south", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "995671643f691c79ea685827965be8efa96ca43744334348774db30f74e6a860"}68{"id": "only_way-000067", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 7. Rows increase to the south and columns to the east.\nRow 3: column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall\nRow 4: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall\nRow 5: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall\nRow 6: column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall\nRow 7: column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is the agent, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall\nRow 8: column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall\nRow 9: column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is wall\nRow 10: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall\nRow 11: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "fb517f0633875aa7f428abf336cd05c0799226a876d55edfeacd14a82205728c"}69{"id": "plan_survives-000068", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[16,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"14,7\":\"open\",\"14,8\":\"wall\",\"14,9\":\"open\",\"14,10\":\"wall\",\"14,11\":\"open\",\"15,7\":\"open\",\"15,8\":\"wall\",\"15,9\":\"open\",\"15,10\":\"wall\",\"15,11\":\"open\",\"16,7\":\"open\",\"16,8\":\"wall\",\"16,9\":\"agent\",\"16,10\":\"wall\",\"16,11\":\"open\",\"17,7\":\"open\",\"17,8\":\"wall\",\"17,9\":\"open\",\"17,10\":\"wall\",\"17,11\":\"open\",\"18,7\":\"open\",\"18,8\":\"wall\",\"18,9\":\"wall\",\"18,10\":\"wall\",\"18,11\":\"open\"}}", "question": "The agent will attempt this plan in order: south, west, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "fcfd3972dc2f1182274a0e6f9ce2be561d9437505bc84dd2166a35f4c8eb75e7"}70{"id": "distance_band-000069", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,13), goal at (13,1).\n################\n#.............##\n#.##..##.####.##\n#.....#.....#.##\n###.#...#.###.##\n#.....#.#.....##\n#.###.###.#.####\n#...........#A##\n#.#.#.#.###.#.##\n#.#...#.....#.##\n#.#.#.###.#.#.##\n#.#.#.#.#.....##\n#.#.#.#.#####.##\n#G..#.........##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 3, "answer": "20 to 39 moves", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "0b1b0db61f70e4b24b87b6dfda926ca64844c07c924fba6bc5cb0417a61c2b29"}71{"id": "goal_bearing-000070", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,1), goal at (5,7).\n#########\n#A..#...#\n#.###.#.#\n#.#.....#\n#.###.#.#\n#.#...#G#\n#.#.#.#.#\n#...#...#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 6, "answer": "south-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "8716c279fb58139a47446f110d5bdfd911f905b19880310b268ddec8a0b6702f"}72{"id": "first_move-000071", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,2), goal at (5,5).\n#########\n#.......#\n#.###.###\n#.#.#...#\n#.#.###.#\n#....G#.#\n#######.#\n#.A.....#\n#########", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "c094b8a5a8f043e3d7e18e7e6299b33b42cec47377165766091dcb55d4fbb777"}73{"id": "goal_bearing-000072", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,7), goal at (9,9).\n############\n#.........##\n#.#####.####\n#.....#...##\n#####.###.##\n#.....#...##\n#.#####.####\n#...#.#...##\n###.#.###.##\n#......A#G##\n############\n############", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "53baad5f39be54a4137d6cf6f17743be90563e435dc77cbced697e68c30c9037"}74{"id": "reachable_within-000073", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (18,11), goal at (19,1).\n########################\n#.....................##\n#.#.###.###.#########.##\n#.#...#...........#...##\n#.###.###.#.#.###...####\n#.#.......#...#.#.#.#.##\n###.#.###.#.###.#.#.#.##\n#...#...........#.#...##\n#.#.#.#.#########.###.##\n#...#.#...#.........#.##\n#.#.#.#.#.#.#.#.#.#.#.##\n#.#...........#.#.#...##\n#.#####.#.#.#.#.....#.##\n#...........#...#.#...##\n#######.#.###.###.#.#.##\n#.....#.#...#.....#...##\n#.###.#.###.###.###.#.##\n#.....#.....#.......#.##\n#.#.#.###.#A..#.#.#.#.##\n#G#.#.......#...#...#.##\n#.#.#.#.#.###.#.#.#.#.##\n#.....#.......#...#...##\n########################\n########################", "question": "Can the agent reach the goal in at most 19 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "8d1817f6d91f94ff76a0e4441814095db45e2c52fc59c3b126d784c67f567577"}75{"id": "open_count-000074", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 29, column 28. Rows increase to the south and columns to the east.\nRow 27: column 26 is wall, column 27 is open, column 28 is open, column 29 is open, column 30 is open\nRow 28: column 26 is open, column 27 is wall, column 28 is wall, column 29 is wall, column 30 is open\nRow 29: column 26 is wall, column 27 is open, column 28 is the agent, column 29 is open, column 30 is wall\nRow 30: column 26 is wall, column 27 is open, column 28 is open, column 29 is wall, column 30 is wall\nRow 31: column 26 is wall, column 27 is wall, column 28 is open, column 29 is wall, column 30 is wall", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 3, "answer": "3", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "aa6cca5ddcb5f4ff2ed48ee91a577a1fffa09c8444984638d2f2328d30e6595e"}76{"id": "blocked_pair-000075", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[7,8],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"open\",\"5,9\":\"open\",\"5,10\":\"wall\",\"6,6\":\"wall\",\"6,7\":\"wall\",\"6,8\":\"wall\",\"6,9\":\"wall\",\"6,10\":\"wall\",\"7,6\":\"open\",\"7,7\":\"open\",\"7,8\":\"agent\",\"7,9\":\"open\",\"7,10\":\"wall\",\"8,6\":\"wall\",\"8,7\":\"wall\",\"8,8\":\"wall\",\"8,9\":\"wall\",\"8,10\":\"wall\",\"9,6\":\"open\",\"9,7\":\"open\",\"9,8\":\"open\",\"9,9\":\"open\",\"9,10\":\"wall\"}}", "question": "Are BOTH the cell one step east and the cell one step south blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "684ef790e1910d33a25a5a2971a05c19b89428efe0a53f95f8c13e4f9261270e"}77{"id": "cell_kind-000076", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (4,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX########\nX########\nX######..\nX####....\nX#..A..#.\nX#.#..#..\nX.#..##..\nX......#.\nX##.#.##.", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "506584311ed67948642f138feff4a32db8efa6b32dfe5a42baf39ee6661db3ee"}78{"id": "plan_survives-000077", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[0,2],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-3,-1\":\"outside\",\"-3,0\":\"outside\",\"-3,1\":\"outside\",\"-3,2\":\"outside\",\"-3,3\":\"outside\",\"-3,4\":\"outside\",\"-3,5\":\"outside\",\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-2,5\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"open\",\"0,2\":\"agent\",\"0,3\":\"open\",\"0,4\":\"open\",\"0,5\":\"open\",\"1,-1\":\"outside\",\"1,0\":\"open\",\"1,1\":\"wall\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"2,-1\":\"outside\",\"2,0\":\"open\",\"2,1\":\"wall\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"open\",\"2,5\":\"wall\",\"3,-1\":\"outside\",\"3,0\":\"open\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\"}}", "question": "The agent will attempt this plan in order: east, south, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b50480a2971162e05fba109be5cffc802a8ee8f0069daa4ad101999581d9e89c"}79{"id": "snake_food-000078", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,6), travelling east, length 3.\n..........\n..........\n..........\n.....*....\n..........\n....ooH...\n..........\n..........\n..........\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "61e7f5ad6c54bdfc1f401714f8134734b007c904a802f8077d90b062ebcf11b1"}80{"id": "snake_food-000079", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (10,9), travelling south, length 3.\n..............\n..............\n..............\n..............\n........*.....\n..............\n..............\n..............\n.........o....\n.........o....\n.........H....\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "c6507e2315a7a88debe03b55195f54540d529806e77fe61f952abe16fef7aea8"}81{"id": "cell_kind-000080", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (9,5); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n####.##\n#......\n#.#####\n#.#A#..\n#.#.#.#\n#.#.#..\n#.#.###", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "456f6f80c25ebe1a2765421d3e1ac2b81282a789c4fdf3dc93d60e36093de31d"}82{"id": "move_two_step-000081", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 8, column 13. Rows increase to the south and columns to the east.\nRow 5: column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall\nRow 6: column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall\nRow 7: column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall\nRow 8: column 10 is wall, column 11 is open, column 12 is wall, column 13 is the agent, column 14 is wall, column 15 is open, column 16 is wall\nRow 9: column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall\nRow 10: column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall\nRow 11: column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall", "question": "The agent plans to move south, then east. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "0a9430114a5a3a26b2a37db1e19d3a8728cb2ff8daae4c0cd489f2405ae4dcee"}83{"id": "stochastic-000082", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (8,21); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.####\n.#...\n.#A#.\n...#.\n#####", "question": "The actuator executes the intended move with probability 0.85; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.09999999999999998, 0.9], "deterministic": false, "fingerprint": "2a3599f39d6aa523057bc2804f80d79f1185a37ca18714015d318121fa259fb3"}84{"id": "move_legal-000083", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (3,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\n######XXX\n.....#XXX\n.###.#XXX\n...#A#XXX\n##.###XXX\n.#...#XXX\n####.#XXX\n.....#XXX", "question": "The agent wants to move north. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "4098436a675222a0cf6c6778f698cd9c12e8e6fe881306bf6fe8d86b0489f87f"}85{"id": "move_two_step-000084", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[12,5],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"9,2\":\"open\",\"9,3\":\"open\",\"9,4\":\"open\",\"9,5\":\"open\",\"9,6\":\"open\",\"9,7\":\"open\",\"9,8\":\"open\",\"10,2\":\"wall\",\"10,3\":\"wall\",\"10,4\":\"wall\",\"10,5\":\"wall\",\"10,6\":\"wall\",\"10,7\":\"open\",\"10,8\":\"wall\",\"11,2\":\"wall\",\"11,3\":\"open\",\"11,4\":\"wall\",\"11,5\":\"open\",\"11,6\":\"wall\",\"11,7\":\"open\",\"11,8\":\"wall\",\"12,2\":\"wall\",\"12,3\":\"open\",\"12,4\":\"wall\",\"12,5\":\"agent\",\"12,6\":\"wall\",\"12,7\":\"open\",\"12,8\":\"wall\",\"13,2\":\"wall\",\"13,3\":\"open\",\"13,4\":\"wall\",\"13,5\":\"open\",\"13,6\":\"wall\",\"13,7\":\"open\",\"13,8\":\"wall\",\"14,2\":\"wall\",\"14,3\":\"open\",\"14,4\":\"wall\",\"14,5\":\"open\",\"14,6\":\"wall\",\"14,7\":\"open\",\"14,8\":\"wall\",\"15,2\":\"wall\",\"15,3\":\"open\",\"15,4\":\"wall\",\"15,5\":\"open\",\"15,6\":\"wall\",\"15,7\":\"open\",\"15,8\":\"wall\"}}", "question": "The agent plans to move south, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "b3168ae7e8711ff21797af10246b3aca20f22c9573700c9f5a2c9446501f03ab"}86{"id": "first_move-000085", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,6), goal at (9,1).\n############\n#.........##\n#.#######.##\n#.#.......##\n#.#.########\n#.#.#.....##\n#.#.#.###.##\n#.#.#.A.#.##\n#.#.#####.##\n#G#.......##\n############\n############", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "6a8677a6814886a17c521e242b1fdc65f6eccabb35803845f52c57969748a26a"}87{"id": "first_move-000086", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,6), goal at (11,1).\n.....#.#####\n....#.A#####\n#.....######\n##...#######\n.#.#########\n...#########\n.#.#########\n....#.######\n.#....######\n.##..#######\n....########\n.G.#########", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "f1c942878f35e18a40d528bb7d88167471d6d41b422e4fdb13ce0b46e2ce7aae"}88{"id": "cell_kind-000087", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 19, column 19. Rows increase to the south and columns to the east.\nRow 16: column 16 is wall, column 17 is open, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze\nRow 17: column 16 is wall, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze\nRow 18: column 16 is wall, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze\nRow 19: column 16 is open, column 17 is open, column 18 is open, column 19 is the agent, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze\nRow 20: column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze\nRow 21: column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze, column 20 is outside the maze, column 21 is outside the maze, column 22 is outside the maze\nRow 22: column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze, column 20 is outside the maze, column 21 is outside the maze, column 22 is outside the maze", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "f25dc2eeb05d7f389124d4075228f5832275855e660240d1edb96034a51e4854"}89{"id": "distance_band-000088", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (14,11), goal at (1,5).\n########################\n#...#G..#.....#.......##\n#.#.###.#.###.#.###.#.##\n#.#...#.#.#...#.#...#.##\n#.###.#.#.#.#.#.#.#.####\n#...#.#...#.#.#.#.#.#.##\n#.#.#.#####.#.#.#.###.##\n#.#.#.....#.#.#.#.....##\n###.#####.#.#.#.########\n#...#.....#.#.#.#...#.##\n#.###.#####.#.#.#.#.#.##\n#.#...#.....#.#...#...##\n#.#.###.#.###########.##\n#.#.#...#.#.........#.##\n#.#.#.#.###A#######.#.##\n#.#.#.#.#...#...#...#.##\n#.#.#.###.###.#.#.#.#.##\n#.#...#...#...#.#.#.#.##\n#.#####.###.###.#.#.#.##\n#.#.....#...#...#.#.#.##\n#.#.#########.#.#.###.##\n#.............#.#.....##\n########################\n########################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 4, "answer": "40 moves or more", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "d68a18958e9f9f95b51c2e2a5d723fb459eef21717eb9577427ac6201c3bd8d3"}90{"id": "only_way-000089", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[26,15],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"23,12\":\"open\",\"23,13\":\"open\",\"23,14\":\"wall\",\"23,15\":\"open\",\"23,16\":\"open\",\"23,17\":\"open\",\"23,18\":\"wall\",\"24,12\":\"wall\",\"24,13\":\"open\",\"24,14\":\"open\",\"24,15\":\"open\",\"24,16\":\"open\",\"24,17\":\"wall\",\"24,18\":\"wall\",\"25,12\":\"open\",\"25,13\":\"open\",\"25,14\":\"open\",\"25,15\":\"wall\",\"25,16\":\"wall\",\"25,17\":\"open\",\"25,18\":\"open\",\"26,12\":\"open\",\"26,13\":\"open\",\"26,14\":\"wall\",\"26,15\":\"agent\",\"26,16\":\"open\",\"26,17\":\"wall\",\"26,18\":\"open\",\"27,12\":\"open\",\"27,13\":\"open\",\"27,14\":\"wall\",\"27,15\":\"wall\",\"27,16\":\"open\",\"27,17\":\"open\",\"27,18\":\"open\",\"28,12\":\"outside\",\"28,13\":\"outside\",\"28,14\":\"outside\",\"28,15\":\"outside\",\"28,16\":\"outside\",\"28,17\":\"outside\",\"28,18\":\"outside\",\"29,12\":\"outside\",\"29,13\":\"outside\",\"29,14\":\"outside\",\"29,15\":\"outside\",\"29,16\":\"outside\",\"29,17\":\"outside\",\"29,18\":\"outside\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "5c9440f5bd952702bc666fa06c8a1ffc3e0000504fda5a02b0cd05dd4d5d641e"}91{"id": "move_legal-000090", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 3, column 12. Rows increase to the south and columns to the east.\nRow 1: column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is open\nRow 2: column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall\nRow 3: column 10 is wall, column 11 is open, column 12 is the agent, column 13 is open, column 14 is open\nRow 4: column 10 is wall, column 11 is open, column 12 is wall, column 13 is wall, column 14 is wall\nRow 5: column 10 is wall, column 11 is open, column 12 is open, column 13 is open, column 14 is wall", "question": "If the agent attempts one cell south (row+1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "5d27cefa901259d2a3faed17a575182326e5203c1b5ef5fdf19a1afc629947ca"}92{"id": "onward-000091", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 25, column 7. Rows increase to the south and columns to the east.\nRow 23: column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is open\nRow 24: column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open\nRow 25: column 5 is open, column 6 is open, column 7 is the agent, column 8 is wall, column 9 is open\nRow 26: column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall\nRow 27: column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall", "question": "Suppose the agent moves one cell west. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "0884190099c56007db9b8b854fc9bc1504f1bd1b44cc6e02c5c9bb07c3b73d15"}93{"id": "snake_safe-000092", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,7), travelling south, length 3.\n............\n............\n............\n...*........\n............\n............\n......oo....\n.......H....\n............\n............\n............\n............", "question": "If the snake moves west on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "a02cf2895ac653778827cb3747fa626fba18b4352c0c0aedbf0876e1e4a428c7"}94{"id": "blocked_pair-000093", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (7,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#...##XX\n##.#.##XX\n.#.#.##XX\n.#.#.##XX\n.#.#A##XX\n.#.####XX\n.....##XX\n.###.##XX\n.....##XX", "question": "Are BOTH the cell one step east and the cell one step west blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "6dd8b6763f15e44d984391f8fc6be8ea2d43e42c27c7c746af2e7cefed4c008f"}95{"id": "stochastic-000094", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 7, column 1. Rows increase to the south and columns to the east.\nRow 4: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall\nRow 5: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open\nRow 6: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall\nRow 7: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is open, column 3 is open, column 4 is open\nRow 8: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall\nRow 9: column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze\nRow 10: column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze", "question": "The actuator executes the intended move with probability 0.90; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9666666666666667, 0.033333333333333326], "deterministic": false, "fingerprint": "0aaa6f04f2f194eac5ebfc5ca06b6a9f24260707476f7b5d902829b6642e60e9"}96{"id": "snake_food-000095", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,10), travelling south, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n.........oo...\n..........H...\n....*.........\n..............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 7, "answer": "south-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "9d31faed1042af5f87806bcc403f6cb8591abe4633d90e7cd945ecc18c195001"}97{"id": "blocked_pair-000096", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 4. Rows increase to the south and columns to the east.\nRow 4: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is open, column 5 is wall, column 6 is wall, column 7 is open\nRow 5: column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open\nRow 6: column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open\nRow 7: column 1 is open, column 2 is open, column 3 is open, column 4 is the agent, column 5 is open, column 6 is wall, column 7 is open\nRow 8: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 9: column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze\nRow 10: column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze", "question": "Are BOTH the cell one step south and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "489b69a6d04fab2017772aa93c0ead307973c20a2a7be5b03b95c79c4692d758"}98{"id": "snake_safe-000097", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,6), travelling west, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n.......o......\n......Ho......\n..............\n.*............\n..............\n..............\n..............", "question": "If the snake moves east on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "b2af21f80de45809e73b21f5858980eeaf4270d8f9441962fe67def68f1f2dd5"}99{"id": "only_way-000098", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,9); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXX\n#####\n..A##\n#####\n...##", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "2dbaf1a8f1466363d090bea508a49869f484b1970193004bfe9c8cc30012a849"}100{"id": "snake_food-000099", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,7), travelling north, length 3.\n............\n............\n............\n............\n............\n............\n.......H....\n......oo....\n............\n*...........\n............\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 7, "answer": "south-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "e74e3109e0e75102568763c1e82f81a86bd00327c944ce5543cdb8c3f26f4371"}101{"id": "move_two_step-000100", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (30,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXX#.#.#.\nXXX#.#.#.\nXXX#.....\nXXX#.#.#.\nXXX#A#.#.\nXXX#.#...\nXXX#.###.\nXXX#.....\nXXX######", "question": "The agent plans to move north, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "2a6cc5e537940186e132965e625412fb0cec1fa1a949946c7e0457acaa255764"}102{"id": "plan_survives-000101", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 5, column 0. Rows increase to the south and columns to the east.\nRow 2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open, column 3 is open\nRow 3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open\nRow 4: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is open, column 3 is wall\nRow 5: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is the agent, column 1 is open, column 2 is wall, column 3 is wall\nRow 6: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open\nRow 7: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is wall\nRow 8: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open, column 3 is open", "question": "The agent will attempt this plan in order: south, east, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "911504add6672310e6f404394b992a47abb8ac983d9dabd378ba2102ec9b229a"}103{"id": "plan_survives-000102", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 1. Rows increase to the south and columns to the east.\nRow 3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open\nRow 4: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 5: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 6: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 7: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 8: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall\nRow 9: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open\nRow 10: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 11: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall", "question": "The agent will attempt this plan in order: north, east, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "04185fb9174dc2fe4fc282acefe93a448d283afe4a7fcf20c4fd50654a47d76f"}104{"id": "open_count-000103", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 31, column 15. Rows increase to the south and columns to the east.\nRow 27: column 11 is open, column 12 is wall, column 13 is open, column 14 is open, column 15 is open, column 16 is wall, column 17 is open, column 18 is open, column 19 is open\nRow 28: column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open, column 18 is wall, column 19 is wall\nRow 29: column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open, column 18 is wall, column 19 is open\nRow 30: column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is open, column 18 is wall, column 19 is wall\nRow 31: column 11 is open, column 12 is wall, column 13 is open, column 14 is open, column 15 is the agent, column 16 is wall, column 17 is open, column 18 is open, column 19 is open\nRow 32: column 11 is open, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is open, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is open\nRow 33: column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open\nRow 34: column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall\nRow 35: column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 2, "answer": "2", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "60f30c280d80b7316334d009138119dde59ad9c4efc8bb6dab274edd2e836a09"}105{"id": "stochastic-000104", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 3, column 8. Rows increase to the south and columns to the east.\nRow 1: column 6 is open, column 7 is open, column 8 is open, column 9 is open, column 10 is open\nRow 2: column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall\nRow 3: column 6 is wall, column 7 is open, column 8 is the agent, column 9 is open, column 10 is open\nRow 4: column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall\nRow 5: column 6 is wall, column 7 is open, column 8 is open, column 9 is open, column 10 is open", "question": "The actuator executes the intended move with probability 0.85; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9, 0.10000000000000002], "deterministic": false, "fingerprint": "7c4f7114f3cd9e3af8caedbe2d336585ddf3966b6c72d184879a6667b418df9e"}106{"id": "blocked_pair-000105", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 11, column 0. Rows increase to the south and columns to the east.\nRow 8: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is wall\nRow 9: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is wall, column 3 is wall\nRow 10: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is wall, column 3 is wall\nRow 11: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is the agent, column 1 is open, column 2 is wall, column 3 is wall\nRow 12: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze\nRow 13: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze\nRow 14: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze", "question": "Are BOTH the cell one step north and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "3631dd4bac8c151160219ca48e66ecf536b731c745e44f42951d5c1fabae2044"}107{"id": "move_two_step-000106", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (8,9); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n......#\n#####..\n....#..\n###A#..\n#.#.#.#\n#.#####\n.......", "question": "The agent plans to move north, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "a471cb8be880b99d561fc37dc9ceb49ed45805579f3b87f3d14bb1647b7dee8f"}108{"id": "move_legal-000107", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (7,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#.#.\n##.#.\n.#A#.\n.####\n.....", "question": "The agent wants to move west. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "8d183580501dbc6e2a9ac16ccccf7899f1a36f5145dc3717c59ab8219345c584"}109{"id": "plan_survives-000108", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 21, column 18. Rows increase to the south and columns to the east.\nRow 17: column 14 is open, column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is open, column 21 is open, column 22 is open\nRow 18: column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is wall, column 22 is wall\nRow 19: column 14 is open, column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is open, column 21 is open, column 22 is open\nRow 20: column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is wall, column 22 is wall\nRow 21: column 14 is open, column 15 is open, column 16 is wall, column 17 is open, column 18 is the agent, column 19 is open, column 20 is open, column 21 is open, column 22 is open\nRow 22: column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is wall, column 22 is wall\nRow 23: column 14 is open, column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is open, column 21 is open, column 22 is open\nRow 24: column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is open, column 22 is wall\nRow 25: column 14 is open, column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is wall, column 21 is open, column 22 is wall", "question": "The agent will attempt this plan in order: east, east, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "deae89fdec031d06c1a7a67340f702bccb9bca3329a49789b3798c03af128d0d"}110{"id": "blocked_pair-000109", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 3, column 1. Rows increase to the south and columns to the east.\nRow 1: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open\nRow 2: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is open\nRow 3: column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is wall, column 3 is open\nRow 4: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open\nRow 5: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open", "question": "Are BOTH the cell one step south and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "a8bc687be4308264d0ec4f97db6183aa1ab17121804ccefbeef66b7eb0eb0ece"}111{"id": "only_way-000110", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[9,8],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"6,5\":\"open\",\"6,6\":\"wall\",\"6,7\":\"wall\",\"6,8\":\"wall\",\"6,9\":\"open\",\"6,10\":\"wall\",\"6,11\":\"wall\",\"7,5\":\"open\",\"7,6\":\"wall\",\"7,7\":\"open\",\"7,8\":\"open\",\"7,9\":\"open\",\"7,10\":\"wall\",\"7,11\":\"wall\",\"8,5\":\"open\",\"8,6\":\"wall\",\"8,7\":\"open\",\"8,8\":\"wall\",\"8,9\":\"wall\",\"8,10\":\"wall\",\"8,11\":\"wall\",\"9,5\":\"open\",\"9,6\":\"wall\",\"9,7\":\"open\",\"9,8\":\"agent\",\"9,9\":\"open\",\"9,10\":\"wall\",\"9,11\":\"wall\",\"10,5\":\"wall\",\"10,6\":\"wall\",\"10,7\":\"wall\",\"10,8\":\"wall\",\"10,9\":\"wall\",\"10,10\":\"wall\",\"10,11\":\"wall\",\"11,5\":\"wall\",\"11,6\":\"wall\",\"11,7\":\"wall\",\"11,8\":\"wall\",\"11,9\":\"wall\",\"11,10\":\"wall\",\"11,11\":\"wall\",\"12,5\":\"outside\",\"12,6\":\"outside\",\"12,7\":\"outside\",\"12,8\":\"outside\",\"12,9\":\"outside\",\"12,10\":\"outside\",\"12,11\":\"outside\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 5, "answer": "more than one of them", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "ccb75ccf3a1b66886e2f4eade116577a63e569791e13e134edf7d2904f09207c"}112{"id": "blocked_pair-000111", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[22,29],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"20,27\":\"open\",\"20,28\":\"wall\",\"20,29\":\"wall\",\"20,30\":\"wall\",\"20,31\":\"wall\",\"21,27\":\"open\",\"21,28\":\"wall\",\"21,29\":\"open\",\"21,30\":\"wall\",\"21,31\":\"open\",\"22,27\":\"wall\",\"22,28\":\"wall\",\"22,29\":\"agent\",\"22,30\":\"wall\",\"22,31\":\"open\",\"23,27\":\"open\",\"23,28\":\"wall\",\"23,29\":\"open\",\"23,30\":\"wall\",\"23,31\":\"open\",\"24,27\":\"open\",\"24,28\":\"wall\",\"24,29\":\"open\",\"24,30\":\"wall\",\"24,31\":\"wall\"}}", "question": "Are BOTH the cell one step west and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "e1df4b7852a2e77754e0b3df1043c36aa58e7ba9e7eed80d0b78e449c608e882"}113{"id": "move_two_step-000112", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[4,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"open\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"open\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"agent\",\"4,4\":\"wall\",\"4,5\":\"open\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"open\"}}", "question": "The agent plans to move south, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "ab7857eef7927763f247138d49d276d08147772652b04b10b81ad862d479c737"}114{"id": "snake_safe-000113", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,4), travelling south, length 3.\n........\n........\n........\n........\n...oo...\n....H...\n....*...\n........", "question": "If the snake moves east on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "527be497a538c0ce93ab09ae74b9aad65d2774784b6e1c4680578c56f7aef32a"}115{"id": "open_count-000114", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (6,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX##....##\nX#..##...\nX......#.\nX.##.#.#.\nX...A....\nX###...##\nX....#.##\nXXXXXXXXX\nXXXXXXXXX", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 4, "answer": "4", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "64ee28ad227c0604255c85c12b155f898c54b0e4b80f78acd07ceb6d227642ac"}116{"id": "move_legal-000115", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (11,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n...##\n.####\n.#A##\n.#.##\n...##", "question": "If the agent attempts one cell west (col-1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "bcc12ce3aeab5948ce6f7ac5af111acc4cd540256b6c5346913592525d82ed71"}117{"id": "open_count-000116", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[0,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-3,-2\":\"outside\",\"-3,-1\":\"outside\",\"-3,0\":\"outside\",\"-3,1\":\"outside\",\"-3,2\":\"outside\",\"-3,3\":\"outside\",\"-3,4\":\"outside\",\"-2,-2\":\"outside\",\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"open\",\"0,1\":\"agent\",\"0,2\":\"open\",\"0,3\":\"wall\",\"0,4\":\"open\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"wall\",\"1,4\":\"open\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"open\",\"2,1\":\"open\",\"2,2\":\"open\",\"2,3\":\"open\",\"2,4\":\"open\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"open\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"open\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 3, "answer": "3", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "72a1f761ba0f3f62bbf98ed5dc3dcbf0ab40cfceabe6c4ce4a0ecd8f9ffa7af0"}118{"id": "blocked_pair-000117", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[19,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"17,7\":\"open\",\"17,8\":\"open\",\"17,9\":\"open\",\"17,10\":\"open\",\"17,11\":\"open\",\"18,7\":\"open\",\"18,8\":\"wall\",\"18,9\":\"wall\",\"18,10\":\"wall\",\"18,11\":\"open\",\"19,7\":\"open\",\"19,8\":\"wall\",\"19,9\":\"agent\",\"19,10\":\"open\",\"19,11\":\"open\",\"20,7\":\"wall\",\"20,8\":\"wall\",\"20,9\":\"wall\",\"20,10\":\"wall\",\"20,11\":\"wall\",\"21,7\":\"outside\",\"21,8\":\"outside\",\"21,9\":\"outside\",\"21,10\":\"outside\",\"21,11\":\"outside\"}}", "question": "Are BOTH the cell one step south and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "6840a44bef45a2f8625b5dbc6cd5a72cd575ee0d4de7d7117b2eeb20a844351d"}119{"id": "plan_survives-000118", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[33,33],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"30,30\":\"wall\",\"30,31\":\"open\",\"30,32\":\"wall\",\"30,33\":\"open\",\"30,34\":\"wall\",\"30,35\":\"wall\",\"30,36\":\"outside\",\"31,30\":\"open\",\"31,31\":\"open\",\"31,32\":\"open\",\"31,33\":\"open\",\"31,34\":\"wall\",\"31,35\":\"wall\",\"31,36\":\"outside\",\"32,30\":\"wall\",\"32,31\":\"wall\",\"32,32\":\"wall\",\"32,33\":\"wall\",\"32,34\":\"wall\",\"32,35\":\"wall\",\"32,36\":\"outside\",\"33,30\":\"open\",\"33,31\":\"open\",\"33,32\":\"open\",\"33,33\":\"agent\",\"33,34\":\"wall\",\"33,35\":\"wall\",\"33,36\":\"outside\",\"34,30\":\"wall\",\"34,31\":\"wall\",\"34,32\":\"wall\",\"34,33\":\"wall\",\"34,34\":\"wall\",\"34,35\":\"wall\",\"34,36\":\"outside\",\"35,30\":\"wall\",\"35,31\":\"wall\",\"35,32\":\"wall\",\"35,33\":\"wall\",\"35,34\":\"wall\",\"35,35\":\"wall\",\"35,36\":\"outside\",\"36,30\":\"outside\",\"36,31\":\"outside\",\"36,32\":\"outside\",\"36,33\":\"outside\",\"36,34\":\"outside\",\"36,35\":\"outside\",\"36,36\":\"outside\"}}", "question": "The agent will attempt this plan in order: south, west, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a7194369e4184e468b2913e98f61e86c91655477ec7d754f1a526025fdf8c697"}120{"id": "plan_survives-000119", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,6],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"-1,6\":\"outside\",\"-1,7\":\"outside\",\"-1,8\":\"outside\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"0,8\":\"wall\",\"1,4\":\"wall\",\"1,5\":\"open\",\"1,6\":\"agent\",\"1,7\":\"open\",\"1,8\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"2,6\":\"wall\",\"2,7\":\"open\",\"2,8\":\"wall\",\"3,4\":\"open\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"3,8\":\"wall\"}}", "question": "The agent will attempt this plan in order: west, west, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "40484efbff9113cd0cd8f9542b4059323d69190bd344d55a9726ec3c8ca5e183"}121{"id": "snake_food-000120", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,6), travelling south, length 3.\n............\n............\n............\n............\n............\n............\n.....oo.....\n......H.....\n............\n............\n............\n......*.....", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 1, "answer": "south", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "c0e013736847b166a57015d050ba6f112486044e6433b10e507964d3b9b1cb84"}122{"id": "only_way-000121", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 0. Rows increase to the south and columns to the east.\nRow -3: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze\nRow -2: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze\nRow -1: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze\nRow 0: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is open\nRow 1: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is the agent, column 1 is open, column 2 is open, column 3 is open, column 4 is open\nRow 2: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is open, column 3 is open, column 4 is open\nRow 3: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is open\nRow 4: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open, column 3 is wall, column 4 is wall\nRow 5: column -4 is outside the maze, column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "407c30353456cc4f0ba53c4168f354d7170c270b54c0c49c598578604b91c631"}123{"id": "reachable_within-000122", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,1), goal at (7,5).\n############\n#.......#.##\n#A#####.#.##\n#.#...#.#.##\n#.#.#####.##\n#.#.......##\n#.#.###.#.##\n#.#..G#.#.##\n#.#####.#.##\n#.......#.##\n############\n############", "question": "Can the agent reach the goal in at most 25 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "cf88f0a2a8368e56e11afa3eef543114b74fa72d03f8455fd42d15983ec31e43"}124{"id": "reachable_within-000123", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,3), goal at (5,9).\n################\n#.............##\n#.###########.##\n#...#.......#.##\n###.#######.#.##\n#.#......G#.#.##\n#.#########.#.##\n#...........#.##\n#.#########.#.##\n#.#A........#.##\n#.###########.##\n#.............##\n#.##############\n#.............##\n################\n################", "question": "Can the agent reach the goal in at most 69 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "1e212f00f2cf56f18a9a13205a0fada91da57a1214db7b6bf6b31b8eee1155fc"}125{"id": "goal_bearing-000124", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (8,19), goal at (17,3).\n#####################\n#.........#.#.......#\n#.#####.#.#.#.#.###.#\n#.#...#.#.#.#.#.#.#.#\n#.#.#.#.#.#.#.#.#.#.#\n#.#.#.#.#.#.#.#.#...#\n#.#.###.#.#.#.#.#.###\n#.#...#.#.#.#.#.#.#.#\n#.#.#.#.#.#.#.#.#.#A#\n#.#.#.#.#.#.#.#.#.#.#\n#.#.#.#.#.#.###.#.#.#\n#.#.#.#.#.#.#...#...#\n#.#.#.#.#.#.#.#######\n#.#.#.#.#.#.#.#.....#\n#.#.#.#.#.#.#.#.###.#\n#.#.#.#.#.#.#.#.#.#.#\n#.#.#.#.#.#.#.#.#.#.#\n#.#G#.#.#.#.#.....#.#\n#.###.#.###.#######.#\n#.....#.............#\n#####################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 7, "answer": "south-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "dd04b178cc4f44b72dfdceb8e85e622f74893cda6710062e9b06cb8e1e7cf209"}126{"id": "snake_food-000125", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,8), travelling south, length 3.\n...*..........\n..............\n..............\n..............\n.......oo.....\n........H.....\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "c51f40569a4b06ba92b20034ba945112e918a3b68cac2e98043781331fd5cf0a"}127{"id": "move_legal-000126", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 0, column 5. Rows increase to the south and columns to the east.\nRow -3: column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow -2: column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow -1: column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow 0: column 2 is open, column 3 is open, column 4 is open, column 5 is the agent, column 6 is wall, column 7 is open, column 8 is open\nRow 1: column 2 is open, column 3 is wall, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is open\nRow 2: column 2 is open, column 3 is wall, column 4 is open, column 5 is wall, column 6 is open, column 7 is open, column 8 is open\nRow 3: column 2 is open, column 3 is open, column 4 is open, column 5 is wall, column 6 is open, column 7 is open, column 8 is open", "question": "If the agent attempts one cell west (col-1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "22318689cf3a13eea65f909254398e327de8cc40351eb5f8ffc90cdd1d6b43eb"}128{"id": "stochastic-000127", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (8,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#.#.#.#.\n.#.#.#.#.\n.#.#.#.#.\n.#.#.#.#.\n##.#A#.#.\n...#.#.#.\n.#.#.#.#.\n.#.#.#.#.\n.#.###.#.", "question": "The actuator executes the intended move with probability 0.90; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9333333333333333, 0.06666666666666665], "deterministic": false, "fingerprint": "27f8b5f6e3860378c8b77c60be93e3c7e0e67f0518191c962e5087601723b110"}129{"id": "reachable_within-000128", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (14,19), goal at (21,7).\n########################\n#.....#.......#...#...##\n#.#####.#####.#.#.###.##\n#.....#...#.#...#.....##\n#.###.###.#.#########.##\n#.#...#...#.....#.....##\n#.#.###.###.#.###.######\n#.#...#.#...#.......#.##\n#.###.#.###########.#.##\n#...#.#.........#.#...##\n###.#.#########.#.###.##\n#...#.........#.#.....##\n#.#####.#####.#.###.####\n#...#...#.....#...#.#.##\n###.#.###########.#A#.##\n#...#...#.........#...##\n#.#####.#.###########.##\n#.#.#...#.#.........#.##\n#.#.#.#####.#######.#.##\n#.#.#.....#.......#.#.##\n#.#.#####.#######.#.#.##\n#......G#.........#...##\n########################\n########################", "question": "Can the agent reach the goal in at most 111 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "f50c83cc692349d12ffc23ac01699416e68376d35630866a0f5c1ca853bd897d"}130{"id": "only_way-000129", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 19, column 13. Rows increase to the south and columns to the east.\nRow 16: column 10 is wall, column 11 is open, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is open, column 16 is wall\nRow 17: column 10 is wall, column 11 is open, column 12 is open, column 13 is open, column 14 is open, column 15 is open, column 16 is open\nRow 18: column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall\nRow 19: column 10 is open, column 11 is open, column 12 is open, column 13 is the agent, column 14 is wall, column 15 is open, column 16 is open\nRow 20: column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall\nRow 21: column 10 is wall, column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is open, column 16 is open\nRow 22: column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "9a0bddd6bd232b96aea4548cdf7cb19b0b03385c2ef132bbceb4bb0763d532cb"}131{"id": "open_count-000130", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 11, column 11. Rows increase to the south and columns to the east.\nRow 7: column 7 is open, column 8 is open, column 9 is open, column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is wall\nRow 8: column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall\nRow 9: column 7 is open, column 8 is open, column 9 is open, column 10 is open, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall\nRow 10: column 7 is open, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall\nRow 11: column 7 is open, column 8 is wall, column 9 is open, column 10 is open, column 11 is the agent, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall\nRow 12: column 7 is open, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall\nRow 13: column 7 is open, column 8 is open, column 9 is open, column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is wall\nRow 14: column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall\nRow 15: column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 2, "answer": "2", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "718a1032df29e0a0639a25eccbf01236bd6fac04a3a8b3e599cf4f2ac0fcf84f"}132{"id": "reachable_within-000131", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,5), goal at (7,5).\n############\n#...#.....##\n#.#.#.###.##\n#.#.#.#.#.##\n#.###.#.#.##\n#....A..#.##\n#.#####.#.##\n#.#.#G#.#.##\n#.#.#.###.##\n#...#.....##\n############\n############", "question": "Can the agent reach the goal in at most 23 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "7a7afbc628490e5ee40509db40a102db7c23606c6ecfa2e7eb1cf21271686c4f"}133{"id": "reachable_within-000132", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,7), goal at (1,9).\n############\n#........G##\n#.##########\n#.#....A..##\n#.#.########\n#.#.....#.##\n#.#####.#.##\n#.....#...##\n#.###.###.##\n#...#.....##\n############\n############", "question": "Can the agent reach the goal in at most 44 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "766fe027907f8cc91ce81f44e47b2f1cc19c00c86839bfff9b7d413f2ed2c44e"}134{"id": "blocked_pair-000133", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (11,14); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#...XX\n.##..XX\n..#..XX\n.#.A.XX\n###..XX\n.#...XX\n..###XX", "question": "Are BOTH the cell one step north and the cell one step west blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "c54d07743b915275fe99b14921735a3b0523820fc6bf98093ed1e77e09e815a0"}135{"id": "goal_bearing-000134", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,9), goal at (7,21).\n############################\n#...............#.....#...##\n#.#####.#######.###.#.###.##\n#.#...#.#...#.......#...#.##\n#.#.#.###.#.###########.#.##\n#.#.#.#...#.......#...#...##\n#.#.#.#.#########.#.#.###.##\n#...#...#A#.....#...#G#...##\n#########.#.#.#########.####\n#...#.....#.#.........#.#.##\n#.#.#.#####.#########.#.#.##\n#.#.#.....#.#...#...#.#.#.##\n#.#.#####.#.#.#.###.#.#.#.##\n#.#.....#.#.#.#.....#.#...##\n#.#####.#.#.#.#####.#.###.##\n#.....#.#.#...#.#...#...#.##\n#.###.#.#.#####.#.#####.#.##\n#.#.#.#...#.....#...#.#...##\n#.#.#.#####.###.###.#.######\n#...#...#.....#...#...#...##\n#####.#.#.#######.###.#.#.##\n#...#.#.#.#.#.......#...#.##\n#.#.###.#.#.#.###########.##\n#.#.....#...#.....#.....#.##\n#.#########.#####.#.###.#.##\n#...............#.....#...##\n############################\n############################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "43f6ed1077c437ceea3a4c4f1f45809d5f7c6f5b13bf961f770c11b6ffceab2e"}136{"id": "cell_kind-000135", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (10,5); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#.#.###.\n...#.#...\n####.#.##\n.....#...\n.###A####\n.#.....#.\n.#######.\n.........\n#########", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "250e1faf0a6e4862f4ca8765175adb9bdce5185d5d2002e4e4bfdd195fed9b51"}137{"id": "first_move-000136", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,6), goal at (11,11).\n############\n############\n############\n############\n#..##..#####\n....##..#.##\n...#...##..#\n.....#.#..##\n..#.........\n.##...A###..\n.........#..\n...#...####G", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "969705f62c6e4ba87cacf9bb38df7dbc48f92e2604a4a8fc9b85d0dc2bd84069"}138{"id": "first_move-000137", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,7), goal at (5,3).\n############\n#...#.....##\n#.#.#.###.##\n#.#.#.#...##\n#.#.#.###.##\n#.#G#..A#.##\n#.#######.##\n#.#.....#.##\n#.#.#.###.##\n#...#.....##\n############\n############", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "306f8bf48b587e6f654404ddab06ba8548765d856e5a3bdf7889ab871e863da6"}139{"id": "reachable_within-000138", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,13), goal at (9,3).\n################\n#.......#.....##\n#.#####.#.###.##\n#.#...#.#.#.#.##\n#.#.###.#.#.#.##\n#.#.....#.#.#.##\n#.#######.#.#.##\n#...........#.##\n#.###########.##\n#.#G#........A##\n#.#.############\n#.#...........##\n#.###########.##\n#.............##\n################\n################", "question": "Can the agent reach the goal in at most 57 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "2ea76cf7f55bc2a44685eb37e5fb067210d7412e856c6b8eca873fea7faa3f18"}140{"id": "reachable_within-000139", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,3), goal at (3,11).\n################\n#.............##\n#.#######.######\n#.#A..#...#G..##\n#.#.#.#.#####.##\n#.#.#...#.....##\n#.#######.######\n#...#...#...#.##\n#.#.#.#.###.#.##\n#.#...#.#...#.##\n#.#####.#.###.##\n#.....#...#...##\n#####.#######.##\n#.............##\n################\n################", "question": "Can the agent reach the goal in at most 61 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "445c85c6644b0bf0d056225ca1170242b2aef5a38009f61a28cb90e596c728db"}141{"id": "distance_band-000140", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (4,7), goal at (1,13).\n################\n#...........#G##\n#.#.#.#####.#.##\n#.#.........#.##\n#.###..A#.#.#.##\n#...#...#.....##\n#.#.#...########\n#...#.#.......##\n###...#.#.###.##\n#.........#...##\n#.#.#######.####\n#...#.....#.#.##\n#.###.###.#.#.##\n#.............##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "333eee3d552f19ba6bfe95cb45c212a8dc9fb12bb01e4dafd177fcd30118bab5"}142{"id": "distance_band-000141", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 40x40. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (35,5), goal at (24,0).\n#......###...#...#######################\n#..##......#........##..################\n....###.#.##.#..#....##..###############\n.##..##.#..#..#.##.#.....#...##.########\n.#..####...##.......###.##.#.#.....#####\n.######..#.....##.####......#.#.########\n########...##.#......##.###......#######\n########.#....#..########..#.#.#########\n#######..#.###....#.###...##..##.#######\n########..##........##..##..#....#######\n#########.....#........##..#.##...######\n#########.#...#.#..#..#...#......#..####\n########...........#.#......#...#..#####\n#####.#.....#....#................######\n..#...#..#.##.##...######.###.....#.####\n....#......######......##..##..#.#...#..\n#..#....########..##..##.#..........#...\n####..#...#######.##.###..#.#.....##...#\n#####...#...##.###....##........##.#..##\n######.......#....#...##...#.#.#.......#\n######.#...#.#.#.###....##..#.#.....#...\n#########...####.....#..............##..\n###........###......###..##.......#....#\n##....###..##..#.##.####..#.#.##......#.\nG##....#..#..#.####....#..####.....#....\n.#...#.......#.#.........#####..###.##..\n...###.##.....##.###...#.######......#..\n#####.#..##.#...###..#......##.##..#.#..\n#.###..#...#.##...#.#......#......#..###\n...###.#..#...#..#........#....#...#....\n....##....###.#.###.#..#....#..#.....##.\n###....#...##.##.#..##...#..........#.#.\n#.#....#....#.....#...#....##...##.#....\n....#####...##....##..###...##...#.#.#.#\n..#.#....#....###......######......###.#\n###..A#.......##..##.#.###........#..#..\n#..##..#.#...#....#....#....##...###....\n#......#............#..#..#....#....#..#\n###.....#..###.#.##.....#..#....##....##\n###...#...######..##.#.###.##.......#.##", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 4, "answer": "40 moves or more", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "d45751d5303e8795d85230c734d54be52910be94e1c6341932ab040d56f54c2d"}143{"id": "blocked_pair-000142", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[19,13],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"17,11\":\"open\",\"17,12\":\"wall\",\"17,13\":\"open\",\"17,14\":\"wall\",\"17,15\":\"open\",\"18,11\":\"open\",\"18,12\":\"wall\",\"18,13\":\"open\",\"18,14\":\"wall\",\"18,15\":\"open\",\"19,11\":\"open\",\"19,12\":\"wall\",\"19,13\":\"agent\",\"19,14\":\"wall\",\"19,15\":\"open\",\"20,11\":\"wall\",\"20,12\":\"wall\",\"20,13\":\"wall\",\"20,14\":\"wall\",\"20,15\":\"wall\",\"21,11\":\"outside\",\"21,12\":\"outside\",\"21,13\":\"outside\",\"21,14\":\"outside\",\"21,15\":\"outside\"}}", "question": "Are BOTH the cell one step east and the cell one step south blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "836a6bfa667eeeaea689737629d586cef8ea4a66d1d218e771d45adf9c4496d9"}144{"id": "blocked_pair-000143", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[8,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"wall\",\"5,9\":\"open\",\"5,10\":\"wall\",\"5,11\":\"wall\",\"5,12\":\"outside\",\"6,6\":\"wall\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"open\",\"6,10\":\"wall\",\"6,11\":\"wall\",\"6,12\":\"outside\",\"7,6\":\"wall\",\"7,7\":\"open\",\"7,8\":\"wall\",\"7,9\":\"open\",\"7,10\":\"wall\",\"7,11\":\"wall\",\"7,12\":\"outside\",\"8,6\":\"wall\",\"8,7\":\"open\",\"8,8\":\"wall\",\"8,9\":\"agent\",\"8,10\":\"wall\",\"8,11\":\"wall\",\"8,12\":\"outside\",\"9,6\":\"open\",\"9,7\":\"open\",\"9,8\":\"wall\",\"9,9\":\"open\",\"9,10\":\"wall\",\"9,11\":\"wall\",\"9,12\":\"outside\",\"10,6\":\"wall\",\"10,7\":\"wall\",\"10,8\":\"wall\",\"10,9\":\"wall\",\"10,10\":\"wall\",\"10,11\":\"wall\",\"10,12\":\"outside\",\"11,6\":\"wall\",\"11,7\":\"wall\",\"11,8\":\"wall\",\"11,9\":\"wall\",\"11,10\":\"wall\",\"11,11\":\"wall\",\"11,12\":\"outside\"}}", "question": "Are BOTH the cell one step west and the cell one step south blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "8d40780ae94d07004876c42e0d13f4c2779d612313a282dc26ccf0123ea84088"}145{"id": "open_count-000144", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (27,17); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n###.#.#\n#.#...#\n#.#####\n#..A..#\n#.#####\n..#....\n#.#.###", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 2, "answer": "2", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "89f0f613179d396e1d558f7998b0c06b719580d176d9d2e9816c2ef56c7dbc12"}146{"id": "open_count-000145", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 8, column 2. Rows increase to the south and columns to the east.\nRow 5: column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open\nRow 6: column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is wall, column 3 is wall, column 4 is open, column 5 is wall\nRow 7: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is wall, column 4 is open, column 5 is open\nRow 8: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is the agent, column 3 is wall, column 4 is open, column 5 is wall\nRow 9: column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 10: column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 11: column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 2, "answer": "2", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "9017919fa777cd9da919ec41d3f3053c42ff37233e00b8d5e4fe064edb9d0d3b"}147{"id": "cell_kind-000146", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[8,6],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"6,4\":\"wall\",\"6,5\":\"open\",\"6,6\":\"open\",\"6,7\":\"open\",\"6,8\":\"wall\",\"7,4\":\"wall\",\"7,5\":\"wall\",\"7,6\":\"open\",\"7,7\":\"wall\",\"7,8\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"open\",\"8,6\":\"agent\",\"8,7\":\"open\",\"8,8\":\"wall\",\"9,4\":\"outside\",\"9,5\":\"outside\",\"9,6\":\"outside\",\"9,7\":\"outside\",\"9,8\":\"outside\",\"10,4\":\"outside\",\"10,5\":\"outside\",\"10,6\":\"outside\",\"10,7\":\"outside\",\"10,8\":\"outside\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "06fa16f9e145f66a0eeba320506ea511bb0ab9d6c0376b253cfef9144b4aa325"}148{"id": "cell_kind-000147", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\n#########\n.........\n##.###.##\n.#.#A#.#.\n.#.#.###.\n...#.....\n##.######\n.......#.", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "671b341d5995b564d26a88551002465f643ddee575b701ca97fb465ea4755cf5"}149{"id": "only_way-000148", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[0,10],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-4,6\":\"outside\",\"-4,7\":\"outside\",\"-4,8\":\"outside\",\"-4,9\":\"outside\",\"-4,10\":\"outside\",\"-4,11\":\"outside\",\"-4,12\":\"outside\",\"-4,13\":\"outside\",\"-4,14\":\"outside\",\"-3,6\":\"outside\",\"-3,7\":\"outside\",\"-3,8\":\"outside\",\"-3,9\":\"outside\",\"-3,10\":\"outside\",\"-3,11\":\"outside\",\"-3,12\":\"outside\",\"-3,13\":\"outside\",\"-3,14\":\"outside\",\"-2,6\":\"outside\",\"-2,7\":\"outside\",\"-2,8\":\"outside\",\"-2,9\":\"outside\",\"-2,10\":\"outside\",\"-2,11\":\"outside\",\"-2,12\":\"outside\",\"-2,13\":\"outside\",\"-2,14\":\"outside\",\"-1,6\":\"outside\",\"-1,7\":\"outside\",\"-1,8\":\"outside\",\"-1,9\":\"outside\",\"-1,10\":\"outside\",\"-1,11\":\"outside\",\"-1,12\":\"outside\",\"-1,13\":\"outside\",\"-1,14\":\"outside\",\"0,6\":\"open\",\"0,7\":\"open\",\"0,8\":\"wall\",\"0,9\":\"open\",\"0,10\":\"agent\",\"0,11\":\"wall\",\"0,12\":\"wall\",\"0,13\":\"wall\",\"0,14\":\"wall\",\"1,6\":\"wall\",\"1,7\":\"open\",\"1,8\":\"wall\",\"1,9\":\"open\",\"1,10\":\"wall\",\"1,11\":\"wall\",\"1,12\":\"wall\",\"1,13\":\"wall\",\"1,14\":\"wall\",\"2,6\":\"open\",\"2,7\":\"open\",\"2,8\":\"open\",\"2,9\":\"open\",\"2,10\":\"open\",\"2,11\":\"open\",\"2,12\":\"wall\",\"2,13\":\"wall\",\"2,14\":\"wall\",\"3,6\":\"open\",\"3,7\":\"open\",\"3,8\":\"open\",\"3,9\":\"wall\",\"3,10\":\"wall\",\"3,11\":\"wall\",\"3,12\":\"wall\",\"3,13\":\"open\",\"3,14\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"open\",\"4,8\":\"open\",\"4,9\":\"wall\",\"4,10\":\"open\",\"4,11\":\"open\",\"4,12\":\"open\",\"4,13\":\"open\",\"4,14\":\"open\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "8388be71531479cea85aa90718002d8f8086ea59cdab92f893513c6ac22d3dd7"}150{"id": "onward-000149", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[0,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"open\",\"0,1\":\"agent\",\"0,2\":\"open\",\"0,3\":\"open\",\"1,-1\":\"outside\",\"1,0\":\"open\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"open\",\"2,3\":\"open\"}}", "question": "Suppose the agent moves one cell east. From that new cell, would a further step north be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "db55b4cc420c5a9168be9676ab1742bdd6c00065be400e05b83c5dc3ef7850b2"}151{"id": "first_move-000150", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (11,3), goal at (13,13).\n################\n#.....#.....#.##\n#.#.#.#.#.#.#.##\n#...#.#.....#.##\n#.#.#.#.###.#.##\n#.#.#.......#.##\n#.#.#####.###.##\n#.#...#.#.....##\n#####.#...#.#.##\n#...#.#.#...#.##\n#.#.#.#.#####.##\n#.#A..#.......##\n#.#####.#.######\n#.#..........G##\n################\n################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "fe54ef6ded93ab41638cf1b23f2380f37e42d29e06ca0f006bdcf9807fba99e2"}152{"id": "stochastic-000151", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[16,25],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"13,22\":\"open\",\"13,23\":\"open\",\"13,24\":\"open\",\"13,25\":\"open\",\"13,26\":\"wall\",\"13,27\":\"wall\",\"13,28\":\"outside\",\"14,22\":\"wall\",\"14,23\":\"wall\",\"14,24\":\"wall\",\"14,25\":\"wall\",\"14,26\":\"wall\",\"14,27\":\"wall\",\"14,28\":\"outside\",\"15,22\":\"wall\",\"15,23\":\"open\",\"15,24\":\"open\",\"15,25\":\"open\",\"15,26\":\"wall\",\"15,27\":\"wall\",\"15,28\":\"outside\",\"16,22\":\"wall\",\"16,23\":\"wall\",\"16,24\":\"wall\",\"16,25\":\"agent\",\"16,26\":\"wall\",\"16,27\":\"wall\",\"16,28\":\"outside\",\"17,22\":\"wall\",\"17,23\":\"open\",\"17,24\":\"wall\",\"17,25\":\"open\",\"17,26\":\"wall\",\"17,27\":\"wall\",\"17,28\":\"outside\",\"18,22\":\"wall\",\"18,23\":\"open\",\"18,24\":\"wall\",\"18,25\":\"open\",\"18,26\":\"wall\",\"18,27\":\"wall\",\"18,28\":\"outside\",\"19,22\":\"wall\",\"19,23\":\"open\",\"19,24\":\"wall\",\"19,25\":\"open\",\"19,26\":\"wall\",\"19,27\":\"wall\",\"19,28\":\"outside\"}}", "question": "The actuator executes the intended move with probability 0.70; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.7999999999999999, 0.20000000000000004], "deterministic": false, "fingerprint": "a0f4d939459d43762a54c17a6be69e890c8bea13617b69782fc688a685651db1"}153{"id": "distance_band-000152", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (11,7), goal at (1,9).\n################\n#.......#G....##\n###.###.#####.##\n#...#.#.....#.##\n#####.#####.#.##\n#...#.......#.##\n#.###.#######.##\n#...#.#.......##\n###.#.#####.####\n#...#.#...#...##\n#.###.#.#.###.##\n#...#.#A#.#...##\n#.#.#.#.#.#.#.##\n#.#.....#...#.##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 3, "answer": "20 to 39 moves", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "6d3bd7cf66bbc5c123d292514238ca2234b75de580aa27a0b53bf2219e5aa2bf"}154{"id": "distance_band-000153", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (8,15), goal at (12,0).\n################\n################\n################\n##########...###\n######.#..#.####\n.#.###....#.##.#\n....###.....##..\n#....#...#....#.\n..#......#.....A\n###....#.###.###\n.....#..###.....\n..#...#.........\nG###.###..##..#.\n#######.....#.##\n######..##.#####\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 3, "answer": "20 to 39 moves", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "0a7d389bf63875335b2f399b081640e42727bcb5c64e1869f854e63f806c1733"}155{"id": "blocked_pair-000154", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#######\n#......\n#.#####\n#.#A...\n#.#####\n#.#....\n#.#####", "question": "Are BOTH the cell one step south and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "edb3c23e931104ffb0213928fa0a340eddcf0519b7f8c3b2d58c1ba0535ebbaf"}156{"id": "stochastic-000155", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (3,5); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\n#########\n.........\n.#######.\n....A..#.\n########.\n.......#.\n.#####.#.\n.#.....#.", "question": "The actuator executes the intended move with probability 0.75; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.16666666666666663, 0.8333333333333334], "deterministic": false, "fingerprint": "7dddb2ef782d4a48a048be60d4e2836a714fc76550f366b310435c8742490e77"}157{"id": "stochastic-000156", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (7,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#######\n#......\n#.#####\n#.#A...\n#.#####\n#......\n#######", "question": "The actuator executes the intended move with probability 0.90; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9666666666666667, 0.033333333333333326], "deterministic": false, "fingerprint": "ac5a0ac5212b670fb52a3bf92c9ffc78fa584ccaf4bc3c859793f12f221c6d0d"}158{"id": "snake_food-000157", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,8), travelling north, length 3.\n............\n............\n............\n............\n............\n............\n........H...\n*.......o...\n........o...\n............\n............\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 7, "answer": "south-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "e09959dc5153fcd1d65e4142cbba52e8410f4d506038512b4ee7cf81162e37d0"}159{"id": "stochastic-000158", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 2, column 19. Rows increase to the south and columns to the east.\nRow -2: column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze, column 20 is outside the maze, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow -1: column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze, column 20 is outside the maze, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 0: column 15 is open, column 16 is open, column 17 is wall, column 18 is open, column 19 is open, column 20 is open, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 1: column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is open, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 2: column 15 is open, column 16 is open, column 17 is open, column 18 is wall, column 19 is the agent, column 20 is open, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 3: column 15 is open, column 16 is wall, column 17 is wall, column 18 is open, column 19 is wall, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 4: column 15 is wall, column 16 is open, column 17 is wall, column 18 is open, column 19 is open, column 20 is open, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 5: column 15 is wall, column 16 is open, column 17 is wall, column 18 is wall, column 19 is open, column 20 is open, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 6: column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is open, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze", "question": "The actuator executes the intended move with probability 0.85; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9, 0.10000000000000002], "deterministic": false, "fingerprint": "416698892967bd6931c2d4500cd40446eaea75542cf504e8abc34e38aba7d30e"}160{"id": "stochastic-000159", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[11,13],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"7,9\":\"open\",\"7,10\":\"open\",\"7,11\":\"open\",\"7,12\":\"wall\",\"7,13\":\"open\",\"7,14\":\"wall\",\"7,15\":\"wall\",\"7,16\":\"outside\",\"7,17\":\"outside\",\"8,9\":\"open\",\"8,10\":\"wall\",\"8,11\":\"wall\",\"8,12\":\"wall\",\"8,13\":\"open\",\"8,14\":\"wall\",\"8,15\":\"wall\",\"8,16\":\"outside\",\"8,17\":\"outside\",\"9,9\":\"open\",\"9,10\":\"wall\",\"9,11\":\"open\",\"9,12\":\"open\",\"9,13\":\"open\",\"9,14\":\"wall\",\"9,15\":\"wall\",\"9,16\":\"outside\",\"9,17\":\"outside\",\"10,9\":\"wall\",\"10,10\":\"wall\",\"10,11\":\"open\",\"10,12\":\"wall\",\"10,13\":\"wall\",\"10,14\":\"wall\",\"10,15\":\"wall\",\"10,16\":\"outside\",\"10,17\":\"outside\",\"11,9\":\"open\",\"11,10\":\"open\",\"11,11\":\"open\",\"11,12\":\"wall\",\"11,13\":\"agent\",\"11,14\":\"wall\",\"11,15\":\"wall\",\"11,16\":\"outside\",\"11,17\":\"outside\",\"12,9\":\"open\",\"12,10\":\"wall\",\"12,11\":\"wall\",\"12,12\":\"wall\",\"12,13\":\"open\",\"12,14\":\"wall\",\"12,15\":\"wall\",\"12,16\":\"outside\",\"12,17\":\"outside\",\"13,9\":\"open\",\"13,10\":\"open\",\"13,11\":\"open\",\"13,12\":\"open\",\"13,13\":\"open\",\"13,14\":\"wall\",\"13,15\":\"wall\",\"13,16\":\"outside\",\"13,17\":\"outside\",\"14,9\":\"wall\",\"14,10\":\"wall\",\"14,11\":\"wall\",\"14,12\":\"wall\",\"14,13\":\"wall\",\"14,14\":\"wall\",\"14,15\":\"wall\",\"14,16\":\"outside\",\"14,17\":\"outside\",\"15,9\":\"wall\",\"15,10\":\"wall\",\"15,11\":\"wall\",\"15,12\":\"wall\",\"15,13\":\"wall\",\"15,14\":\"wall\",\"15,15\":\"wall\",\"15,16\":\"outside\",\"15,17\":\"outside\"}}", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.4, 0.6], "deterministic": false, "fingerprint": "367d0056cd458c4c26d4f6ca69794e50b3510f940fc72951e6b23f6d7f5461eb"}161{"id": "plan_survives-000160", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 4, column 3. Rows increase to the south and columns to the east.\nRow 1: column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open\nRow 2: column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall\nRow 3: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall\nRow 4: column 0 is wall, column 1 is open, column 2 is wall, column 3 is the agent, column 4 is wall, column 5 is open, column 6 is wall\nRow 5: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall\nRow 6: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall\nRow 7: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall", "question": "The agent will attempt this plan in order: south, south, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "062337be5609efa7e5200b8c743d3a65f44cd90ea53442826c1c6cf2827acfbe"}162{"id": "cell_kind-000161", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[5,34],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"2,31\":\"open\",\"2,32\":\"open\",\"2,33\":\"wall\",\"2,34\":\"open\",\"2,35\":\"open\",\"2,36\":\"outside\",\"2,37\":\"outside\",\"3,31\":\"open\",\"3,32\":\"open\",\"3,33\":\"open\",\"3,34\":\"open\",\"3,35\":\"wall\",\"3,36\":\"outside\",\"3,37\":\"outside\",\"4,31\":\"open\",\"4,32\":\"open\",\"4,33\":\"wall\",\"4,34\":\"open\",\"4,35\":\"wall\",\"4,36\":\"outside\",\"4,37\":\"outside\",\"5,31\":\"wall\",\"5,32\":\"wall\",\"5,33\":\"open\",\"5,34\":\"agent\",\"5,35\":\"open\",\"5,36\":\"outside\",\"5,37\":\"outside\",\"6,31\":\"open\",\"6,32\":\"open\",\"6,33\":\"open\",\"6,34\":\"open\",\"6,35\":\"wall\",\"6,36\":\"outside\",\"6,37\":\"outside\",\"7,31\":\"wall\",\"7,32\":\"open\",\"7,33\":\"open\",\"7,34\":\"open\",\"7,35\":\"open\",\"7,36\":\"outside\",\"7,37\":\"outside\",\"8,31\":\"wall\",\"8,32\":\"open\",\"8,33\":\"open\",\"8,34\":\"open\",\"8,35\":\"open\",\"8,36\":\"outside\",\"8,37\":\"outside\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "d98de736fbd9633afec67c9b3ab678bb14731d13cec2217a84b9c7255291f7fd"}163{"id": "snake_food-000162", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (1,4), travelling west, length 3.\n........\n.*..Ho..\n.....o..\n........\n........\n........\n........\n........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "4f362b4bac9018d7dac48f2d9d3c297b664af1ca206d09cd93681dd1343272f5"}164{"id": "reachable_within-000163", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (21,13), goal at (19,21).\n################################\n#...#...#...........#.........##\n#.#.#.#.###.#######.#.#####.#.##\n#.#...#...#.....#.#...#...#.#.##\n#.#######.#####.#.#####.#.#.####\n#.#...#...#...#.......#.#.#...##\n#.###.#.###.#.#####.###.#.###.##\n#...#.#...#.#.......#...#...#.##\n###.#.###.#########.#.#####.#.##\n#.#...#.#...#.....#.#...#.#...##\n#.###.#.###.#.###.#.###.#.###.##\n#...#.....#...#...#.#.#.#...#.##\n###.#####.#####.###.#.#.#.###.##\n#.......#.....#...#...#.#...#.##\n#.#.#########.###.###.#.###.#.##\n#.#.#.......#.#.#.#...#.#...#.##\n#.#.#.###.###.#.#.###.#.#.#.#.##\n#.#.#.#...#...#.#...#.#.#.#.#.##\n#.###.#.###.###.###.###.#.###.##\n#.#...#.....#...#...#G#.#.....##\n#.#.###########.#.###.#.########\n#.#...#.....#A..#.....#.#.....##\n#.###.###.#.#.#.#######.#.###.##\n#...#.#...#...#...#.........#.##\n#.#.#.#.#########.#.#######.#.##\n#.#...#.#.....#...#.#...#...#.##\n#.#####.###.###.#####.#.#.###.##\n#...#.....#.#...#.....#.#...#.##\n###.#####.#.#.###.#####.#####.##\n#.........#.......#...........##\n################################\n################################", "question": "Can the agent reach the goal in at most 162 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "0eba4c39650e845a04920956f01207b3e6f109b5f7a910ea86b2bc00f61384e7"}165{"id": "move_two_step-000164", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[8,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"wall\",\"7,3\":\"open\",\"8,-1\":\"outside\",\"8,0\":\"wall\",\"8,1\":\"agent\",\"8,2\":\"wall\",\"8,3\":\"open\",\"9,-1\":\"outside\",\"9,0\":\"wall\",\"9,1\":\"open\",\"9,2\":\"wall\",\"9,3\":\"open\",\"10,-1\":\"outside\",\"10,0\":\"wall\",\"10,1\":\"wall\",\"10,2\":\"wall\",\"10,3\":\"wall\"}}", "question": "The agent plans to move south, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "19e4c0bf4cb7c05dd99224bada5e04031ef419fd3402b7f0547e77dc7182fa69"}166{"id": "blocked_pair-000165", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,27],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,24\":\"outside\",\"-2,25\":\"outside\",\"-2,26\":\"outside\",\"-2,27\":\"outside\",\"-2,28\":\"outside\",\"-2,29\":\"outside\",\"-2,30\":\"outside\",\"-1,24\":\"outside\",\"-1,25\":\"outside\",\"-1,26\":\"outside\",\"-1,27\":\"outside\",\"-1,28\":\"outside\",\"-1,29\":\"outside\",\"-1,30\":\"outside\",\"0,24\":\"wall\",\"0,25\":\"open\",\"0,26\":\"open\",\"0,27\":\"wall\",\"0,28\":\"outside\",\"0,29\":\"outside\",\"0,30\":\"outside\",\"1,24\":\"open\",\"1,25\":\"open\",\"1,26\":\"open\",\"1,27\":\"agent\",\"1,28\":\"outside\",\"1,29\":\"outside\",\"1,30\":\"outside\",\"2,24\":\"open\",\"2,25\":\"open\",\"2,26\":\"wall\",\"2,27\":\"open\",\"2,28\":\"outside\",\"2,29\":\"outside\",\"2,30\":\"outside\",\"3,24\":\"open\",\"3,25\":\"open\",\"3,26\":\"wall\",\"3,27\":\"open\",\"3,28\":\"outside\",\"3,29\":\"outside\",\"3,30\":\"outside\",\"4,24\":\"open\",\"4,25\":\"open\",\"4,26\":\"open\",\"4,27\":\"open\",\"4,28\":\"outside\",\"4,29\":\"outside\",\"4,30\":\"outside\"}}", "question": "Are BOTH the cell one step east and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "557e7e08eee0d28acc382b15746b4ac0678042675450d4217102afaa03708945"}167{"id": "first_move-000166", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,7), goal at (1,15).\n...#...###..#..#.##.##.#####\n.......#......#G.#.....#####\n#.......#.##.###...##.######\n...##..#.....#####.....###.#\n.###........#####..#..#.#...\n.###...#.#...###...#....#..#\n####....#...#.##.#...#......\n#####.##...#..#.#..#..#.....\n#######.#..#.....##...##..#.\n#####.#A..#..##..####..#.#..\n#####....#.#.#....#..#...##.\n#.###.#.##......#.....#..###\n......#####.#.....#.#......#\n......#..##.#....#.......#.#\n.#............#.##..........\n.#..#..#.......##.#.#..##..#\n##.......#..###........#..##\n...##.#.###....#.##.........\n....####.#...##....#.....#.#\n.....###........#.#........#\n.#........##.##.....#.#.#.#.\n..##....#.......#.....#.....\n.#.#...##...##..#.#.#.#.##..\n.....#...#####.#...##.....##\n...#.#.#######.##.#..#..#.##\n###############..#.....#####\n###############...####.#####\n################.......#####", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "6fc44acbe74499f47f0857f675e2335609b5195cc8a76423274cf3b2925cb632"}168{"id": "only_way-000167", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (15,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#######\n#.....#\n#.###.#\n#.#A..#\n#.#####\n#......\n#######", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "07ed1c3f36972b91791f4836ae6c5b74aa17a6d154b12e47f1d61a7f13dfed6a"}169{"id": "blocked_pair-000168", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (5,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.#####\n#.#.#..\n#.#.###\n#.#A..#\n#.###.#\n#...#..\n#.#.###", "question": "Are BOTH the cell one step north and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "f13161e504a868828af6eab64e7c54be18f218ef353b9f1cc6fde9f8aa01dd63"}170{"id": "move_two_step-000169", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 5. Rows increase to the south and columns to the east.\nRow -3: column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze\nRow -2: column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze\nRow -1: column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze\nRow 0: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is open, column 5 is wall, column 6 is open, column 7 is open, column 8 is open, column 9 is open\nRow 1: column 1 is wall, column 2 is wall, column 3 is open, column 4 is open, column 5 is the agent, column 6 is open, column 7 is wall, column 8 is wall, column 9 is open\nRow 2: column 1 is wall, column 2 is wall, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is wall, column 8 is open, column 9 is open\nRow 3: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is open, column 9 is wall\nRow 4: column 1 is wall, column 2 is wall, column 3 is open, column 4 is open, column 5 is wall, column 6 is open, column 7 is open, column 8 is open, column 9 is open\nRow 5: column 1 is wall, column 2 is open, column 3 is open, column 4 is open, column 5 is wall, column 6 is open, column 7 is open, column 8 is wall, column 9 is open", "question": "The agent plans to move east, then east. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "c3cf3859f7701a370a189eafd225cbc951f57cac75ba75ce3bf129b509aea94b"}171{"id": "snake_safe-000170", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,7), travelling north, length 3.\n............\n.....*......\n............\n............\n............\n.......H....\n.......o....\n.......o....\n............\n............\n............\n............", "question": "If the snake moves north on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "22ed76a65e935bdcab6970b0f1bac89f4ae9acce145c31ea05778071d20fe448"}172{"id": "plan_survives-000171", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[2,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,-3\":\"outside\",\"-2,-2\":\"outside\",\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-2,5\":\"outside\",\"-1,-3\":\"outside\",\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"0,-3\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"1,-3\":\"outside\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"2,-3\":\"outside\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"agent\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"3,-3\":\"outside\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"4,-3\":\"outside\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"wall\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\",\"5,-3\":\"outside\",\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"6,-3\":\"outside\",\"6,-2\":\"outside\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"open\"}}", "question": "The agent will attempt this plan in order: west, west, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "2dbfac6e0cb4601d435340667fdf48e2d143b5750a5cf2e29d65a246034108ef"}173{"id": "plan_survives-000172", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 6, column 13. Rows increase to the south and columns to the east.\nRow 2: column 9 is open, column 10 is wall, column 11 is wall, column 12 is open, column 13 is open, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 3: column 9 is open, column 10 is wall, column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 4: column 9 is wall, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 5: column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 6: column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is the agent, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 7: column 9 is open, column 10 is open, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 8: column 9 is wall, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 9: column 9 is open, column 10 is open, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 10: column 9 is wall, column 10 is wall, column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze", "question": "The agent will attempt this plan in order: north, south, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "9ce4d2c687313b993864d4fcf5b721b7ed356efc2ba0ac3a82a05ec7bd2c188e"}174{"id": "reachable_within-000173", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (22,13), goal at (31,4).\n.#.#.....######..##...##..###.##\n...#.......###...#..............\n#....#.#....#.......#..#........\n##...#..##.#.........#..........\n#...............#.............#.\n##.#.##.#...#........#....#.....\n....#....#...#.######.#.........\n..#...#.###.........#.........#.\n.........#...##..............#..\n...#....##....#.#....#..#.......\n..#.....#.#....#...........#....\n#..#.#.....#...##....#....#..#..\n####......#.#..##..#...........#\n..##............##.#...#.#..#.##\n..##.......#.......#.#......#...\n...#........#.#.##......#......#\n...##.#....#.#...#.....#......#.\n....##....##....#.#......##.....\n.#...##..##....##.#....##...##..\n....###.###........#...#........\n.....#..#..#...#.......#........\n.#.#...#.....#....#......#....##\n.#.#...#.....A.....##....#......\n.#......#......#..##............\n.......#.......#.####...#.#...##\n#.#.#.....#..#.#..#..##..#..#...\n###...#......#..................\n##..#.....#............#....#..#\n#...##...#..#..##....#..##......\n#..####....#...#..#......#...#..\n.....##.....#.##.........#.#..#.\n...#G#........#.....#...........", "question": "Can the agent reach the goal in at most 26 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "453653e87846ac57933f44c586b7fad86ffbe7dd13f129c7d59a488fff5212df"}175{"id": "first_move-000174", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,2), goal at (5,5).\n#########\n#.....#.#\n#.###.#.#\n#.A.....#\n#.#####.#\n#...#G..#\n#.#.#####\n#.......#\n#########", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b1fad45727fbd52288f0764f283c47ecdec23d1237735fe83893edf9c55b598d"}176{"id": "first_move-000175", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,15), goal at (31,25).\n####################################\n#.....................#...........##\n#.###################.#.#########.##\n#.#...................#.#.......#.##\n#.#####################.#.#####.#.##\n#.....................#.#.#...#.#.##\n#.###################.#.#.#.#.#.#.##\n#.#...#.............#.#.#.#.#.#.#.##\n#.###.#.###########.#.#.#.#.#.#.#.##\n#.#...#.#......A#...#.#.#.#.#.#.#.##\n#.#.###.#######.#.#.#.#.#.#.#.#.#.##\n#.#.............#.#.#.#.#.#.#.#.#.##\n#.###############.#.#.#.#.#.#.#.#.##\n#.#.............#.#.#.#.#.#.#.#.#.##\n#.#.###########.#.#.#.#.#.#.#.#.#.##\n#.#.........#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#######.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.....#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.###.#.#.#.#.#.#.#.#.###.#.#.##\n#.#.#...#.#.#.#.#.#.#.#.#.....#.#.##\n#.#.#####.#.#.#.#.#.#.#.#######.#.##\n#.#.......#.#.#.#.#.#.#.#.......#.##\n#.#######.#.#.#.#.#.#.#.#.#####.#.##\n#.#.....#.#.#.#.#.#.#.#.#.#.....#.##\n#.#.#####.#.#.#.#.###.#.#.#######.##\n#.#.......#.#.#.#...#.#.#.......#.##\n#.#########.#.#.#.#.#.#.#.#####.#.##\n#...........#.#.#.#.#.#.#.....#.#.##\n#############.#.###.#.#.#######.#.##\n#...................#...........#.##\n###############################.#.##\n#........................G#.....#.##\n#.###############################.##\n#.................................##\n####################################\n####################################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "3e4eb9431064e884fe676e8dc9417c5bc0cbcfb0c176886d5dc71fb18eaa7faf"}177{"id": "snake_safe-000176", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (4,7), travelling north, length 3.\n............\n............\n..*.........\n............\n.......H....\n......oo....\n............\n............\n............\n............\n............\n............", "question": "If the snake moves west on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "970c6fbc0ce1e749992e50b8ef35e7e9ab6d26455a097dafb065199effb32d76"}178{"id": "blocked_pair-000177", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 17, column 5. Rows increase to the south and columns to the east.\nRow 15: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open\nRow 16: column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open\nRow 17: column 3 is open, column 4 is wall, column 5 is the agent, column 6 is wall, column 7 is open\nRow 18: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open\nRow 19: column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open", "question": "Are BOTH the cell one step west and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "da600876ad7108de0f87bf67b795a2e41e22a7cc545e3b25913fc615217a66ab"}179{"id": "move_two_step-000178", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (17,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXX#####\nXX#...#\nXX#.#.#\nXX#A#..\nXX#.###\nXX#.#..\nXX#####", "question": "The agent plans to move south, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "18fd3551de2f5585479d98a1a6c457c1a356fc494d8b706e6464d9a016a6c0e4"}180{"id": "move_legal-000179", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (9,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#####.#\n....#.#\n###.#.#\n..#A..#\n#######\n#######\nXXXXXXX", "question": "The agent wants to move north. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "3a0151ef25625e8f90cf2eb4833ec8a2e3f1a5aad901dfbc9f5a94989a94a8e1"}181{"id": "onward-000180", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[11,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"7,-3\":\"outside\",\"7,-2\":\"outside\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"wall\",\"7,2\":\"open\",\"7,3\":\"wall\",\"7,4\":\"open\",\"7,5\":\"open\",\"8,-3\":\"outside\",\"8,-2\":\"outside\",\"8,-1\":\"outside\",\"8,0\":\"open\",\"8,1\":\"open\",\"8,2\":\"open\",\"8,3\":\"open\",\"8,4\":\"open\",\"8,5\":\"open\",\"9,-3\":\"outside\",\"9,-2\":\"outside\",\"9,-1\":\"outside\",\"9,0\":\"wall\",\"9,1\":\"open\",\"9,2\":\"open\",\"9,3\":\"open\",\"9,4\":\"open\",\"9,5\":\"open\",\"10,-3\":\"outside\",\"10,-2\":\"outside\",\"10,-1\":\"outside\",\"10,0\":\"wall\",\"10,1\":\"open\",\"10,2\":\"open\",\"10,3\":\"open\",\"10,4\":\"wall\",\"10,5\":\"wall\",\"11,-3\":\"outside\",\"11,-2\":\"outside\",\"11,-1\":\"outside\",\"11,0\":\"open\",\"11,1\":\"agent\",\"11,2\":\"wall\",\"11,3\":\"wall\",\"11,4\":\"wall\",\"11,5\":\"wall\",\"12,-3\":\"outside\",\"12,-2\":\"outside\",\"12,-1\":\"outside\",\"12,0\":\"outside\",\"12,1\":\"outside\",\"12,2\":\"outside\",\"12,3\":\"outside\",\"12,4\":\"outside\",\"12,5\":\"outside\",\"13,-3\":\"outside\",\"13,-2\":\"outside\",\"13,-1\":\"outside\",\"13,0\":\"outside\",\"13,1\":\"outside\",\"13,2\":\"outside\",\"13,3\":\"outside\",\"13,4\":\"outside\",\"13,5\":\"outside\",\"14,-3\":\"outside\",\"14,-2\":\"outside\",\"14,-1\":\"outside\",\"14,0\":\"outside\",\"14,1\":\"outside\",\"14,2\":\"outside\",\"14,3\":\"outside\",\"14,4\":\"outside\",\"14,5\":\"outside\",\"15,-3\":\"outside\",\"15,-2\":\"outside\",\"15,-1\":\"outside\",\"15,0\":\"outside\",\"15,1\":\"outside\",\"15,2\":\"outside\",\"15,3\":\"outside\",\"15,4\":\"outside\",\"15,5\":\"outside\"}}", "question": "Suppose the agent moves one cell west. From that new cell, would a further step north be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "3ac232347921709275b14086e99d9df82b3bac96f612affa3007c1d5c39cf476"}182{"id": "distance_band-000181", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,5), goal at (5,1).\n################\n#.....#.......##\n#.###A#.#.###.##\n#...#...#...#.##\n###.#####.#.#.##\n#G..#.....#.#.##\n#.#.#.###.#.#.##\n#...#.....#...##\n###.#..##.##.###\n#.......#.#...##\n#.#####.#.#.#.##\n#.#.....#.....##\n#.#####.#.#.#.##\n#...........#.##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "9238350fa0335bc1173f05513d17e2d7935f7c6d82cfad4e5b49de894113610b"}183{"id": "blocked_pair-000182", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (27,26); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n...#.XX\n##..#XX\n#..#.XX\n...A.XX\nXXXXXXX\nXXXXXXX\nXXXXXXX", "question": "Are BOTH the cell one step south and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "b73114cdc35f0a76b450d07734437a52f247b205861768f80eb6edf0b4decdb5"}184{"id": "plan_survives-000183", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[33,32],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"31,30\":\"open\",\"31,31\":\"open\",\"31,32\":\"open\",\"31,33\":\"open\",\"31,34\":\"wall\",\"32,30\":\"open\",\"32,31\":\"open\",\"32,32\":\"wall\",\"32,33\":\"open\",\"32,34\":\"wall\",\"33,30\":\"wall\",\"33,31\":\"open\",\"33,32\":\"agent\",\"33,33\":\"open\",\"33,34\":\"wall\",\"34,30\":\"wall\",\"34,31\":\"wall\",\"34,32\":\"wall\",\"34,33\":\"wall\",\"34,34\":\"wall\",\"35,30\":\"wall\",\"35,31\":\"wall\",\"35,32\":\"wall\",\"35,33\":\"wall\",\"35,34\":\"wall\"}}", "question": "The agent will attempt this plan in order: east, east, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "0ebab8f84c1bc38d80c00978a54ec159d2d452f39ebeee467100b46c09b017ac"}185{"id": "reachable_within-000184", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (10,1), goal at (9,13).\n########################\n#.....................##\n#.###################.##\n#.#.................#.##\n#.#.###############.#.##\n#.#.#.............#.#.##\n#.#.#.###.#######.#.#.##\n#.#.#.#.#.#.....#.#.#.##\n#.#.#.#.#.#.###.#.#.#.##\n#.#.#...#.#.#G..#.#.#.##\n#A#.#####.#.#####.#.#.##\n#.#.......#.......#.#.##\n#.#################.#.##\n#...................#.##\n#.###################.##\n#.#...................##\n#.######################\n#.#...................##\n#.#.#######.#########.##\n#.#.......#.#.......#.##\n#.#########.#.#######.##\n#...........#.........##\n########################\n########################", "question": "Can the agent reach the goal in at most 100 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "f63a9394e0889c64dc3915ea170a99b7278fe78fcf63888a3959eb1e7ff7e51f"}186{"id": "only_way-000185", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (5,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.###.#\n....#.#\n#####.#\n..#A#..\n#.#.#.#\n#.#.#.#\n#.#.#.#", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "f75e097b2023827d9cea43f1f4d6b135c276611fb4c9943ebfe7f04ad61de49d"}187{"id": "plan_survives-000186", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 0. Rows increase to the south and columns to the east.\nRow -1: column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze\nRow 0: column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open\nRow 1: column -2 is outside the maze, column -1 is outside the maze, column 0 is the agent, column 1 is wall, column 2 is wall\nRow 2: column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open\nRow 3: column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open", "question": "The agent will attempt this plan in order: north, north, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "c17869fd3cfa70341befef9a8c2c790d5ecabfaf5a67ce82a88d597c7cf17ec5"}188{"id": "move_legal-000187", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[13,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"9,3\":\"open\",\"9,4\":\"wall\",\"9,5\":\"open\",\"9,6\":\"wall\",\"9,7\":\"open\",\"9,8\":\"open\",\"9,9\":\"open\",\"9,10\":\"open\",\"9,11\":\"open\",\"10,3\":\"wall\",\"10,4\":\"wall\",\"10,5\":\"wall\",\"10,6\":\"wall\",\"10,7\":\"open\",\"10,8\":\"wall\",\"10,9\":\"open\",\"10,10\":\"wall\",\"10,11\":\"open\",\"11,3\":\"open\",\"11,4\":\"open\",\"11,5\":\"open\",\"11,6\":\"open\",\"11,7\":\"open\",\"11,8\":\"wall\",\"11,9\":\"open\",\"11,10\":\"open\",\"11,11\":\"open\",\"12,3\":\"open\",\"12,4\":\"wall\",\"12,5\":\"wall\",\"12,6\":\"wall\",\"12,7\":\"wall\",\"12,8\":\"wall\",\"12,9\":\"wall\",\"12,10\":\"wall\",\"12,11\":\"open\",\"13,3\":\"open\",\"13,4\":\"wall\",\"13,5\":\"open\",\"13,6\":\"open\",\"13,7\":\"agent\",\"13,8\":\"open\",\"13,9\":\"open\",\"13,10\":\"open\",\"13,11\":\"open\",\"14,3\":\"wall\",\"14,4\":\"wall\",\"14,5\":\"wall\",\"14,6\":\"wall\",\"14,7\":\"wall\",\"14,8\":\"wall\",\"14,9\":\"wall\",\"14,10\":\"wall\",\"14,11\":\"wall\",\"15,3\":\"wall\",\"15,4\":\"wall\",\"15,5\":\"wall\",\"15,6\":\"wall\",\"15,7\":\"wall\",\"15,8\":\"wall\",\"15,9\":\"wall\",\"15,10\":\"wall\",\"15,11\":\"wall\",\"16,3\":\"outside\",\"16,4\":\"outside\",\"16,5\":\"outside\",\"16,6\":\"outside\",\"16,7\":\"outside\",\"16,8\":\"outside\",\"16,9\":\"outside\",\"16,10\":\"outside\",\"16,11\":\"outside\",\"17,3\":\"outside\",\"17,4\":\"outside\",\"17,5\":\"outside\",\"17,6\":\"outside\",\"17,7\":\"outside\",\"17,8\":\"outside\",\"17,9\":\"outside\",\"17,10\":\"outside\",\"17,11\":\"outside\"}}", "question": "The agent wants to move east. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "ad12c90d91bbe8a52476b755f6cddd576c96b01a811e1e47e7eee2a5270987df"}189{"id": "stochastic-000188", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 7, column 17. Rows increase to the south and columns to the east.\nRow 3: column 13 is open, column 14 is wall, column 15 is open, column 16 is open, column 17 is open, column 18 is wall, column 19 is open, column 20 is open, column 21 is open\nRow 4: column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is wall\nRow 5: column 13 is open, column 14 is open, column 15 is open, column 16 is wall, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is open\nRow 6: column 13 is wall, column 14 is wall, column 15 is open, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is open, column 20 is wall, column 21 is open\nRow 7: column 13 is open, column 14 is wall, column 15 is open, column 16 is open, column 17 is the agent, column 18 is wall, column 19 is open, column 20 is wall, column 21 is open\nRow 8: column 13 is open, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is open, column 20 is wall, column 21 is open\nRow 9: column 13 is open, column 14 is open, column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is open, column 21 is open\nRow 10: column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is wall\nRow 11: column 13 is open, column 14 is open, column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is open, column 21 is open", "question": "The actuator executes the intended move with probability 0.90; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9666666666666667, 0.033333333333333326], "deterministic": false, "fingerprint": "6ddd9c6aedc27b75c4b2b329684993084d4914c79785f983ea6161441056d1b5"}190{"id": "goal_bearing-000189", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (6,5), goal at (9,5).\n############\n#.........##\n#.#######.##\n#.#.....#.##\n#.#.###.#.##\n#.#.#.#.#.##\n#.#.#A#.#.##\n#.#.#...#.##\n#.#.#####.##\n#...#G....##\n############\n############", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 1, "answer": "south", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "ff9b9c9417cd29fa2dd8391fc107ede0bfae83194faec79016f0a3600aa0b11d"}191{"id": "onward-000190", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (4,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX########\nX#.....#.\nX#.###.#.\nX#.#.#...\nX#.#A#.##\nX#.#.#...\nX#.#.####\nX#.#...#.\nX#.###.##", "question": "Suppose the agent moves one cell north. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "2038152cf3fd20b32918b56a18ff06f12d4dbaad5f4f20d780742150c9385a9c"}192{"id": "open_count-000191", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[14,19],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"11,16\":\"wall\",\"11,17\":\"wall\",\"11,18\":\"open\",\"11,19\":\"open\",\"11,20\":\"wall\",\"11,21\":\"outside\",\"11,22\":\"outside\",\"12,16\":\"wall\",\"12,17\":\"wall\",\"12,18\":\"wall\",\"12,19\":\"open\",\"12,20\":\"open\",\"12,21\":\"outside\",\"12,22\":\"outside\",\"13,16\":\"wall\",\"13,17\":\"wall\",\"13,18\":\"open\",\"13,19\":\"open\",\"13,20\":\"open\",\"13,21\":\"outside\",\"13,22\":\"outside\",\"14,16\":\"wall\",\"14,17\":\"wall\",\"14,18\":\"open\",\"14,19\":\"agent\",\"14,20\":\"open\",\"14,21\":\"outside\",\"14,22\":\"outside\",\"15,16\":\"wall\",\"15,17\":\"wall\",\"15,18\":\"wall\",\"15,19\":\"open\",\"15,20\":\"wall\",\"15,21\":\"outside\",\"15,22\":\"outside\",\"16,16\":\"wall\",\"16,17\":\"wall\",\"16,18\":\"wall\",\"16,19\":\"wall\",\"16,20\":\"wall\",\"16,21\":\"outside\",\"16,22\":\"outside\",\"17,16\":\"wall\",\"17,17\":\"wall\",\"17,18\":\"wall\",\"17,19\":\"wall\",\"17,20\":\"wall\",\"17,21\":\"outside\",\"17,22\":\"outside\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 4, "answer": "4", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "367f39dcea9b73a047d14ce9c3f18d09a667b0d03ad3a6ed9329cbfc0cde6c09"}193{"id": "onward-000192", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 16, column 2. Rows increase to the south and columns to the east.\nRow 12: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall\nRow 13: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is open\nRow 14: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is open\nRow 15: column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open, column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall\nRow 16: column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is the agent, column 3 is open, column 4 is wall, column 5 is open, column 6 is open\nRow 17: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is wall\nRow 18: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is open\nRow 19: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is wall, column 6 is open\nRow 20: column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open", "question": "Suppose the agent moves one cell west. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "975511aa9f55de456105562b4f5ea3e20d545e2d34bae72387ec687eea0cb4a7"}194{"id": "reachable_within-000193", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,17), goal at (17,13).\n################################\n#...............#.............##\n#.#######.#####.#.##############\n#.#.....#.#...#.#A#...........##\n#.#.###.#.#.#.#.#.###########.##\n#.#.#.#.#.#.#.#.#.....#.....#.##\n#.#.#.#.#.#.#.#.#.#####.###.#.##\n#.#.#.#.#.#.#.#.#.........#.#.##\n#.#.#.#.#.#.###.###########.#.##\n#.#.#...#.#.................#.##\n#.#.#.###.###################.##\n#.#.#...#.....................##\n#.#.############################\n#.#...........................##\n#.#.#####################.###.##\n#.#.#...................#.#...##\n#.#.#.#####.###########.#.######\n#.#.#.#...#.#G........#.#.....##\n#.#.#.#.#.#.#########.#.#####.##\n#.#.#.#.#.#.#.........#.#...#.##\n#.#.#.###.#.#.#######.#.#.#.#.##\n#.#.#.....#.#.#.......#.#.#.#.##\n#.#.#######.#.#########.###.#.##\n#.#.........#...............#.##\n#.###########################.##\n#.............................##\n#############################.##\n#...........................#.##\n#.#####################.#####.##\n#.....................#.......##\n################################\n################################", "question": "Can the agent reach the goal in at most 226 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "9c053f8f4418cafbba745c5e5692a6401f1a42b406001c59e121ca6e75d31e98"}195{"id": "only_way-000194", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[17,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"13,3\":\"open\",\"13,4\":\"wall\",\"13,5\":\"open\",\"13,6\":\"open\",\"13,7\":\"open\",\"13,8\":\"open\",\"13,9\":\"open\",\"13,10\":\"wall\",\"13,11\":\"open\",\"14,3\":\"open\",\"14,4\":\"wall\",\"14,5\":\"open\",\"14,6\":\"wall\",\"14,7\":\"wall\",\"14,8\":\"wall\",\"14,9\":\"wall\",\"14,10\":\"wall\",\"14,11\":\"open\",\"15,3\":\"open\",\"15,4\":\"wall\",\"15,5\":\"open\",\"15,6\":\"wall\",\"15,7\":\"open\",\"15,8\":\"open\",\"15,9\":\"open\",\"15,10\":\"open\",\"15,11\":\"open\",\"16,3\":\"open\",\"16,4\":\"wall\",\"16,5\":\"open\",\"16,6\":\"wall\",\"16,7\":\"open\",\"16,8\":\"wall\",\"16,9\":\"wall\",\"16,10\":\"wall\",\"16,11\":\"wall\",\"17,3\":\"open\",\"17,4\":\"wall\",\"17,5\":\"open\",\"17,6\":\"wall\",\"17,7\":\"agent\",\"17,8\":\"wall\",\"17,9\":\"open\",\"17,10\":\"open\",\"17,11\":\"open\",\"18,3\":\"open\",\"18,4\":\"wall\",\"18,5\":\"open\",\"18,6\":\"wall\",\"18,7\":\"wall\",\"18,8\":\"wall\",\"18,9\":\"open\",\"18,10\":\"wall\",\"18,11\":\"wall\",\"19,3\":\"open\",\"19,4\":\"wall\",\"19,5\":\"open\",\"19,6\":\"open\",\"19,7\":\"open\",\"19,8\":\"open\",\"19,9\":\"open\",\"19,10\":\"open\",\"19,11\":\"open\",\"20,3\":\"wall\",\"20,4\":\"wall\",\"20,5\":\"wall\",\"20,6\":\"wall\",\"20,7\":\"wall\",\"20,8\":\"wall\",\"20,9\":\"wall\",\"20,10\":\"wall\",\"20,11\":\"wall\",\"21,3\":\"outside\",\"21,4\":\"outside\",\"21,5\":\"outside\",\"21,6\":\"outside\",\"21,7\":\"outside\",\"21,8\":\"outside\",\"21,9\":\"outside\",\"21,10\":\"outside\",\"21,11\":\"outside\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "87a6a73bb79bafa8b137e806d98ed709bea6c2bbd6eb89fba99ada21409cb3a4"}196{"id": "goal_bearing-000195", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,2), goal at (6,5).\n#########\n#.A.....#\n#.#####.#\n#.......#\n#.#.###.#\n#...#...#\n###.#G#.#\n#.....#.#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 6, "answer": "south-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "51601302b18280b389d3cbdbd1d27c5bed8c8361185d85a2f0cda6b0b696c3bd"}197{"id": "goal_bearing-000196", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (30,19), goal at (3,19).\n####################################\n#...#.......#...............#.....##\n###.#.###.###.###.#########.#.###.##\n#...#.#.#.#...#...#G#...#.#.#...#.##\n#.#.#.#.#.#.###.###.#.#.#.#.#####.##\n#.#.#.#.#.#.#...#...#.#...#.......##\n#.###.#.#.#.#####.###.###.#######.##\n#.#...#.#.#...#...#...#...#...#.#.##\n#.#.###.#.###.#.###.#####.#.#.#.#.##\n#.#.#...#...#...#.......#.#.#...#.##\n#.#.#.#.###.#####.#####.#.#.#####.##\n#.....#...#.....#...#.#.#...#.....##\n#######.#####.#.###.#.#.#####.######\n#...#.#.#.....#...#.#.#.....#.....##\n#.#.#.#.#.#######.#.#.#####.#####.##\n#.#.#...#...#.....#.#.....#.......##\n#.#.#######.#####.#.###.############\n#.#.........#...#.#...#.......#...##\n#.#########.#.#.#####.#.#####.#.#.##\n#.....#.....#.#.......#.....#...#.##\n#.###.#######.#############.#####.##\n#...#...#.....#...........#.#.....##\n###.###.#.#####.#.#######.#.#.###.##\n#.#...#...#.....#.#.....#.#.#.#...##\n#.###.###########.###.###.###.#.####\n#...#...#.......#...#...#.....#.#.##\n#.#.###.###.###.###.###.#######.#.##\n#.#...#...#.#.......#.........#...##\n#.#.#####.#.#########.###########.##\n#.#.#.....#...#.....#...........#.##\n#.###.#####.#.#.###A#.#########.#.##\n#...#.#...#.#.#...#.#.......#...#.##\n#.#.#.#.#.###.#.#.#########.#.###.##\n#.#.....#.....#.#...........#.....##\n####################################\n####################################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "1168dc5bd0d3e64264484e95523930da5a5277eca3ebf80b442a1ec691cc2622"}198{"id": "cell_kind-000197", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"2,1\":\"wall\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"open\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"agent\",\"3,4\":\"open\",\"3,5\":\"open\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "1747a18b4634ac1465d3894711da40dec5c6eb7cd9921342a32fbc30ea4d0b8b"}199{"id": "snake_food-000198", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,8), travelling north, length 3.\n............\n............\n............\n............\n............\n............\n...*....H...\n.......oo...\n............\n............\n............\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "aaf795e7c8e802bacf5b5dfe3f38bb4aa9eade7ad468cac94c65482ee222d0f3"}200{"id": "distance_band-000199", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (14,3), goal at (15,0).\n.#.....##...#...\n.#..#...#....#..\n...##....#..#...\n..##........#.#.\n#.....#........#\n#....###......#.\n#......#...#....\n............#..#\n#......#.##...#.\n.#.#.#.#.#......\n.....#.#...#.#..\n.###...##...####\n#.......#.######\n####.#...#######\n#.#A..#.########\nG...#.#.########", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "1e1adae21e1d4632c71216aae5650bca2794df06d9f380d75c00ebac6bfdff2a"}201{"id": "first_move-000200", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,9), goal at (5,1).\n################\n#.......#.....##\n#####.###.###.##\n#...#....A#.#.##\n#.#.#.#####.#.##\n#G#.#.#...#.#.##\n###.#.#.#.#.#.##\n#...#...#...#.##\n#.###.#####.#.##\n#.#...#...#.#.##\n#.#####.#.###.##\n#...#...#...#.##\n###.#.#####.#.##\n#.....#.......##\n################\n################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "7ec3f30f853f870295ac862a3db76b6b72f15ad7c1019e00b9674429e97595e6"}202{"id": "stochastic-000201", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (14,14); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#...X\n.##.X\n..A.X\n...#X\nXXXXX", "question": "The actuator executes the intended move with probability 0.70; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.7, 0.30000000000000004], "deterministic": false, "fingerprint": "523a79002e98e4cbed7bf5a9067e814529e784ce9a87a5827521fad55278612d"}203{"id": "only_way-000202", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (13,17); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#...\n.###.\n.#A#.\n.#.#.\n.#.#.", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "5a5d011ec360f7e149ba1b801ffdd288e936d8005974389c02c399c1a13872ed"}204{"id": "blocked_pair-000203", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"agent\",\"3,4\":\"open\",\"3,5\":\"open\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"open\",\"5,5\":\"open\"}}", "question": "Are BOTH the cell one step west and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "cd20cb2fce03c974eea1fa1d5b4a7704dc09393b6093ffee82dfc4701679cdc5"}205{"id": "open_count-000204", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 3, column 1. Rows increase to the south and columns to the east.\nRow -1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 0: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open\nRow 2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open\nRow 3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is open, column 3 is open, column 4 is wall, column 5 is open\nRow 4: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall\nRow 5: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 6: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 7: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 3, "answer": "3", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "65d00bff96627ec18515caa18c66327ddd8b867eed119456a197ed17d09eb0aa"}206{"id": "snake_safe-000205", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,8), travelling south, length 3.\n..............\n..............\n..............\n..............\n..............\n.......oo*....\n........H.....\n..............\n..............\n..............\n..............\n..............\n..............\n..............", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "22476d0353cbd9828eeca165e55d13f78332963a54d22eb750ec3bf1f104f5e2"}207{"id": "open_count-000206", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[19,18],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"15,14\":\"open\",\"15,15\":\"open\",\"15,16\":\"open\",\"15,17\":\"open\",\"15,18\":\"wall\",\"15,19\":\"open\",\"15,20\":\"open\",\"15,21\":\"outside\",\"15,22\":\"outside\",\"16,14\":\"open\",\"16,15\":\"wall\",\"16,16\":\"open\",\"16,17\":\"open\",\"16,18\":\"open\",\"16,19\":\"open\",\"16,20\":\"open\",\"16,21\":\"outside\",\"16,22\":\"outside\",\"17,14\":\"wall\",\"17,15\":\"wall\",\"17,16\":\"open\",\"17,17\":\"wall\",\"17,18\":\"open\",\"17,19\":\"open\",\"17,20\":\"open\",\"17,21\":\"outside\",\"17,22\":\"outside\",\"18,14\":\"open\",\"18,15\":\"open\",\"18,16\":\"open\",\"18,17\":\"open\",\"18,18\":\"open\",\"18,19\":\"open\",\"18,20\":\"open\",\"18,21\":\"outside\",\"18,22\":\"outside\",\"19,14\":\"open\",\"19,15\":\"wall\",\"19,16\":\"open\",\"19,17\":\"open\",\"19,18\":\"agent\",\"19,19\":\"open\",\"19,20\":\"wall\",\"19,21\":\"outside\",\"19,22\":\"outside\",\"20,14\":\"open\",\"20,15\":\"wall\",\"20,16\":\"open\",\"20,17\":\"wall\",\"20,18\":\"open\",\"20,19\":\"wall\",\"20,20\":\"wall\",\"20,21\":\"outside\",\"20,22\":\"outside\",\"21,14\":\"outside\",\"21,15\":\"outside\",\"21,16\":\"outside\",\"21,17\":\"outside\",\"21,18\":\"outside\",\"21,19\":\"outside\",\"21,20\":\"outside\",\"21,21\":\"outside\",\"21,22\":\"outside\",\"22,14\":\"outside\",\"22,15\":\"outside\",\"22,16\":\"outside\",\"22,17\":\"outside\",\"22,18\":\"outside\",\"22,19\":\"outside\",\"22,20\":\"outside\",\"22,21\":\"outside\",\"22,22\":\"outside\",\"23,14\":\"outside\",\"23,15\":\"outside\",\"23,16\":\"outside\",\"23,17\":\"outside\",\"23,18\":\"outside\",\"23,19\":\"outside\",\"23,20\":\"outside\",\"23,21\":\"outside\",\"23,22\":\"outside\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 4, "answer": "4", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "d574c0d57dfa5e7ca5df38cac836d6172eb0810b439304da2233b912094be1e9"}208{"id": "reachable_within-000207", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,19), goal at (1,17).\n########################\n#...............#G....##\n###.####..##..#.#####.##\n#.............#.....#.##\n#.###.#####.#.#####.#.##\n#.#.....#...#...#...#.##\n#.#####.#.#.#.#.#.###.##\n#.#...#.....#.#.#.#...##\n#.#.#.##.##.#.###.#.#.##\n#.#.#.....#.......#.#.##\n#.#.#####.#.#.#.#.#.#.##\n#...#...#...#.....#.#.##\n#.###.#.#.#.#.#.###.#.##\n#.....#.#...#.#.#..A..##\n#.#.#.#.#.###.#.#.#.#.##\n#...#...#.....#...#...##\n#.###.#.###...#####...##\n#...#.....#.........#.##\n###.#.#.###.##.##.###.##\n#...#.#.....#.......#.##\n#.###.#.#.###.###.#.#.##\n#.......#.............##\n########################\n########################", "question": "Can the agent reach the goal in at most 17 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "7fc712675ff41268334691b0b104488c707623e1172947add759eca0a2489daf"}209{"id": "first_move-000208", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (6,2), goal at (11,11).\n......#.#..#\n#.........##\n..#...#..###\n.....#..####\n.#...#.#####\n.#####..####\n..A#.....###\n.#.###..#.##\n..#....#....\n.#.#...#....\n.........##.\n.#........#G", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "03a0c40ab83f17ec1bef77029d0537b554f8d54958482f6c6a7f169637670ac8"}210{"id": "first_move-000209", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (4,19), goal at (17,7).\n#####################\n#...........#.......#\n#.###.#####.###.###.#\n#.#.#...#.#...#.#...#\n#.#.###.#.###.#.#.#A#\n#.#...#.#...#.#.#.#.#\n#.#.#.#.#.#.#.###.#.#\n#...#.#.#.#.#.....#.#\n#######.#.#.#######.#\n#...#...#.#.#...#...#\n#.#.#.###.###.#.#.###\n#.#.#.#.#...#.#...#.#\n#.#.#.#.#.#.#.#####.#\n#.#...#...#.#.......#\n#.#######.#.#######.#\n#.......#.#...#...#.#\n#######.#.#.###.#.#.#\n#.....#G#.#...#.#.#.#\n#.###.###.###.#.#.#.#\n#...#.......#...#...#\n#####################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "c1e616aa5a1d13dd5d06ffb841e38df0570d396a2927c1307900d5631b436009"}211{"id": "move_two_step-000210", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 0, column 32. Rows increase to the south and columns to the east.\nRow -3: column 29 is outside the maze, column 30 is outside the maze, column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze\nRow -2: column 29 is outside the maze, column 30 is outside the maze, column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze\nRow -1: column 29 is outside the maze, column 30 is outside the maze, column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze\nRow 0: column 29 is open, column 30 is open, column 31 is open, column 32 is the agent, column 33 is open, column 34 is open, column 35 is open\nRow 1: column 29 is open, column 30 is open, column 31 is open, column 32 is open, column 33 is wall, column 34 is wall, column 35 is open\nRow 2: column 29 is open, column 30 is open, column 31 is open, column 32 is open, column 33 is open, column 34 is open, column 35 is open\nRow 3: column 29 is open, column 30 is open, column 31 is open, column 32 is open, column 33 is open, column 34 is open, column 35 is wall", "question": "The agent plans to move east, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "4dbfa6ce1abc29e81017a195236b776cc8331c5fada7f825af6281ebcd9026f0"}212{"id": "snake_food-000211", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (11,6), travelling west, length 3.\n..............\n..............\n..............\n..............\n..............\n......*.......\n..............\n..............\n..............\n..............\n.......o......\n......Ho......\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "528405340e6433c4d12a770c6c689e7b5e0dd5787955c485f9128781a0f34330"}213{"id": "onward-000212", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[7,10],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"5,8\":\"wall\",\"5,9\":\"open\",\"5,10\":\"wall\",\"5,11\":\"open\",\"5,12\":\"open\",\"6,8\":\"wall\",\"6,9\":\"open\",\"6,10\":\"wall\",\"6,11\":\"wall\",\"6,12\":\"wall\",\"7,8\":\"wall\",\"7,9\":\"open\",\"7,10\":\"agent\",\"7,11\":\"open\",\"7,12\":\"wall\",\"8,8\":\"wall\",\"8,9\":\"wall\",\"8,10\":\"wall\",\"8,11\":\"wall\",\"8,12\":\"wall\",\"9,8\":\"wall\",\"9,9\":\"open\",\"9,10\":\"open\",\"9,11\":\"open\",\"9,12\":\"wall\"}}", "question": "Suppose the agent moves one cell east. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "15c9a162563ead12446771a5da2f65fa31912ed765c172800b31e881be8cbb04"}214{"id": "move_two_step-000213", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (7,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX..#.\nX....\nX.A..\nX##..\nXXXXX", "question": "The agent plans to move south, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "508b9233a5fc6a32ef41debc6a5216b674def6e576196275d4ca5ab81df92f09"}215{"id": "blocked_pair-000214", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[19,2],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"17,0\":\"wall\",\"17,1\":\"open\",\"17,2\":\"open\",\"17,3\":\"open\",\"17,4\":\"open\",\"18,0\":\"open\",\"18,1\":\"open\",\"18,2\":\"wall\",\"18,3\":\"wall\",\"18,4\":\"wall\",\"19,0\":\"open\",\"19,1\":\"wall\",\"19,2\":\"agent\",\"19,3\":\"open\",\"19,4\":\"open\",\"20,0\":\"wall\",\"20,1\":\"open\",\"20,2\":\"open\",\"20,3\":\"open\",\"20,4\":\"open\",\"21,0\":\"outside\",\"21,1\":\"outside\",\"21,2\":\"outside\",\"21,3\":\"outside\",\"21,4\":\"outside\"}}", "question": "Are BOTH the cell one step south and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "28436e95e796ed12c657743cd842e04ffda8dcff27a70d6e6a45493727f8695d"}216{"id": "onward-000215", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 14, column 8. Rows increase to the south and columns to the east.\nRow 11: column 5 is wall, column 6 is open, column 7 is open, column 8 is open, column 9 is wall, column 10 is open, column 11 is open\nRow 12: column 5 is open, column 6 is open, column 7 is wall, column 8 is open, column 9 is open, column 10 is open, column 11 is open\nRow 13: column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is wall, column 10 is open, column 11 is open\nRow 14: column 5 is open, column 6 is open, column 7 is open, column 8 is the agent, column 9 is wall, column 10 is open, column 11 is open\nRow 15: column 5 is open, column 6 is open, column 7 is wall, column 8 is open, column 9 is open, column 10 is open, column 11 is open\nRow 16: column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 17: column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze", "question": "Suppose the agent moves one cell west. From that new cell, would a further step north be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "a9103c20f6f7bb1a94c32602e9d30c99dbebd1f697459f408aefe8b90ef4990d"}217{"id": "onward-000216", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (25,29); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n...####\n...####\n.#.###.\n###A...\n.##..##\n...###.\n##.....", "question": "Suppose the agent moves one cell south. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "975b5d2700ec2dec4afa0e23a854a07e1d6cb67f986a58a0e97f4f997a13b9c5"}218{"id": "move_two_step-000217", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,16); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\n#########\n........#\n#######.#\n....A.#.#\n#######.#\n........#\n#########\n........#", "question": "The agent plans to move east, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "220e1db4a3f900f310cacb1a3d8b61d201ed13ff9a19de2f09a59effb255fb56"}219{"id": "distance_band-000218", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (8,6), goal at (0,21).\n..#####.#.#.......###G.#\n.#.###..#.....###..##..#\n...###..#..#.....#.###.#\n....#........######.#...\n...#..#.#...#.##.#...#..\n#.#...#######...........\n...........#.##..#.###..\n...#....###........#..##\n##.#..A###..#...##...###\n##..#...#..........#..##\n###.##...#..#....#.#####\n####.........#.....##...\n#####.....#....######.##\n####....#.........#.....\n#####.....#..#...#..#.##\n#####.#....#....#...#...\n#####.##...####.........\n#######........##..#..#.\n##.##...#.#...#.........\n##.#....#.#..##....#..#.\n.............#...#......\n..##..##...#..##.#.##.##\n#..#...#..##..##....#.##\n....#########....#...###", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 3, "answer": "20 to 39 moves", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "e698cde72a7f37ee6b7cc105efb886d4b7e874ecdfa942a14095fddef208f6eb"}220{"id": "distance_band-000219", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (8,14), goal at (15,15).\n#######...###...\n########........\n####.#......#...\n####....#....#.#\n########..#.....\n####..#.#..#....\n###...#......#..\n#####....#..#...\n####.....#.##.A.\n#####..#...#....\n#####.#.##..##.#\n.#..#.#...##....\n.#..#.#..####...\n..#.....#####...\n#...#....#####..\n..#.#...#######G", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 1, "answer": "5 to 9 moves", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3c61029c33892b87333f8be6e714922f06fdd7e50b9ce90bc21e26f307e398cb"}221{"id": "distance_band-000220", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,2), goal at (5,1).\n################\n#.............##\n#.#..##.#####.##\n#.A.....#.....##\n#.#.#.###.###.##\n#G#...#.......##\n#####.#.#.######\n#...#.#.#.....##\n#.###.#.#.###.##\n#.....#.....#.##\n###.#...#.###.##\n#.#.#.#.#.....##\n#.#.#.#.###.#.##\n#.....#...#...##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "be3541b11c28d9a90820070d6a0dca100d8a911638623f32db5654bb9548fac0"}222{"id": "cell_kind-000221", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[8,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"wall\",\"6,10\":\"wall\",\"6,11\":\"wall\",\"7,7\":\"open\",\"7,8\":\"wall\",\"7,9\":\"open\",\"7,10\":\"wall\",\"7,11\":\"wall\",\"8,7\":\"wall\",\"8,8\":\"wall\",\"8,9\":\"agent\",\"8,10\":\"wall\",\"8,11\":\"wall\",\"9,7\":\"open\",\"9,8\":\"open\",\"9,9\":\"open\",\"9,10\":\"wall\",\"9,11\":\"wall\",\"10,7\":\"wall\",\"10,8\":\"wall\",\"10,9\":\"wall\",\"10,10\":\"wall\",\"10,11\":\"wall\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "2bc551074babba55f10e7f3a4e3bb6e1108ecd92602f7b149a44cd3032e2f598"}223{"id": "snake_food-000222", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (9,7), travelling south, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n.......o......\n.......o......\n.......H*.....\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "710f743f0b433ceeedad65e1cdc3649f617d14694982a80e28e7ebf78b73935e"}224{"id": "goal_bearing-000223", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (12,1), goal at (1,15).\n....#.###..##.##\n#..............G\n...#.##.....#.#.\n.##...##..#...#.\n....#..#........\n...........##...\n...##.##..#.....\n.#......#......#\n.#....##.#......\n...#...#........\n...............#\n.##....#....#...\n.A#..#....#....#\n..#.............\n........##...#..\n........#..#....", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3fc1ac56d8993d46bc899bdf1e40362474a890d441d06bb410aa075204b41193"}225{"id": "only_way-000224", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 13, column 1. Rows increase to the south and columns to the east.\nRow 10: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall\nRow 11: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open\nRow 12: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall\nRow 13: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is open, column 3 is open, column 4 is open\nRow 14: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall\nRow 15: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall\nRow 16: column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "9c1d2e1bd660ec72af2b65df476153c3b24822bcd3b9a80bb359e63af643484d"}226{"id": "cell_kind-000225", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 3, column 1. Rows increase to the south and columns to the east.\nRow -1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 0: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is wall, column 5 is open\nRow 2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open\nRow 3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is open, column 3 is open, column 4 is wall, column 5 is open\nRow 4: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 5: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open\nRow 6: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 7: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "cad5b67eb7fa95c8561147c945c7c2a9882293ccb902415247c8591f2378bbc0"}227{"id": "move_two_step-000226", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 6, column 8. Rows increase to the south and columns to the east.\nRow 3: column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 4: column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 5: column 5 is wall, column 6 is wall, column 7 is open, column 8 is open, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 6: column 5 is open, column 6 is open, column 7 is open, column 8 is the agent, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 7: column 5 is open, column 6 is open, column 7 is wall, column 8 is open, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 8: column 5 is open, column 6 is open, column 7 is wall, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 9: column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze", "question": "The agent plans to move west, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "a2fe1be6084683d14f0c3a35c159b6f8befb4471761adc76bac711b28645617a"}228{"id": "cell_kind-000227", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 1. Rows increase to the south and columns to the east.\nRow -3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow -2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow -1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 0: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is open, column 5 is open\nRow 1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is the agent, column 2 is wall, column 3 is open, column 4 is open, column 5 is open\nRow 2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall\nRow 3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open, column 3 is open, column 4 is open, column 5 is wall\nRow 4: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is wall, column 4 is open, column 5 is open\nRow 5: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "8fbad8185312911522f5c2907b731f963453b1478f4cb6b793f3c3b489892e26"}229{"id": "goal_bearing-000228", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,7), goal at (7,1).\n#########\n#.#...#A#\n#.#.#.#.#\n#...#.#.#\n#.###.#.#\n#.#...#.#\n#.#.###.#\n#G#.....#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 7, "answer": "south-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "4f0439a1d7fc0505773173e480a6ad5eebc6e6e42091457f1acba420fe7fa417"}230{"id": "onward-000229", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (4,23); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.###.\n.#.#.\n.#A#.\n.#.#.\n.#.#.", "question": "Suppose the agent moves one cell south. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "6bbb658d29d62fb00ce2b071875478373b7b0021993e625f94bc88c08c99f3a4"}231{"id": "move_legal-000230", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 1, column 24. Rows increase to the south and columns to the east.\nRow -3: column 20 is outside the maze, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze, column 24 is outside the maze, column 25 is outside the maze, column 26 is outside the maze, column 27 is outside the maze, column 28 is outside the maze\nRow -2: column 20 is outside the maze, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze, column 24 is outside the maze, column 25 is outside the maze, column 26 is outside the maze, column 27 is outside the maze, column 28 is outside the maze\nRow -1: column 20 is outside the maze, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze, column 24 is outside the maze, column 25 is outside the maze, column 26 is outside the maze, column 27 is outside the maze, column 28 is outside the maze\nRow 0: column 20 is wall, column 21 is wall, column 22 is wall, column 23 is wall, column 24 is wall, column 25 is wall, column 26 is wall, column 27 is wall, column 28 is outside the maze\nRow 1: column 20 is wall, column 21 is open, column 22 is open, column 23 is open, column 24 is the agent, column 25 is open, column 26 is wall, column 27 is wall, column 28 is outside the maze\nRow 2: column 20 is wall, column 21 is wall, column 22 is wall, column 23 is open, column 24 is wall, column 25 is open, column 26 is wall, column 27 is wall, column 28 is outside the maze\nRow 3: column 20 is open, column 21 is open, column 22 is open, column 23 is open, column 24 is wall, column 25 is open, column 26 is wall, column 27 is wall, column 28 is outside the maze\nRow 4: column 20 is wall, column 21 is wall, column 22 is wall, column 23 is wall, column 24 is wall, column 25 is open, column 26 is wall, column 27 is wall, column 28 is outside the maze\nRow 5: column 20 is wall, column 21 is open, column 22 is open, column 23 is open, column 24 is wall, column 25 is open, column 26 is wall, column 27 is wall, column 28 is outside the maze", "question": "The agent wants to move west. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "1a2a845fcd56ea78be3bc7562e8a26ee41b906c65775441d29680e299482ff50"}232{"id": "cell_kind-000231", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (7,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n....X\n.#..X\n.#A.X\n....X\nXXXXX", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "9ff2903ff90261b69dafa396672fc2c06490b41474ae079bfa03fb155247b461"}233{"id": "onward-000232", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,26); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX\n#########\n#...A....\n#####.###\n..#...#.#\n#.#.###.#\n#.#......", "question": "Suppose the agent moves one cell east. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "9e8df70188a334c03795155c324b304793c7359142a8f72fdf91e375d1fe0e86"}234{"id": "snake_food-000233", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,8), travelling north, length 3.\n............\n............\n............\n............\n....*.......\n............\n........H...\n.......oo...\n............\n............\n............\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "7298e8beaa586e397b25fdc9439ba49e004b0bab10fdb33a9679f034ca042c74"}235{"id": "goal_bearing-000234", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (17,19), goal at (3,3).\n#####################\n#...............#...#\n#.####.##.#.#..##.#.#\n#..G#...#...........#\n#.#.###.#.#.#.#####.#\n#.....#.....#...#...#\n#.###.#####.#####.###\n#.#.......#.........#\n#.#.##.##.###.#.###.#\n#.......#...#.#.#...#\n#.##.##.###...#.###.#\n#.....#.#...#.#...#.#\n#.#.###.#.#######.#.#\n#.#.#...#.........#.#\n#.###.#######.#.#.#.#\n#.....#.......#.....#\n#.#.#####.###.#.###.#\n#.......#.#.#...#..A#\n#.#.###.#...###.#.#.#\n#...................#\n#####################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "d3a42e967ab1c24bb7b10255e84c09263c84fc1b2e8d0d9507f5e36d10e9ee44"}236{"id": "onward-000235", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 15, column 18. Rows increase to the south and columns to the east.\nRow 12: column 15 is wall, column 16 is wall, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is open\nRow 13: column 15 is open, column 16 is open, column 17 is open, column 18 is wall, column 19 is open, column 20 is open, column 21 is open\nRow 14: column 15 is open, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is open, column 20 is wall, column 21 is open\nRow 15: column 15 is open, column 16 is wall, column 17 is open, column 18 is the agent, column 19 is open, column 20 is wall, column 21 is open\nRow 16: column 15 is open, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is open\nRow 17: column 15 is open, column 16 is wall, column 17 is open, column 18 is open, column 19 is open, column 20 is wall, column 21 is open\nRow 18: column 15 is open, column 16 is wall, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is open", "question": "Suppose the agent moves one cell east. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "847cba343cbeaa6d00657b00196f140cff423f2305e951945d998b3c65bc5c92"}237{"id": "distance_band-000236", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (14,4), goal at (15,5).\n#...............\n###.###....###..\n#..#.#.#.#..#...\n...#........#...\n#..#...#...###..\n.......##.#.....\n.#...##.##.#....\n..#........#....\n#..##...##......\n...#..#..##.#..#\n.#..#......###..\n..#..###....###.\n.........#.###..\n..#......###...#\n#...A#.......#..\n.....G#.........", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "9f6563e5cf1b4430a235a06d0f67c7af5e373bd70a7cc05d8ea524286b8522d0"}238{"id": "move_legal-000237", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (31,24); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#...#...#\n###.###.#\n....#...#\n#####.###\n..#.A.#..\n#.#####.#\n#........\n#########\n#########", "question": "Is the cell one step south of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "0d5fa053cbaf61cb456a5bdbd92b77e620b84c667cfe2d4cdcb40bd9f663cf7f"}239{"id": "distance_band-000238", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 40x40. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (31,11), goal at (33,23).\n########################################\n#...............#...........#.#...#...##\n#.#######.#####.#.#########.#.#.#.#.#.##\n#.#.....#.#...#...#.......#...#.#...#.##\n#.#.###.#.#.#.###########.###.#.#####.##\n#...#...#...#...........#.....#.....#.##\n#####.#################.###########.#.##\n#...#.#.....#...........#...#.....#.#.##\n###.#.#.#####.#######.###.#.#.#.###.#.##\n#...#.#.#.....#...#...#...#.#.#.....#.##\n#.#.#.#.#.#####.#.#.###.###.#.#######.##\n#.#.#.#.#...#.#.#...#...#...#...#.....##\n#.###.#.###.#.#.#####.###.#####.#.######\n#.....#...#.#.#...#.....#.....#.#.....##\n#.#######.#.#.#.###.#######.###.#####.##\n#.#.......#.#...#...#.#.....#.......#.##\n#.###.###.#.#####.###.#.#####.#######.##\n#.#...#.#.#.....#...#.#.........#...#.##\n#.#.###.#.#####.###.#.###########.#.#.##\n#.#...#...#.......#.#...#.......#.#...##\n#.#.#.###.#.#####.#.#.#.#.#####.#.###.##\n#.#.#...#.#.#.#...#.#.#.#...#.#...#...##\n#.#####.#.#.#.#.###.###.###.#.#####.####\n#.#.....#.#.#.....#.#...#.#.#.....#...##\n#.#.#######.#######.#.#.#.#.###.#.###.##\n#.#.......#.......#.#.#.#.#...#.#.#...##\n#.#######.#######.#.#.#.#.###.###.#.####\n#.#.......#.....#.....#.....#...#...#.##\n#.#.#####.#.#.#################.#.###.##\n#.#.....#.#.#.#.............#...#.#...##\n#.#####.#.###.#.#####.#####.#.###.###.##\n#.......#..A#.#.#...#.#...#...#.#.....##\n###########.#.#.###.#.#.#.#####.#####.##\n#...#.....#...#...#...#G#.....#.#.....##\n###.#.###.###.###.###########.#.#.######\n#...#.#.....#.#.#.........#...#...#...##\n#.###.#.#####.#.#########.#.###.###.#.##\n#.....#.................#...#.......#.##\n########################################\n########################################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 4, "answer": "40 moves or more", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "86715a10b56e77afd7002d67dfcc8e892e372494f6732c033289fa95e5a1423b"}240{"id": "move_legal-000239", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (7,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#.#.....\n##.#.###.\n...#...#.\n.###.###.\n...#A#...\n##.###.#.\n.#.....#.\n.###.####\n.....#...", "question": "The agent wants to move north. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "4e4621969cd5387335a36ff7193cbcc6b5235f2ed6eec3acaf85d92e96e85258"}241{"id": "first_move-000240", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,4), goal at (7,7).\n############\n#.#.A.#...##\n#.#.#.#.#.##\n#.#.#...#.##\n#.#.#####.##\n#.#.#...#.##\n#.###.#.#.##\n#.....#G#.##\n#.#######.##\n#.........##\n############\n############", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "cd800567952907c4e6220f73276f725346a49a671e5be054c2d51b4f20b9b74f"}242{"id": "goal_bearing-000241", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,7), goal at (5,3).\n#########\n#.......#\n#.#####A#\n#.#...#.#\n#.#.#.#.#\n#.#G#.#.#\n#.###.#.#\n#.....#.#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 7, "answer": "south-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "66465ff4dcc8b463a0be3a9fed7c62915f738fdc7e8763914e216af683c46656"}243{"id": "distance_band-000242", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,13), goal at (3,9).\n################\n#.....#.#....A##\n#.###.#.#.###.##\n#.#...#.#G#...##\n###.#.#.###.#.##\n#...#.#...#.#.##\n#.###.###.#.####\n#.#.....#.#...##\n#.#######.###.##\n#.#...#...#...##\n#.#.#.#.###.#.##\n#.#.#.#.#...#.##\n#.#.#.#.#.###.##\n#...#.....#...##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 1, "answer": "5 to 9 moves", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "34613002254382869c831e96abe5dfd8543b0cc3a949a224358cc16b7b8c90e3"}244{"id": "stochastic-000243", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[15,5],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"12,2\":\"wall\",\"12,3\":\"wall\",\"12,4\":\"wall\",\"12,5\":\"wall\",\"12,6\":\"wall\",\"12,7\":\"open\",\"12,8\":\"wall\",\"13,2\":\"open\",\"13,3\":\"open\",\"13,4\":\"open\",\"13,5\":\"open\",\"13,6\":\"wall\",\"13,7\":\"open\",\"13,8\":\"wall\",\"14,2\":\"wall\",\"14,3\":\"wall\",\"14,4\":\"wall\",\"14,5\":\"open\",\"14,6\":\"wall\",\"14,7\":\"open\",\"14,8\":\"wall\",\"15,2\":\"open\",\"15,3\":\"open\",\"15,4\":\"wall\",\"15,5\":\"agent\",\"15,6\":\"wall\",\"15,7\":\"open\",\"15,8\":\"open\",\"16,2\":\"wall\",\"16,3\":\"open\",\"16,4\":\"wall\",\"16,5\":\"open\",\"16,6\":\"wall\",\"16,7\":\"wall\",\"16,8\":\"wall\",\"17,2\":\"open\",\"17,3\":\"open\",\"17,4\":\"wall\",\"17,5\":\"open\",\"17,6\":\"wall\",\"17,7\":\"open\",\"17,8\":\"open\",\"18,2\":\"wall\",\"18,3\":\"wall\",\"18,4\":\"wall\",\"18,5\":\"wall\",\"18,6\":\"wall\",\"18,7\":\"open\",\"18,8\":\"wall\"}}", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.7333333333333334, 0.26666666666666666], "deterministic": false, "fingerprint": "ab1a34509220cd8c0d6f0a5115fc86a03dc2f38ff9b0fbc8f1dcb93badf2b39e"}245{"id": "first_move-000244", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (14,1), goal at (3,25).\n############################\n#.............#...#.......##\n#.###.#.##.##.###...###.#.##\n#.......#...#...#.#.#...#G##\n#.#..######.###.#.#.#.######\n#.....#.......#.#.........##\n#.###.#.#.#####.#.#######.##\n#...#...#.......#.....#...##\n###.##########..#.#.#.#.####\n#.#.....#.............#...##\n#.#####.#.###.#.####.##.#.##\n#.....#...........#...#.#.##\n#.###.#######.###...#.#.#.##\n#.#.............#.#.......##\n#A..#######.#.#...#.###.#.##\n#.#.........#.....#.#.....##\n#.#.#.#.#.#...#.#.#.#####.##\n#.....#.....#.#...#.....#.##\n#.#.###.###...###.#.#.#...##\n#.#.....#...#.#...#...#.#.##\n#...#.#.#.###...###.###.#.##\n#.#...........#.....#...#.##\n#.#.#.####.##...#.###.#.####\n#.#.#.#.......#.#.........##\n#.#...#.#.#.###.#######.#.##\n#.........#...............##\n############################\n############################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "8fff9edf3a0a5ad839ed0f8c2c57e2e7f0c09bfdd73762ed0e8a49be4da75fff"}246{"id": "cell_kind-000245", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 8, column 15. Rows increase to the south and columns to the east.\nRow 4: column 11 is open, column 12 is open, column 13 is open, column 14 is open, column 15 is open, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall\nRow 5: column 11 is wall, column 12 is open, column 13 is wall, column 14 is wall, column 15 is open, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall\nRow 6: column 11 is wall, column 12 is open, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall\nRow 7: column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is open, column 16 is open, column 17 is wall, column 18 is wall, column 19 is wall\nRow 8: column 11 is wall, column 12 is wall, column 13 is wall, column 14 is open, column 15 is the agent, column 16 is open, column 17 is open, column 18 is wall, column 19 is wall\nRow 9: column 11 is wall, column 12 is wall, column 13 is wall, column 14 is open, column 15 is open, column 16 is open, column 17 is wall, column 18 is wall, column 19 is wall\nRow 10: column 11 is wall, column 12 is wall, column 13 is wall, column 14 is open, column 15 is wall, column 16 is open, column 17 is open, column 18 is wall, column 19 is wall\nRow 11: column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall\nRow 12: column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "7ef760ee27f2f6cb46d2fd2a8ca50563889ce7f72244db7e2d691371c487844b"}247{"id": "reachable_within-000246", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (15,3), goal at (28,31).\n.......#.#..#...#.##.#..#.......\n.#.#..##..#..#..#........##...##\n......#..#......##...#......#..#\n#.....#....#.#.......#...#.#....\n....#..##.##.##.##.......#..###.\n#...#.###.....#.......#.#.#.##..\n....##...#.....##..#.......##..#\n....#.#...#........##......#.#.#\n......#..#..#....#.#...##.#.....\n......##......##.....###..#.##.#\n.....#...##.##..................\n.#...#...#...#....##.....##...#.\n###.#.##..#.......##...####.#...\n.##....##...#.#...#......##.....\n.#......#.#....#...#.##...#..#..\n...A..##..#.....#..#.......##.##\n..#.......#.......#.............\n.#..#...##.###..##......####...#\n....#......####.##.#.##.#..#....\n......#...####....#.....#....#.#\n#....#....####.###..##.#.#.#..##\n..#.....########....###.......##\n.#.##.....#..#....##.#....#...##\n....#..####.#...........#..#..##\n.#.#####.#..###.#....#.......##.\n#.............##...#..#.....#...\n...##....#............#...#....#\n..#.#..#.......##.....##..#..#..\n.#....#.#..#.#..#..#.#..#......G\n#....#......##.......#.#.#.#...#\n##.....#..####.##.###...........\n#...##..#..###.#.......#.##....#", "question": "Can the agent reach the goal in at most 52 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "add2d69af66104c5493572a9c13359ba0c459d83f7d96d9d213967111b7ca3b8"}248{"id": "move_legal-000247", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (9,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.###.#\n#.#.#.#\n#.#.#.#\n#.#A#.#\n#.#.#.#\n#.#.#.#\n#.#.#.#", "question": "The agent wants to move south. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "17ceed73bebc99521952d2b3958fe974e21bea6c6bdb4760c1fb059d0271809b"}249{"id": "snake_food-000248", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,10), travelling south, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n.........oo...\n..........H...\n......*.......\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 7, "answer": "south-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "bfef2ce78afa0cb74dbadfa43a0c409462e59fa6f294915e7f6ff5dbb7af6e96"}250{"id": "cell_kind-000249", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (6,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n....#.#\n#####.#\n#...#..\n#.#A###\n#.#.#..\n#.#####\n#.#....", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "34077d3eca00e0ce8e81789e00d3e69671ca198a6c375d1ed7bde581468fa15f"}251{"id": "move_legal-000250", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 5, column 5. Rows increase to the south and columns to the east.\nRow 3: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open\nRow 4: column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 5: column 3 is open, column 4 is wall, column 5 is the agent, column 6 is wall, column 7 is open\nRow 6: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open\nRow 7: column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open", "question": "If the agent attempts one cell south (row+1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "a28e8b652a4eee483e5ebb1c6261029ee2889475ea20102434cbfb4494b9be30"}252{"id": "goal_bearing-000251", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,7), goal at (2,7).\n#........\n..#..#...\n......#G#\n...#.#.##\n..#......\n..#....A.\n#...#####\n##......#\n####..###", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "54e5f075d93e526dd6b86794ad917da61b3eaf49eac06c81b49b8a0387ca5c82"}253{"id": "distance_band-000252", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,20), goal at (29,21).\n################################\n#...#.....#...............#...##\n#.#.#.###.#.#######.#####.#.#.##\n#.#...#...#...........#.....#.##\n#.###.#.###.#.###.###.#.###.#.##\n#...#...#.....#...#.....#...#.##\n#.#.###.#######.####..#.###.#.##\n#.....#.........#.....#.......##\n#.###.#.#.#.#.#.#.###.##..###.##\n#.#.#.......#...#...#.......#.##\n#...###.#.#.#.#####.#..####.#.##\n#.#.#.....#...#...#.#...#...#.##\n#.#.#.#..####.#.#...#.#...#.####\n#.#.#.#.........#...A.#.#.....##\n#.#.#.#.####.####.#.###.##..#.##\n#.#.#...#.........#.....#...#.##\n#.#.###.#.###.#.#.#.###...#.#.##\n#.#...........#.#.#...#.#.#.#.##\n#.###.#.#####.#.#####.###.###.##\n#...#.#.#...#.#...............##\n###.###.#.###.#####.#######.#.##\n#...#...#.....#...#.......#.#.##\n#.###.###.#####.#.###.###.#...##\n#.#...#.........#...#.#.......##\n#.###.#.#######.###.#.#.#####.##\n#.#...#.....#...#.....#.......##\n#...#####.#.#.#.#.#.#####.#..###\n#.#.#.....#...#.#.......#.....##\n#.#.#.#.###.###.#######.#.###.##\n#...#.#..............G#.......##\n################################\n################################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 3, "answer": "20 to 39 moves", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "c26b92464b806b00f53783bae59018f3524372fb1a5ad4f3415b48abb5565e4c"}254{"id": "cell_kind-000253", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[4,17],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,13\":\"wall\",\"0,14\":\"wall\",\"0,15\":\"wall\",\"0,16\":\"wall\",\"0,17\":\"wall\",\"0,18\":\"wall\",\"0,19\":\"wall\",\"0,20\":\"wall\",\"0,21\":\"outside\",\"1,13\":\"open\",\"1,14\":\"open\",\"1,15\":\"open\",\"1,16\":\"open\",\"1,17\":\"open\",\"1,18\":\"open\",\"1,19\":\"open\",\"1,20\":\"wall\",\"1,21\":\"outside\",\"2,13\":\"wall\",\"2,14\":\"wall\",\"2,15\":\"wall\",\"2,16\":\"wall\",\"2,17\":\"wall\",\"2,18\":\"wall\",\"2,19\":\"open\",\"2,20\":\"wall\",\"2,21\":\"outside\",\"3,13\":\"open\",\"3,14\":\"open\",\"3,15\":\"open\",\"3,16\":\"wall\",\"3,17\":\"open\",\"3,18\":\"wall\",\"3,19\":\"open\",\"3,20\":\"wall\",\"3,21\":\"outside\",\"4,13\":\"wall\",\"4,14\":\"wall\",\"4,15\":\"open\",\"4,16\":\"wall\",\"4,17\":\"agent\",\"4,18\":\"wall\",\"4,19\":\"wall\",\"4,20\":\"wall\",\"4,21\":\"outside\",\"5,13\":\"open\",\"5,14\":\"open\",\"5,15\":\"open\",\"5,16\":\"wall\",\"5,17\":\"open\",\"5,18\":\"open\",\"5,19\":\"open\",\"5,20\":\"wall\",\"5,21\":\"outside\",\"6,13\":\"open\",\"6,14\":\"wall\",\"6,15\":\"open\",\"6,16\":\"wall\",\"6,17\":\"wall\",\"6,18\":\"wall\",\"6,19\":\"open\",\"6,20\":\"wall\",\"6,21\":\"outside\",\"7,13\":\"open\",\"7,14\":\"wall\",\"7,15\":\"open\",\"7,16\":\"wall\",\"7,17\":\"open\",\"7,18\":\"wall\",\"7,19\":\"open\",\"7,20\":\"wall\",\"7,21\":\"outside\",\"8,13\":\"wall\",\"8,14\":\"wall\",\"8,15\":\"open\",\"8,16\":\"wall\",\"8,17\":\"open\",\"8,18\":\"wall\",\"8,19\":\"open\",\"8,20\":\"wall\",\"8,21\":\"outside\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "314e48575f3bcfc14b5275132cc36512751ea0a3ad5201e94c5ff704472d9437"}255{"id": "first_move-000254", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,5), goal at (1,1).\n#########\n#G......#\n#######.#\n#.....#.#\n#.#####.#\n#.#.....#\n#.#####.#\n#....A..#\n#########", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "4fd1f5256bb1c356989686cd487befb35cdd4faa55c499805d4d1cd344d2eaff"}256{"id": "distance_band-000255", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 40x40. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (33,34), goal at (39,39).\n...#..#.#######.#...#...##..###....####.\n......#..#..###.#..#..#.#.#..#..........\n#....##.....###...#..##.#.#......###..#.\n.##.....#.......#.#....#....#.#...#..#..\n..#...##......#....#..##...###.....###..\n..........#.#...#...#.....##..#..#......\n...#.##.................#...#......##...\n#.......#......#....#.#...###..##....#..\n......#...#.#....###..#.....#.........##\n.##..#..#.##.......#...........##.......\n.....#.....#...........##....#....#.....\n.#.........##.#....##..#.........#..#...\n..###.#.......#..............#.#..#....#\n#.........#...##...#.#........##....#...\n##.#......#...##.....#..##...#..#.##....\n###....##.......#.....#......##........#\n###....##.#..#.....#...###....#......#..\n####..#.#.###...#....###...#.#.....#.#..\n######...##..##.....#......##...##....##\n####...##..............##..##..#......##\n###.........#....#...#.#....#.###.....##\n####..#............##....#..........#.#.\n#....##..#.....#...#..#........#.##.....\n.##.####........##..........#.#........#\n......#.##.....#.#......................\n.#....#.....#....###...#.........#..#...\n......##.##.#.#.##............#.........\n.......##........#.......#....#...#.##..\n....###...#......#....#....###..#..#....\n.....##....#....#.....#.####.......#.#.#\n.....##..#......##..............##...#.#\n........###...#..#......##.#.....#...###\n........##........##...#......#..#..####\n...#...#........#...#.#..#...###..A.#...\n.......#....##..#.......#.....#....##.#.\n..#....#...#...#......#.#...##...#.....#\n.#.##.......###.........##...#.#....#...\n#...#.........######.#..###.......#..#..\n.......#.#.#..##........####.###...#.#..\n#........#.#.......#....###..#......###G", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "1790bc8fdca41bae3df35a0c7221625f94762d7ac5d8f81c365b7287de46a156"}257{"id": "stochastic-000256", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 6, column 7. Rows increase to the south and columns to the east.\nRow 3: column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze\nRow 4: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze\nRow 5: column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze\nRow 6: column 4 is wall, column 5 is open, column 6 is wall, column 7 is the agent, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze\nRow 7: column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze\nRow 8: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze\nRow 9: column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze", "question": "The actuator executes the intended move with probability 0.90; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9333333333333333, 0.06666666666666665], "deterministic": false, "fingerprint": "3ed4e21ca3b8ed4f4e5a3d40b6cd27f7fbda1b99870bcce1dc4bc33a350677c5"}258{"id": "cell_kind-000257", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 9, column 21. Rows increase to the south and columns to the east.\nRow 6: column 18 is wall, column 19 is wall, column 20 is wall, column 21 is open, column 22 is wall, column 23 is wall, column 24 is wall\nRow 7: column 18 is open, column 19 is open, column 20 is open, column 21 is open, column 22 is wall, column 23 is open, column 24 is open\nRow 8: column 18 is wall, column 19 is wall, column 20 is wall, column 21 is wall, column 22 is wall, column 23 is open, column 24 is wall\nRow 9: column 18 is open, column 19 is open, column 20 is wall, column 21 is the agent, column 22 is open, column 23 is open, column 24 is open\nRow 10: column 18 is wall, column 19 is open, column 20 is wall, column 21 is wall, column 22 is wall, column 23 is wall, column 24 is wall\nRow 11: column 18 is open, column 19 is open, column 20 is wall, column 21 is open, column 22 is open, column 23 is open, column 24 is open\nRow 12: column 18 is wall, column 19 is wall, column 20 is wall, column 21 is open, column 22 is wall, column 23 is wall, column 24 is wall", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a99008eebb0dfe98ccdd9cbf1966c2cf48f16e25ab6d00342e7f7fc4ba544adb"}259{"id": "stochastic-000258", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 0, column 6. Rows increase to the south and columns to the east.\nRow -2: column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow -1: column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow 0: column 4 is wall, column 5 is open, column 6 is the agent, column 7 is open, column 8 is open\nRow 1: column 4 is open, column 5 is wall, column 6 is open, column 7 is open, column 8 is wall\nRow 2: column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is open", "question": "The actuator executes the intended move with probability 0.75; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.08333333333333337, 0.9166666666666666], "deterministic": false, "fingerprint": "761fb60b1f40a37d4b05d0a6f5bfb3c0def33a7c8ac2e79810f8432e15fa7318"}260{"id": "plan_survives-000259", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[9,8],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"6,5\":\"wall\",\"6,6\":\"wall\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"open\",\"6,10\":\"wall\",\"6,11\":\"wall\",\"7,5\":\"open\",\"7,6\":\"open\",\"7,7\":\"open\",\"7,8\":\"wall\",\"7,9\":\"open\",\"7,10\":\"wall\",\"7,11\":\"wall\",\"8,5\":\"open\",\"8,6\":\"wall\",\"8,7\":\"wall\",\"8,8\":\"wall\",\"8,9\":\"open\",\"8,10\":\"wall\",\"8,11\":\"wall\",\"9,5\":\"open\",\"9,6\":\"wall\",\"9,7\":\"open\",\"9,8\":\"agent\",\"9,9\":\"open\",\"9,10\":\"wall\",\"9,11\":\"wall\",\"10,5\":\"wall\",\"10,6\":\"wall\",\"10,7\":\"wall\",\"10,8\":\"wall\",\"10,9\":\"wall\",\"10,10\":\"wall\",\"10,11\":\"wall\",\"11,5\":\"wall\",\"11,6\":\"wall\",\"11,7\":\"wall\",\"11,8\":\"wall\",\"11,9\":\"wall\",\"11,10\":\"wall\",\"11,11\":\"wall\",\"12,5\":\"outside\",\"12,6\":\"outside\",\"12,7\":\"outside\",\"12,8\":\"outside\",\"12,9\":\"outside\",\"12,10\":\"outside\",\"12,11\":\"outside\"}}", "question": "The agent will attempt this plan in order: north, north, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "38d3cef4d5819542833323ba83b87b543a547c10b2e14554666504b240b96611"}261{"id": "open_count-000260", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,2],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-2,5\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"open\",\"0,1\":\"open\",\"0,2\":\"open\",\"0,3\":\"open\",\"0,4\":\"open\",\"0,5\":\"open\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"agent\",\"1,3\":\"open\",\"1,4\":\"wall\",\"1,5\":\"open\",\"2,-1\":\"outside\",\"2,0\":\"open\",\"2,1\":\"open\",\"2,2\":\"open\",\"2,3\":\"open\",\"2,4\":\"open\",\"2,5\":\"wall\",\"3,-1\":\"outside\",\"3,0\":\"open\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"wall\",\"3,4\":\"open\",\"3,5\":\"wall\",\"4,-1\":\"outside\",\"4,0\":\"open\",\"4,1\":\"open\",\"4,2\":\"open\",\"4,3\":\"open\",\"4,4\":\"open\",\"4,5\":\"open\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 4, "answer": "4", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "afc85d41f7044ac475b9fee251b4ec0c49a50188c97250d78e141cef126205c0"}262{"id": "move_legal-000261", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[23,25],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"21,23\":\"open\",\"21,24\":\"open\",\"21,25\":\"open\",\"21,26\":\"wall\",\"21,27\":\"wall\",\"22,23\":\"wall\",\"22,24\":\"wall\",\"22,25\":\"open\",\"22,26\":\"wall\",\"22,27\":\"wall\",\"23,23\":\"open\",\"23,24\":\"open\",\"23,25\":\"agent\",\"23,26\":\"wall\",\"23,27\":\"wall\",\"24,23\":\"wall\",\"24,24\":\"wall\",\"24,25\":\"open\",\"24,26\":\"wall\",\"24,27\":\"wall\",\"25,23\":\"open\",\"25,24\":\"open\",\"25,25\":\"open\",\"25,26\":\"wall\",\"25,27\":\"wall\"}}", "question": "If the agent attempts one cell east (col+1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "5acda9894af105c4fcb83b8f39b3d46eb617e66b7d8877500febed93a29ba5d3"}263{"id": "only_way-000262", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (19,19); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.#.#XX\n#.#.#XX\n#.#.#XX\n..#A#XX\n#####XX\nXXXXXXX\nXXXXXXX", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "f14c0c2c6b53dd9fed4ae3e80afc6bcfda058bd721c9af09e26afa054b6780b2"}264{"id": "plan_survives-000263", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,33],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,30\":\"wall\",\"0,31\":\"wall\",\"0,32\":\"wall\",\"0,33\":\"wall\",\"0,34\":\"wall\",\"0,35\":\"wall\",\"0,36\":\"outside\",\"1,30\":\"open\",\"1,31\":\"open\",\"1,32\":\"wall\",\"1,33\":\"open\",\"1,34\":\"wall\",\"1,35\":\"wall\",\"1,36\":\"outside\",\"2,30\":\"wall\",\"2,31\":\"open\",\"2,32\":\"wall\",\"2,33\":\"open\",\"2,34\":\"wall\",\"2,35\":\"wall\",\"2,36\":\"outside\",\"3,30\":\"open\",\"3,31\":\"open\",\"3,32\":\"open\",\"3,33\":\"agent\",\"3,34\":\"wall\",\"3,35\":\"wall\",\"3,36\":\"outside\",\"4,30\":\"wall\",\"4,31\":\"open\",\"4,32\":\"wall\",\"4,33\":\"open\",\"4,34\":\"wall\",\"4,35\":\"wall\",\"4,36\":\"outside\",\"5,30\":\"open\",\"5,31\":\"open\",\"5,32\":\"open\",\"5,33\":\"open\",\"5,34\":\"wall\",\"5,35\":\"wall\",\"5,36\":\"outside\",\"6,30\":\"wall\",\"6,31\":\"wall\",\"6,32\":\"wall\",\"6,33\":\"open\",\"6,34\":\"wall\",\"6,35\":\"wall\",\"6,36\":\"outside\"}}", "question": "The agent will attempt this plan in order: north, north, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "0a185dc61be3a7a808440d1d7fe8cfb9255df3e3dbed18931c012b55dc85cc1a"}265{"id": "move_two_step-000264", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (7,5); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.......X\n........X\n...#....X\n........X\n....A.##X\n..#.....X\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX", "question": "The agent plans to move west, then east. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "fa6975c4a3fd4387e9ebbc7820f0d160c60d6d87bbcf62ea6db5f76b2b523e1a"}266{"id": "cell_kind-000265", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[9,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"5,-3\":\"outside\",\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"6,-3\":\"outside\",\"6,-2\":\"outside\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"open\",\"7,-3\":\"outside\",\"7,-2\":\"outside\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"wall\",\"7,3\":\"open\",\"7,4\":\"wall\",\"7,5\":\"open\",\"8,-3\":\"outside\",\"8,-2\":\"outside\",\"8,-1\":\"outside\",\"8,0\":\"wall\",\"8,1\":\"open\",\"8,2\":\"wall\",\"8,3\":\"open\",\"8,4\":\"wall\",\"8,5\":\"open\",\"9,-3\":\"outside\",\"9,-2\":\"outside\",\"9,-1\":\"outside\",\"9,0\":\"wall\",\"9,1\":\"agent\",\"9,2\":\"wall\",\"9,3\":\"open\",\"9,4\":\"wall\",\"9,5\":\"open\",\"10,-3\":\"outside\",\"10,-2\":\"outside\",\"10,-1\":\"outside\",\"10,0\":\"wall\",\"10,1\":\"wall\",\"10,2\":\"wall\",\"10,3\":\"wall\",\"10,4\":\"wall\",\"10,5\":\"wall\",\"11,-3\":\"outside\",\"11,-2\":\"outside\",\"11,-1\":\"outside\",\"11,0\":\"wall\",\"11,1\":\"wall\",\"11,2\":\"wall\",\"11,3\":\"wall\",\"11,4\":\"wall\",\"11,5\":\"wall\",\"12,-3\":\"outside\",\"12,-2\":\"outside\",\"12,-1\":\"outside\",\"12,0\":\"outside\",\"12,1\":\"outside\",\"12,2\":\"outside\",\"12,3\":\"outside\",\"12,4\":\"outside\",\"12,5\":\"outside\",\"13,-3\":\"outside\",\"13,-2\":\"outside\",\"13,-1\":\"outside\",\"13,0\":\"outside\",\"13,1\":\"outside\",\"13,2\":\"outside\",\"13,3\":\"outside\",\"13,4\":\"outside\",\"13,5\":\"outside\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "8c38eec98ff25545b8f290e73b86f5a5cd5163e5d31ca124f2b92f73dbf0edff"}267{"id": "move_two_step-000266", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (23,25); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n###.##X\n#...##X\n#.####X\n#..A##X\n#.#.##X\n....##X\n######X", "question": "The agent plans to move north, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "a6899fb48aa19fa293e91bd9e45fa5a9996e9f746df9ebff1c1426e651b8c011"}268{"id": "cell_kind-000267", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[32,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"30,7\":\"wall\",\"30,8\":\"wall\",\"30,9\":\"open\",\"30,10\":\"open\",\"30,11\":\"open\",\"31,7\":\"open\",\"31,8\":\"open\",\"31,9\":\"open\",\"31,10\":\"open\",\"31,11\":\"open\",\"32,7\":\"wall\",\"32,8\":\"wall\",\"32,9\":\"agent\",\"32,10\":\"wall\",\"32,11\":\"open\",\"33,7\":\"open\",\"33,8\":\"open\",\"33,9\":\"open\",\"33,10\":\"wall\",\"33,11\":\"open\",\"34,7\":\"wall\",\"34,8\":\"wall\",\"34,9\":\"wall\",\"34,10\":\"wall\",\"34,11\":\"wall\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "718bda8a9073155859559a5961a37388f0f11ec4b2f2e408a5d776ed840aefab"}269{"id": "only_way-000268", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\n#########\n.#...#...\n##.#.#.##\n...#A#...\n.#####.#.\n.#...#.#.\n##.#.####\n.#.#.....", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "d71cc128fc9406f7b3c6c3a0f27b303e7c00c5aaadf2c47c4efa3d9d2cacedd4"}270{"id": "cell_kind-000269", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (9,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXX#.#.#\nXX#.#.#\nXX#.#.#\nXX#A#..\nXX#####\nXX#####\nXXXXXXX", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "7b760e70fc19f70bfebc6557d4a9e781a6e83be8c9535356ef97fa450baa9ef5"}271{"id": "open_count-000270", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 9, column 25. Rows increase to the south and columns to the east.\nRow 7: column 23 is open, column 24 is wall, column 25 is open, column 26 is wall, column 27 is wall\nRow 8: column 23 is open, column 24 is wall, column 25 is open, column 26 is wall, column 27 is wall\nRow 9: column 23 is open, column 24 is wall, column 25 is the agent, column 26 is wall, column 27 is wall\nRow 10: column 23 is open, column 24 is wall, column 25 is wall, column 26 is wall, column 27 is wall\nRow 11: column 23 is open, column 24 is open, column 25 is open, column 26 is wall, column 27 is wall", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 1, "answer": "1", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "82b4f493792b55360177a4dbc963f775d624aef1c665328b5922789168fb3020"}272{"id": "onward-000271", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (26,25); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.....\n.....\n..A..\n.....\nXXXXX", "question": "Suppose the agent moves one cell south. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "56863e32d438bb7b8252a7105ac876f7d38bc81d41b6e6d1354d7a2ccc6e0144"}273{"id": "snake_food-000272", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (9,8), travelling south, length 3.\n............\n............\n............\n............\n............\n............\n............\n........o...\n........o...\n..*.....H...\n............\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "431e9bef7dc5c2e93a9da6784bbf2ad79b771f9b25836079a0958dc522405c63"}274{"id": "reachable_within-000273", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,4), goal at (1,3).\n############\n#.#G....#.##\n#.#####.#.##\n#.....#...##\n#.#######.##\n#.....#...##\n#.###.#.####\n#.#.#...#.##\n#.#.#####.##\n#...A.....##\n############\n############", "question": "Can the agent reach the goal in at most 28 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "db916f20b48b6b5db21400dd7322efe73cf2d36eff6e13a6b01f40be33edab72"}275{"id": "stochastic-000274", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (8,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX#.##\nX#...\nX#A##\nX#...\nX#.#.", "question": "The actuator executes the intended move with probability 0.80; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.8666666666666667, 0.1333333333333333], "deterministic": false, "fingerprint": "9a88d16efcd102ee60c9df018758233fd239735f8b9e061af7ebf99cdf7f8af7"}276{"id": "stochastic-000275", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[7,2],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"wall\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"open\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"agent\",\"7,3\":\"open\",\"7,4\":\"wall\",\"7,5\":\"open\",\"8,-1\":\"outside\",\"8,0\":\"wall\",\"8,1\":\"wall\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"wall\",\"9,-1\":\"outside\",\"9,0\":\"wall\",\"9,1\":\"open\",\"9,2\":\"open\",\"9,3\":\"open\",\"9,4\":\"open\",\"9,5\":\"open\",\"10,-1\":\"outside\",\"10,0\":\"wall\",\"10,1\":\"open\",\"10,2\":\"wall\",\"10,3\":\"wall\",\"10,4\":\"wall\",\"10,5\":\"open\"}}", "question": "The actuator executes the intended move with probability 0.75; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.16666666666666663, 0.8333333333333334], "deterministic": false, "fingerprint": "cb4852035a366873954e3fb3c803d6170a035b9813c40619ef9a6444ae069ac4"}277{"id": "snake_food-000276", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,7), travelling east, length 3.\n..........\n..........\n.........*\n..........\n..........\n.....ooH..\n..........\n..........\n..........\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "646aa605e98e6259d058168fe15ef3fd5e6e781fdcefcb7255a4e022033e6996"}278{"id": "only_way-000277", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[25,5],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"22,2\":\"wall\",\"22,3\":\"open\",\"22,4\":\"wall\",\"22,5\":\"wall\",\"22,6\":\"wall\",\"22,7\":\"wall\",\"22,8\":\"wall\",\"23,2\":\"wall\",\"23,3\":\"open\",\"23,4\":\"open\",\"23,5\":\"open\",\"23,6\":\"open\",\"23,7\":\"open\",\"23,8\":\"wall\",\"24,2\":\"wall\",\"24,3\":\"wall\",\"24,4\":\"wall\",\"24,5\":\"wall\",\"24,6\":\"wall\",\"24,7\":\"open\",\"24,8\":\"wall\",\"25,2\":\"open\",\"25,3\":\"open\",\"25,4\":\"open\",\"25,5\":\"agent\",\"25,6\":\"wall\",\"25,7\":\"open\",\"25,8\":\"open\",\"26,2\":\"wall\",\"26,3\":\"wall\",\"26,4\":\"wall\",\"26,5\":\"wall\",\"26,6\":\"wall\",\"26,7\":\"wall\",\"26,8\":\"wall\",\"27,2\":\"wall\",\"27,3\":\"wall\",\"27,4\":\"wall\",\"27,5\":\"wall\",\"27,6\":\"wall\",\"27,7\":\"wall\",\"27,8\":\"wall\",\"28,2\":\"outside\",\"28,3\":\"outside\",\"28,4\":\"outside\",\"28,5\":\"outside\",\"28,6\":\"outside\",\"28,7\":\"outside\",\"28,8\":\"outside\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "6ce78bc111750ce7b60d98ca4dba5034c7cd2118901072caadbaeebef558accd"}279{"id": "stochastic-000278", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (3,20); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\n####.XXXX\n###..XXXX\n####.XXXX\n.###AXXXX\n.#...XXXX\n.....XXXX\n...##XXXX\n....#XXXX", "question": "The actuator executes the intended move with probability 0.80; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.8666666666666667, 0.1333333333333333], "deterministic": false, "fingerprint": "7373af4d528dbe087e8ecf98eab7d93bbad6aacf1ebe3ad17424501c46dff06d"}280{"id": "snake_food-000279", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,4), travelling west, length 3.\n..........\n..........\n..........\n..........\n..........\n..........\n.....o..*.\n....Ho....\n..........\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "88968c81aec4c0b8d8c17db87d753d3eb0a5abc6340c15e85ee305bd7fba642c"}281{"id": "only_way-000280", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[14,14],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"12,12\":\"open\",\"12,13\":\"wall\",\"12,14\":\"open\",\"12,15\":\"open\",\"12,16\":\"outside\",\"13,12\":\"open\",\"13,13\":\"open\",\"13,14\":\"open\",\"13,15\":\"open\",\"13,16\":\"outside\",\"14,12\":\"wall\",\"14,13\":\"wall\",\"14,14\":\"agent\",\"14,15\":\"open\",\"14,16\":\"outside\",\"15,12\":\"wall\",\"15,13\":\"wall\",\"15,14\":\"open\",\"15,15\":\"wall\",\"15,16\":\"outside\",\"16,12\":\"outside\",\"16,13\":\"outside\",\"16,14\":\"outside\",\"16,15\":\"outside\",\"16,16\":\"outside\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 5, "answer": "more than one of them", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "91f51981fb11f1790a34637943620ec6582f639c1097b238392be3c79d79d922"}282{"id": "distance_band-000281", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (4,21), goal at (3,23).\n###..###..###....#..#...\n####..#.#.#.............\n##..........#........#.#\n#....#.##..#...#..#.#.#G\n...#.###...#.##....##A..\n.......##.....#..#..#...\n##..........#.....#....#\n#........#......#.#...#.\n.##.#.........#..###....\n....#..#..##...##..#.#..\n..................####..\n....#..#....###........#\n.......#.###........#...\n.##..###..#............#\n....#...#......#...#....\n........#.#.....#.#.....\n....#............#...#.#\n..#..###.#......#...#...\n.....#..#..#..#....#..#.\n#..#.....#..#.#...#.....\n.#....#.#.#..........##.\n.....#.....#..##.......#\n#....#.#....#####.....##\n........###...##.......#", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "43d9376f4f8d6ac2f3df3aa46984522863089a9028ff21750f59ac38e361f5ce"}283{"id": "blocked_pair-000282", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (13,11); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.....#.##\n####.#.##\n...#...##\n.#.###.##\n....A..##\n#########\n#########\nXXXXXXXXX\nXXXXXXXXX", "question": "Are BOTH the cell one step north and the cell one step south blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "45c16ad51dc899e0bb90f23e0bc64eaf0206d955e46ae5ff637f85067bfe74a6"}284{"id": "snake_food-000283", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,7), travelling east, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n.*...ooH......\n..............\n..............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "dba480ec0902bd7f6d17589364a502636d5bd0f4d04d493221826dfa1f046eff"}285{"id": "move_two_step-000284", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (26,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#..##.##.\n#........\n#.#.#.#..\n...#####.\n#.#.A.###\n#.....###\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX", "question": "The agent plans to move south, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "bb769fb2e00fffe16c0551c66c0d2004e4176ca679a7a8e19304959460b57dfe"}286{"id": "onward-000285", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 11, column 6. Rows increase to the south and columns to the east.\nRow 8: column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is open, column 9 is wall\nRow 9: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is open, column 9 is open\nRow 10: column 3 is wall, column 4 is wall, column 5 is open, column 6 is open, column 7 is wall, column 8 is wall, column 9 is open\nRow 11: column 3 is wall, column 4 is open, column 5 is open, column 6 is the agent, column 7 is open, column 8 is wall, column 9 is wall\nRow 12: column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is open, column 9 is wall\nRow 13: column 3 is wall, column 4 is open, column 5 is wall, column 6 is open, column 7 is wall, column 8 is open, column 9 is wall\nRow 14: column 3 is wall, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is wall", "question": "Suppose the agent moves one cell north. From that new cell, would a further step east be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "7459bdf072bdc04a912f02beb5a18ae071525a27b3c45962f040fa34b7719877"}287{"id": "blocked_pair-000286", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 30, column 7. Rows increase to the south and columns to the east.\nRow 28: column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open\nRow 29: column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open\nRow 30: column 5 is wall, column 6 is wall, column 7 is the agent, column 8 is wall, column 9 is wall\nRow 31: column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open\nRow 32: column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open", "question": "Are BOTH the cell one step north and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "18c03d5ce5c3583e397e0cabede052f842c1ad780802bcfa3b98850291a56bc8"}288{"id": "move_legal-000287", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 7, column 5. Rows increase to the south and columns to the east.\nRow 5: column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is open\nRow 6: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open\nRow 7: column 3 is open, column 4 is open, column 5 is the agent, column 6 is wall, column 7 is open\nRow 8: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 9: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze", "question": "If the agent attempts one cell north (row-1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "c7337ed9ef4986c237e8185a4cc83c3c345c1e9165a60b1cddb3d861d73a3932"}289{"id": "goal_bearing-000288", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,3), goal at (5,3).\n#########\n#.......#\n#.#####.#\n#.#A..#.#\n#.###.#.#\n#..G#.#.#\n#####.#.#\n#.......#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 1, "answer": "south", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "e61c8893934565334fc517c94a6b6af55b870f9e5215ac68f51c033bee1655b6"}290{"id": "blocked_pair-000289", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,1\":\"open\",\"1,2\":\"wall\",\"1,3\":\"open\",\"1,4\":\"wall\",\"1,5\":\"open\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"open\",\"2,4\":\"wall\",\"2,5\":\"open\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"agent\",\"3,4\":\"wall\",\"3,5\":\"open\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"open\",\"5,5\":\"open\"}}", "question": "Are BOTH the cell one step north and the cell one step west blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "c5babc5dd97e993ae14e84c7fe653856f0fa4377d09dc4566d2f7ceacb32e9fb"}291{"id": "blocked_pair-000290", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,-3\":\"outside\",\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"0,-3\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"1,-3\":\"outside\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"wall\",\"1,5\":\"open\",\"2,-3\":\"outside\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"open\",\"2,4\":\"wall\",\"2,5\":\"open\",\"3,-3\":\"outside\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"agent\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"open\",\"4,-3\":\"outside\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"wall\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\",\"5,-3\":\"outside\",\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"6,-3\":\"outside\",\"6,-2\":\"outside\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"open\",\"7,-3\":\"outside\",\"7,-2\":\"outside\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"wall\",\"7,3\":\"open\",\"7,4\":\"wall\",\"7,5\":\"open\"}}", "question": "Are BOTH the cell one step north and the cell one step west blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "43898e875e1f516af41101bd036496f35d307efaa3245656af2f0556627eda58"}292{"id": "cell_kind-000291", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 26, column 26. Rows increase to the south and columns to the east.\nRow 22: column 22 is open, column 23 is open, column 24 is open, column 25 is open, column 26 is open, column 27 is open, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze\nRow 23: column 22 is open, column 23 is open, column 24 is open, column 25 is wall, column 26 is open, column 27 is open, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze\nRow 24: column 22 is open, column 23 is open, column 24 is wall, column 25 is open, column 26 is open, column 27 is open, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze\nRow 25: column 22 is open, column 23 is open, column 24 is open, column 25 is wall, column 26 is open, column 27 is open, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze\nRow 26: column 22 is wall, column 23 is open, column 24 is open, column 25 is open, column 26 is the agent, column 27 is wall, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze\nRow 27: column 22 is open, column 23 is open, column 24 is open, column 25 is wall, column 26 is open, column 27 is open, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze\nRow 28: column 22 is outside the maze, column 23 is outside the maze, column 24 is outside the maze, column 25 is outside the maze, column 26 is outside the maze, column 27 is outside the maze, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze\nRow 29: column 22 is outside the maze, column 23 is outside the maze, column 24 is outside the maze, column 25 is outside the maze, column 26 is outside the maze, column 27 is outside the maze, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze\nRow 30: column 22 is outside the maze, column 23 is outside the maze, column 24 is outside the maze, column 25 is outside the maze, column 26 is outside the maze, column 27 is outside the maze, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "5c1ee23b047d7bc5c719e418384a0073fda2b4c95928f96bb2afa6ae6ff4aef9"}293{"id": "plan_survives-000292", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[5,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"open\",\"2,8\":\"wall\",\"2,9\":\"wall\",\"2,10\":\"wall\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"3,8\":\"open\",\"3,9\":\"open\",\"3,10\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"wall\",\"4,8\":\"wall\",\"4,9\":\"open\",\"4,10\":\"wall\",\"5,4\":\"open\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"agent\",\"5,8\":\"open\",\"5,9\":\"open\",\"5,10\":\"wall\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"wall\",\"6,10\":\"wall\",\"7,4\":\"open\",\"7,5\":\"open\",\"7,6\":\"wall\",\"7,7\":\"open\",\"7,8\":\"wall\",\"7,9\":\"open\",\"7,10\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"open\",\"8,6\":\"wall\",\"8,7\":\"wall\",\"8,8\":\"wall\",\"8,9\":\"open\",\"8,10\":\"wall\"}}", "question": "The agent will attempt this plan in order: east, north, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "75b30d286ad24004483cfeb6a51e1d14a3a12d031bf629be830b17f00a94d636"}294{"id": "goal_bearing-000293", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (22,19), goal at (3,19).\n############################\n#.....#.....#...........#.##\n#.###.###.#.#.###.#.#.#.#.##\n#...#.....#.....#.#G#.#.#.##\n#.#.###.#.#####.#.###.#.#.##\n#...#.#.#.....#.......#.#.##\n#.#.#.#.##.##...####.##...##\n#.#.#.........#...#.....#.##\n#.#.#.##.##.#####.#.#####.##\n#.#.#.....#.#.........#...##\n#.#.###.#.#.#.#.#####.#.#.##\n#.......#.....#...#.......##\n#.#.#.###.#######.#######.##\n#.#.#.#...#...#...#...#...##\n#.#.#.#.#.#.###.###.#.######\n#.#.....#.#...#...........##\n#.#.#####.###.#####.#####.##\n#.#...#.......#...#...#...##\n#####.##.######.#.#####.#.##\n#.....#.........#.....#.#.##\n#.##.##.#.#.#########.#.#.##\n#.#.....#...#.#.....#...#.##\n#.#.#.#######.#.#.#A#.###.##\n#...#.#.....#...#.#.#...#.##\n###.#.#.#.###.###.###.#.#.##\n#.......#.............#...##\n############################\n############################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "71ed71ba62faa5faa207660b24c0ee5128a47fcd7f1ab816e9efa5d0c1f60420"}295{"id": "snake_safe-000294", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,9), travelling east, length 3.\n........*.....\n..............\n..............\n..............\n..............\n..............\n..............\n.......ooH....\n..............\n..............\n..............\n..............\n..............\n..............", "question": "If the snake moves east on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "806d7fecfcaddd883f5bef9ec7db0531fc109ec57406f2bd967e031cb7f8bbd5"}296{"id": "stochastic-000295", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[17,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"15,1\":\"open\",\"15,2\":\"open\",\"15,3\":\"open\",\"15,4\":\"wall\",\"15,5\":\"open\",\"16,1\":\"wall\",\"16,2\":\"wall\",\"16,3\":\"open\",\"16,4\":\"wall\",\"16,5\":\"open\",\"17,1\":\"open\",\"17,2\":\"open\",\"17,3\":\"agent\",\"17,4\":\"wall\",\"17,5\":\"open\",\"18,1\":\"wall\",\"18,2\":\"wall\",\"18,3\":\"wall\",\"18,4\":\"wall\",\"18,5\":\"open\",\"19,1\":\"open\",\"19,2\":\"open\",\"19,3\":\"open\",\"19,4\":\"wall\",\"19,5\":\"open\"}}", "question": "The actuator executes the intended move with probability 0.80; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.8666666666666667, 0.1333333333333333], "deterministic": false, "fingerprint": "aa1536774a08fea294205ff06c6d314516236ac8aac51b69a58dc8ee154abcb4"}297{"id": "move_two_step-000296", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 10, column 7. Rows increase to the south and columns to the east.\nRow 6: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall\nRow 7: column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is open, column 10 is open, column 11 is open\nRow 8: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall\nRow 9: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is open, column 11 is open\nRow 10: column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall, column 7 is the agent, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open\nRow 11: column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open\nRow 12: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall\nRow 13: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open, column 10 is open, column 11 is open\nRow 14: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall", "question": "The agent plans to move east, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "48778c36eaae2ab36f3e4b915bb2c69dd91ccab4e8a137ab671af7708d86be78"}298{"id": "first_move-000297", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (17,18), goal at (3,11).\n####################################\n#.................................##\n#.###############################.##\n#.#.......#G......................##\n#.#.#####.##########################\n#.#.#...#.........................##\n#.#.#.###################.#######.##\n#.#.#...................#.#.....#.##\n#.#.###################.#.#####.#.##\n#.#.....................#.#.....#.##\n#.#####################.#.#.###.#.##\n#.#.....#.............#.#.#.#.#.#.##\n#.#.###.#.###.#########.#.#.#.#.#.##\n#.#.#.#.#...#...........#.#.#...#.##\n#.#.#.#.#################.#.#####.##\n#.#.#.....................#.......##\n#.#.################################\n#.#...............A...............##\n#.#.#############################.##\n#.#.#...........................#.##\n#.#.#.#########################.#.##\n#.#.#.#.......................#.#.##\n#.#.#.#.#######################.#.##\n#.#...#.........................#.##\n#.#############################.#.##\n#.............................#.#.##\n#############################.#.#.##\n#...............#...........#.#.#.##\n#.#############.#.#########.#.#.#.##\n#.#...........#.#.........#.#.#.#.##\n#.#.#.#########.###########.#.#.#.##\n#.#.#.......................#.#.#.##\n#.###########################.###.##\n#.................................##\n####################################\n####################################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "9b964c87ae4fef6c12cbe40c1ef6b306826ad16f9d6230e16d55c0ce9a5873fa"}299{"id": "cell_kind-000298", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"1,1\":\"open\",\"1,2\":\"wall\",\"1,3\":\"agent\",\"1,4\":\"open\",\"1,5\":\"open\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"open\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3a2278b51e904d477e2689659eabefe7080580ce6ccb443b2c0fb22025d80d9c"}300{"id": "goal_bearing-000299", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (17,1), goal at (19,19).\n#####################\n#.......#...........#\n#.##.##...#.#######.#\n#.#...#.#.#.....#...#\n###.#.#...#####.#.###\n#...#...........#...#\n#.####.##.#.#.#.###.#\n#...#.........#...#.#\n#.#.#.#...#.###.#.#.#\n#...#...............#\n###.###.#####.#.###.#\n#.....#.#.....#...#.#\n#.#####.#.###.###.#.#\n#.#.....#.#.#.#...#.#\n#.#.#.#.#.#.#...###.#\n#.#...#.#.....#.#...#\n#.###.#.#.###.#.###.#\n#A#...#.#.....#.#...#\n#.#.#.#.#####.#.#.#.#\n#...........#.....#G#\n#####################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 6, "answer": "south-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "39cf96b03bc068231af84fa3779253be5111f775101042db3bfba7893216c6a1"}301{"id": "stochastic-000300", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 0, column 33. Rows increase to the south and columns to the east.\nRow -3: column 30 is outside the maze, column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze, column 36 is outside the maze\nRow -2: column 30 is outside the maze, column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze, column 36 is outside the maze\nRow -1: column 30 is outside the maze, column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze, column 36 is outside the maze\nRow 0: column 30 is open, column 31 is open, column 32 is wall, column 33 is the agent, column 34 is open, column 35 is open, column 36 is outside the maze\nRow 1: column 30 is open, column 31 is wall, column 32 is open, column 33 is open, column 34 is wall, column 35 is open, column 36 is outside the maze\nRow 2: column 30 is open, column 31 is open, column 32 is open, column 33 is open, column 34 is open, column 35 is open, column 36 is outside the maze\nRow 3: column 30 is open, column 31 is wall, column 32 is open, column 33 is open, column 34 is open, column 35 is open, column 36 is outside the maze", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.7333333333333334, 0.26666666666666666], "deterministic": false, "fingerprint": "02526ee78726c3d77f5e5e0978cfac851a42f6b66d135b661d5329e2a84c45cd"}302{"id": "first_move-000301", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (15,16), goal at (1,5).\n#####################\n#....G#.............#\n#.#####.###########.#\n#.................#.#\n###################.#\n#...................#\n#.#################.#\n#.#.................#\n#.###################\n#...................#\n#.#################.#\n#.#.........#.....#.#\n#.#.#######.#.###.#.#\n#.#.#.....#.#...#.#.#\n#.#.#.#####.#####.#.#\n#.#.#...........A.#.#\n#.#.#############.#.#\n#...#...........#.#.#\n###.###########.###.#\n#.............#.....#\n#####################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "e66a96839c226d4250e7295f6f55a2e03f1f08ee032039859e7ba54e48c63622"}303{"id": "onward-000302", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (7,18); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n..#.#\n###.#\n#.A.#\n#####\n....#", "question": "Suppose the agent moves one cell east. From that new cell, would a further step east be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "a9376f243d6c46e8d3a15c2b69f32e926321afe8f8fefdd54dc95a0d11c65bf7"}304{"id": "move_legal-000303", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[6,11],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"2,7\":\"open\",\"2,8\":\"wall\",\"2,9\":\"open\",\"2,10\":\"wall\",\"2,11\":\"open\",\"2,12\":\"wall\",\"2,13\":\"open\",\"2,14\":\"wall\",\"2,15\":\"wall\",\"3,7\":\"open\",\"3,8\":\"wall\",\"3,9\":\"open\",\"3,10\":\"open\",\"3,11\":\"open\",\"3,12\":\"open\",\"3,13\":\"open\",\"3,14\":\"wall\",\"3,15\":\"wall\",\"4,7\":\"open\",\"4,8\":\"wall\",\"4,9\":\"open\",\"4,10\":\"wall\",\"4,11\":\"wall\",\"4,12\":\"wall\",\"4,13\":\"open\",\"4,14\":\"wall\",\"4,15\":\"wall\",\"5,7\":\"open\",\"5,8\":\"wall\",\"5,9\":\"open\",\"5,10\":\"wall\",\"5,11\":\"open\",\"5,12\":\"wall\",\"5,13\":\"open\",\"5,14\":\"wall\",\"5,15\":\"wall\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"open\",\"6,10\":\"wall\",\"6,11\":\"agent\",\"6,12\":\"wall\",\"6,13\":\"open\",\"6,14\":\"wall\",\"6,15\":\"wall\",\"7,7\":\"open\",\"7,8\":\"wall\",\"7,9\":\"open\",\"7,10\":\"wall\",\"7,11\":\"open\",\"7,12\":\"open\",\"7,13\":\"open\",\"7,14\":\"wall\",\"7,15\":\"wall\",\"8,7\":\"open\",\"8,8\":\"open\",\"8,9\":\"open\",\"8,10\":\"wall\",\"8,11\":\"open\",\"8,12\":\"wall\",\"8,13\":\"open\",\"8,14\":\"wall\",\"8,15\":\"wall\",\"9,7\":\"open\",\"9,8\":\"wall\",\"9,9\":\"open\",\"9,10\":\"wall\",\"9,11\":\"open\",\"9,12\":\"open\",\"9,13\":\"open\",\"9,14\":\"wall\",\"9,15\":\"wall\",\"10,7\":\"open\",\"10,8\":\"wall\",\"10,9\":\"open\",\"10,10\":\"wall\",\"10,11\":\"wall\",\"10,12\":\"open\",\"10,13\":\"wall\",\"10,14\":\"wall\",\"10,15\":\"wall\"}}", "question": "Is the cell one step north of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "441271683d9f678b3f0ab5c929fdcad59aaba52dd7d52a1ef9201e3fa43c1006"}305{"id": "first_move-000304", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,1), goal at (5,3).\n#########\n#.......#\n#A#####.#\n#.#...#.#\n#.#.#.#.#\n#.#G#.#.#\n#.###.#.#\n#.....#.#\n#########", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "80da2b1a6ed0e68e19245072f4897dc0a19fc135ede96114f35d450c4e1ca387"}306{"id": "blocked_pair-000305", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,6\":\"wall\",\"0,7\":\"wall\",\"0,8\":\"wall\",\"0,9\":\"wall\",\"0,10\":\"wall\",\"0,11\":\"wall\",\"0,12\":\"outside\",\"1,6\":\"open\",\"1,7\":\"open\",\"1,8\":\"open\",\"1,9\":\"open\",\"1,10\":\"wall\",\"1,11\":\"wall\",\"1,12\":\"outside\",\"2,6\":\"wall\",\"2,7\":\"wall\",\"2,8\":\"wall\",\"2,9\":\"open\",\"2,10\":\"wall\",\"2,11\":\"wall\",\"2,12\":\"outside\",\"3,6\":\"open\",\"3,7\":\"open\",\"3,8\":\"wall\",\"3,9\":\"agent\",\"3,10\":\"wall\",\"3,11\":\"wall\",\"3,12\":\"outside\",\"4,6\":\"wall\",\"4,7\":\"open\",\"4,8\":\"wall\",\"4,9\":\"open\",\"4,10\":\"wall\",\"4,11\":\"wall\",\"4,12\":\"outside\",\"5,6\":\"open\",\"5,7\":\"open\",\"5,8\":\"wall\",\"5,9\":\"open\",\"5,10\":\"wall\",\"5,11\":\"wall\",\"5,12\":\"outside\",\"6,6\":\"wall\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"open\",\"6,10\":\"wall\",\"6,11\":\"wall\",\"6,12\":\"outside\"}}", "question": "Are BOTH the cell one step south and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "4fa67aca8e2e39c170508b911bab4ff085fde4eb8aff1680b17c7dcbd4ec2a57"}307{"id": "first_move-000306", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,0), goal at (5,0).\n##.#..###\n.....#.##\n.##..#..#\n.#.....##\n.#...####\nG#....###\n.#....###\nA..#..###\n..#..####", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "6c2a91ff979ceb2a085e8baa54ab9024c6bff41d62d0fc2a0c084b0759faf191"}308{"id": "snake_safe-000307", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,1), travelling west, length 3.\n............\n............\n.....*......\n............\n............\n............\n............\n............\n.Hoo........\n............\n............\n............", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "2ffd5bff993fcd2ddd9d26ea1f2c38fcc2cc0ee36b2be215bce29aa4c49223a7"}309{"id": "goal_bearing-000308", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,12), goal at (13,9).\n################\n#.#...........##\n#.#.#.#######.##\n#.#.#.#.......##\n#.#.#.##########\n#.#.#.........##\n#.#.#########.##\n#.#.#.......#.##\n#.#.#.#####.#.##\n#.#.#.#...#.#.##\n#.#.#.#.###.#.##\n#.#.#.#.....#.##\n#.#.#.#.#####.##\n#.....#..G#.A.##\n################\n################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "4957f861acc41b6d3e1473746af4c1566dab356f796dc3d2142d12cd32c96278"}310{"id": "move_legal-000309", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 16, column 7. Rows increase to the south and columns to the east.\nRow 12: column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall\nRow 13: column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open\nRow 14: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open\nRow 15: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open\nRow 16: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is the agent, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open\nRow 17: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open\nRow 18: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open\nRow 19: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open\nRow 20: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall", "question": "Is the cell one step west of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "4a290c4a05e1a925190bf8a14fc9ad3e78cc74b32b01c49e21351d8f6050ad37"}311{"id": "only_way-000310", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 17, column 5. Rows increase to the south and columns to the east.\nRow 13: column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is open, column 9 is open\nRow 14: column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall\nRow 15: column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is open, column 9 is open\nRow 16: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is wall\nRow 17: column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is the agent, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open\nRow 18: column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open\nRow 19: column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open\nRow 20: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall\nRow 21: column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "0d3facd9bd28f246fbddba2c4bfb29de28eb7d74cf23340d8c529e9b5e72124d"}312{"id": "cell_kind-000311", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[17,15],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"13,11\":\"open\",\"13,12\":\"open\",\"13,13\":\"open\",\"13,14\":\"open\",\"13,15\":\"open\",\"13,16\":\"open\",\"13,17\":\"open\",\"13,18\":\"open\",\"13,19\":\"open\",\"14,11\":\"open\",\"14,12\":\"wall\",\"14,13\":\"wall\",\"14,14\":\"wall\",\"14,15\":\"wall\",\"14,16\":\"wall\",\"14,17\":\"wall\",\"14,18\":\"wall\",\"14,19\":\"wall\",\"15,11\":\"open\",\"15,12\":\"wall\",\"15,13\":\"open\",\"15,14\":\"open\",\"15,15\":\"open\",\"15,16\":\"open\",\"15,17\":\"open\",\"15,18\":\"open\",\"15,19\":\"open\",\"16,11\":\"open\",\"16,12\":\"wall\",\"16,13\":\"open\",\"16,14\":\"wall\",\"16,15\":\"wall\",\"16,16\":\"wall\",\"16,17\":\"wall\",\"16,18\":\"wall\",\"16,19\":\"wall\",\"17,11\":\"open\",\"17,12\":\"wall\",\"17,13\":\"open\",\"17,14\":\"wall\",\"17,15\":\"agent\",\"17,16\":\"open\",\"17,17\":\"open\",\"17,18\":\"open\",\"17,19\":\"open\",\"18,11\":\"open\",\"18,12\":\"wall\",\"18,13\":\"open\",\"18,14\":\"wall\",\"18,15\":\"open\",\"18,16\":\"wall\",\"18,17\":\"wall\",\"18,18\":\"wall\",\"18,19\":\"open\",\"19,11\":\"open\",\"19,12\":\"wall\",\"19,13\":\"open\",\"19,14\":\"wall\",\"19,15\":\"open\",\"19,16\":\"open\",\"19,17\":\"open\",\"19,18\":\"wall\",\"19,19\":\"open\",\"20,11\":\"open\",\"20,12\":\"wall\",\"20,13\":\"open\",\"20,14\":\"wall\",\"20,15\":\"wall\",\"20,16\":\"wall\",\"20,17\":\"wall\",\"20,18\":\"wall\",\"20,19\":\"open\",\"21,11\":\"open\",\"21,12\":\"wall\",\"21,13\":\"open\",\"21,14\":\"open\",\"21,15\":\"open\",\"21,16\":\"open\",\"21,17\":\"open\",\"21,18\":\"open\",\"21,19\":\"open\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "cd91d2dd3c4846e26a0ed25b76a46c16a50a2f3a3b81cabfa0b95a6b93dae4d4"}313{"id": "onward-000312", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 16, column 17. Rows increase to the south and columns to the east.\nRow 13: column 14 is wall, column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is wall\nRow 14: column 14 is wall, column 15 is open, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall\nRow 15: column 14 is wall, column 15 is open, column 16 is wall, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall\nRow 16: column 14 is wall, column 15 is open, column 16 is wall, column 17 is the agent, column 18 is wall, column 19 is open, column 20 is wall\nRow 17: column 14 is wall, column 15 is open, column 16 is open, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall\nRow 18: column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is open, column 20 is wall\nRow 19: column 14 is open, column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is open", "question": "Suppose the agent moves one cell south. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "8510469f9d890e6df7a21a4634c544dae72c976fb67033e286103d8c56b76204"}314{"id": "stochastic-000313", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (13,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.###.#\n#.#.#.#\n#.#.#.#\n#.#A#.#\n#.#.#.#\n#.#.#.#\n#.#.#.#", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.2666666666666667, 0.7333333333333333], "deterministic": false, "fingerprint": "ae5863022f547bbdfcb5a4e545645bb75818be3e5ba1c82a301512b7e0659fb5"}315{"id": "stochastic-000314", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 7, column 7. Rows increase to the south and columns to the east.\nRow 3: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 4: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 5: column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 6: column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 7: column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is the agent, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 8: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 9: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 10: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 11: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze", "question": "The actuator executes the intended move with probability 0.85; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.95, 0.05000000000000001], "deterministic": false, "fingerprint": "bfdb84978737c26f4061405d25d056b515cf15f45f7589e0acf231f099fdc934"}316{"id": "only_way-000315", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[15,5],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"12,2\":\"wall\",\"12,3\":\"open\",\"12,4\":\"wall\",\"12,5\":\"wall\",\"12,6\":\"wall\",\"12,7\":\"wall\",\"12,8\":\"wall\",\"13,2\":\"wall\",\"13,3\":\"open\",\"13,4\":\"wall\",\"13,5\":\"open\",\"13,6\":\"open\",\"13,7\":\"open\",\"13,8\":\"wall\",\"14,2\":\"wall\",\"14,3\":\"open\",\"14,4\":\"wall\",\"14,5\":\"wall\",\"14,6\":\"wall\",\"14,7\":\"open\",\"14,8\":\"wall\",\"15,2\":\"wall\",\"15,3\":\"open\",\"15,4\":\"open\",\"15,5\":\"agent\",\"15,6\":\"wall\",\"15,7\":\"open\",\"15,8\":\"wall\",\"16,2\":\"wall\",\"16,3\":\"wall\",\"16,4\":\"wall\",\"16,5\":\"wall\",\"16,6\":\"wall\",\"16,7\":\"open\",\"16,8\":\"wall\",\"17,2\":\"open\",\"17,3\":\"open\",\"17,4\":\"wall\",\"17,5\":\"open\",\"17,6\":\"open\",\"17,7\":\"open\",\"17,8\":\"wall\",\"18,2\":\"wall\",\"18,3\":\"open\",\"18,4\":\"wall\",\"18,5\":\"open\",\"18,6\":\"wall\",\"18,7\":\"open\",\"18,8\":\"wall\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "d272fdd176519a4c4f4eed7d2fd0e861ee2d4838c51aed8d85e801efd21c8c2a"}317{"id": "first_move-000316", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,4), goal at (5,1).\n#########\n#.......#\n#.#####.#\n#.#.A...#\n#.#.#.#.#\n#G#.#...#\n###.#.#.#\n#...#...#\n#########", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "7bd11b6e1843fa6a82551b5a5df86a3178dd449ab2a93317759cce3df287b154"}318{"id": "distance_band-000317", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,12), goal at (2,0).\n............#...\n#.##..........#.\nG.##.........##.\n##.....#.#.#....\n##....##......#.\n...#.....#......\n#.###.....####..\n..#..#.....#....\n....#...........\n...........#..#.\n....##..........\n##..........##..\n.........##.....\n#.##......#.A##.\n#......#........\n#..............#", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 3, "answer": "20 to 39 moves", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "61138122df7a5f42f8fe141cc3d8e1577d971c06010e89b90f18e0a5c0c7de29"}319{"id": "distance_band-000318", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (4,7), goal at (3,13).\n########################\n#...#.........#.......##\n#.#.#.#.#####.#.###.####\n#.#.#.......#G#...#...##\n#.#.#.#A#.#.###.#.###.##\n#.#...#...........#...##\n#.#.#####.##..#.###.####\n#.........#.......#...##\n###.#######.#####.#.#.##\n#.......#...#.........##\n#.#.#.#.#.#####.########\n#.#...#.#.#...........##\n#...#.#.#...###.#####.##\n#.#.#.....#...........##\n#.#.###.###.####.####.##\n#...................#.##\n#.###########.#.###.#.##\n#.......#...#.....#...##\n#######.#.###.#.#.#.#.##\n#...#...#.......#.....##\n#.#.#.#.####.##.#.#.#.##\n#.....................##\n########################\n########################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b8d245e042a20d4e22a9edc5121ec117033879a5a3f96838dea59ae04b2d1c16"}320{"id": "snake_safe-000319", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,2), travelling west, length 3.\n............\n............\n............\n............\n............\n............\n............\n..Hoo.......\n............\n.......*....\n............\n............", "question": "If the snake moves east on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "2e236c785675ffd462cc8a1221e0e89e166c3ad86486376e01fe80a2265cfa60"}321{"id": "only_way-000320", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[35,0],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"33,-2\":\"outside\",\"33,-1\":\"outside\",\"33,0\":\"open\",\"33,1\":\"open\",\"33,2\":\"wall\",\"34,-2\":\"outside\",\"34,-1\":\"outside\",\"34,0\":\"open\",\"34,1\":\"wall\",\"34,2\":\"open\",\"35,-2\":\"outside\",\"35,-1\":\"outside\",\"35,0\":\"agent\",\"35,1\":\"wall\",\"35,2\":\"open\",\"36,-2\":\"outside\",\"36,-1\":\"outside\",\"36,0\":\"outside\",\"36,1\":\"outside\",\"36,2\":\"outside\",\"37,-2\":\"outside\",\"37,-1\":\"outside\",\"37,0\":\"outside\",\"37,1\":\"outside\",\"37,2\":\"outside\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "91c69851b7d767bb0608722a1ecc7c544281bcad9f2682a5754efa175bf58eef"}322{"id": "reachable_within-000321", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,6), goal at (5,9).\n############\n#.........##\n#.#####.#.##\n#.#...#...##\n#.###.###.##\n#.....A.#G##\n#.#####.####\n#.....#...##\n#.###.###.##\n#.........##\n############\n############", "question": "Can the agent reach the goal in at most 21 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "37d4ae92f9970f0f60a5cc9615996fa2e52bc69d0b93485157a43b1435dd3131"}323{"id": "reachable_within-000322", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,18), goal at (19,1).\n################################\n#.............................##\n#.###########################.##\n#.#...#.......................##\n#.#.#.#.########################\n#.#.#.#.#...#.....A...........##\n#.#.#.#.#.#.#.###############.##\n#.#.#.#.#.#.#.#.............#.##\n#.#.#.#.#.#.#.#.###########.#.##\n#.#.#.#.#.#.#.#...........#.#.##\n#.#.#.#.#.#.#.###########.#.#.##\n#.#.#.#.#.#.#.......#...#.#.#.##\n#.#.#.#.#.#.#######.#.###.#.#.##\n#.#.#.#.#.#.#...#.#.#.....#.#.##\n#.#.#.#.#.#.#.#.#.#.#######.#.##\n#.#.#.#.#.#.#.#.#.#.#.......#.##\n#.#.#.#.#.#.#.#.#.#.#.#####.#.##\n#.#.#.#...#.#.#...#.#.#.....#.##\n#.#.#######.#.#####.#.#######.##\n#G#.......#.#.......#.......#.##\n#########.#.#.#######.#####.#.##\n#.........#.#.#.....#.#...#.#.##\n#.#######.#.#.#.###.#.#.#.#.#.##\n#.#.......#.#.#.#...#.#.#.#.#.##\n#.#####.###.#.#.#####.###.#.#.##\n#.....#...#.#.#...........#.#.##\n#.###.#####.#.#############.#.##\n#.#.#.#.....#...............#.##\n#.#.#.#.#####################.##\n#...#.........................##\n################################\n################################", "question": "Can the agent reach the goal in at most 186 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "d7354162b5d3128335ea39a7f5d5b8cc111b86e1bad2d72035f0624a3f9a0574"}324{"id": "cell_kind-000323", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 0, column 14. Rows increase to the south and columns to the east.\nRow -4: column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze\nRow -3: column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze\nRow -2: column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze\nRow -1: column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze\nRow 0: column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is the agent, column 15 is open, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze\nRow 1: column 10 is open, column 11 is open, column 12 is wall, column 13 is open, column 14 is open, column 15 is open, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze\nRow 2: column 10 is open, column 11 is wall, column 12 is open, column 13 is open, column 14 is open, column 15 is open, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze\nRow 3: column 10 is wall, column 11 is open, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is open, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze\nRow 4: column 10 is open, column 11 is open, column 12 is wall, column 13 is open, column 14 is open, column 15 is open, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "4a9c57a3feb7fc3109d20fc5ea25b210f1d0d5b7d46340a182046cf78114aa28"}325{"id": "plan_survives-000324", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,8); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXX\n#####\n#.A.#\n#.###\n..#.#", "question": "The agent will attempt this plan in order: east, south, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "22403d50e4a86ec4abba02886c506f9f69663577ad712b5c08c2f49178f57d3e"}326{"id": "open_count-000325", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[5,22],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,18\":\"open\",\"1,19\":\"open\",\"1,20\":\"wall\",\"1,21\":\"open\",\"1,22\":\"open\",\"1,23\":\"open\",\"1,24\":\"open\",\"1,25\":\"open\",\"1,26\":\"wall\",\"2,18\":\"wall\",\"2,19\":\"open\",\"2,20\":\"wall\",\"2,21\":\"open\",\"2,22\":\"wall\",\"2,23\":\"wall\",\"2,24\":\"wall\",\"2,25\":\"open\",\"2,26\":\"wall\",\"3,18\":\"open\",\"3,19\":\"open\",\"3,20\":\"open\",\"3,21\":\"open\",\"3,22\":\"open\",\"3,23\":\"open\",\"3,24\":\"open\",\"3,25\":\"open\",\"3,26\":\"wall\",\"4,18\":\"wall\",\"4,19\":\"wall\",\"4,20\":\"wall\",\"4,21\":\"open\",\"4,22\":\"wall\",\"4,23\":\"wall\",\"4,24\":\"wall\",\"4,25\":\"open\",\"4,26\":\"wall\",\"5,18\":\"wall\",\"5,19\":\"open\",\"5,20\":\"open\",\"5,21\":\"open\",\"5,22\":\"agent\",\"5,23\":\"open\",\"5,24\":\"open\",\"5,25\":\"open\",\"5,26\":\"wall\",\"6,18\":\"wall\",\"6,19\":\"open\",\"6,20\":\"wall\",\"6,21\":\"wall\",\"6,22\":\"wall\",\"6,23\":\"wall\",\"6,24\":\"wall\",\"6,25\":\"wall\",\"6,26\":\"wall\",\"7,18\":\"open\",\"7,19\":\"open\",\"7,20\":\"wall\",\"7,21\":\"open\",\"7,22\":\"open\",\"7,23\":\"open\",\"7,24\":\"open\",\"7,25\":\"open\",\"7,26\":\"wall\",\"8,18\":\"wall\",\"8,19\":\"wall\",\"8,20\":\"wall\",\"8,21\":\"open\",\"8,22\":\"wall\",\"8,23\":\"open\",\"8,24\":\"wall\",\"8,25\":\"open\",\"8,26\":\"wall\",\"9,18\":\"open\",\"9,19\":\"open\",\"9,20\":\"open\",\"9,21\":\"open\",\"9,22\":\"wall\",\"9,23\":\"open\",\"9,24\":\"open\",\"9,25\":\"open\",\"9,26\":\"wall\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 2, "answer": "2", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "45099fbe8620cc3a9ba34975abc10d8b19f800734f15d9b416cca588dadbb850"}327{"id": "cell_kind-000326", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[5,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"wall\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"1,6\":\"open\",\"1,7\":\"open\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"2,6\":\"wall\",\"2,7\":\"open\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"open\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"agent\",\"5,4\":\"open\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"wall\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"6,7\":\"open\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"open\",\"7,3\":\"open\",\"7,4\":\"open\",\"7,5\":\"open\",\"7,6\":\"open\",\"7,7\":\"open\",\"8,-1\":\"outside\",\"8,0\":\"wall\",\"8,1\":\"wall\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"wall\",\"8,6\":\"wall\",\"8,7\":\"wall\",\"9,-1\":\"outside\",\"9,0\":\"outside\",\"9,1\":\"outside\",\"9,2\":\"outside\",\"9,3\":\"outside\",\"9,4\":\"outside\",\"9,5\":\"outside\",\"9,6\":\"outside\",\"9,7\":\"outside\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "4c1e1523b9a2e04cdbe05905e218b093ddf7a8e0db96504a3138719354010e53"}328{"id": "snake_safe-000327", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,4), travelling west, length 3.\n..........\n.........*\n..........\n..........\n..........\n.....o....\n....Ho....\n..........\n..........\n..........", "question": "If the snake moves west on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "37a5ef20eb94b8211f868d6acb7748a3bd304a2fed53d96aca29886662be5b36"}329{"id": "goal_bearing-000328", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,3), goal at (5,11).\n#####################\n#.#.................#\n#.#.###############.#\n#.#.#.........#...#.#\n#.#.#.#######.#.#.#.#\n#.#A#......G#.#.#.#.#\n#.###########.#.#.#.#\n#...........#...#.#.#\n#.#########.#.###.#.#\n#.#.......#.#.#.#.#.#\n#.#.#####.#.#.#.#.#.#\n#.#.#...#.#.#.#.#.#.#\n#.#.#.#.#.#.#.#.#.#.#\n#.#.#.#.#.#.#.#.#.#.#\n#.#.#.#.#.#.#.#.#.#.#\n#.#.#.#.#.#.#...#.#.#\n#.#.###.#.#.#####.#.#\n#.#.....#.........#.#\n#.#################.#\n#...................#\n#####################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "6140881739424c57b453a18ce4c8d4aba3ddc03ffd4f647ae3d75fe65bf874fe"}330{"id": "reachable_within-000329", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,2), goal at (4,0).\n..##.#..#...\n.....#....##\n...........#\n#.........##\nG#####......\n....##.#..##\n....#.......\n....#...#..#\n.#..#.#.....\n.#A#.#...###\n.#....#...##\n....#.....##", "question": "Can the agent reach the goal in at most 7 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "1c4ef40478785dc647cb9e531480cbcfdeed4731905172f6c8bac07305290023"}331{"id": "distance_band-000330", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (11,2), goal at (15,0).\n##########.#..#.\n##########......\n#..########.#.#.\n..#.###.##..#.##\n#....#...#.#..##\n....##.....#..##\n#...##....##....\n.#..##.#..###...\n...........##..#\n#......#.####..#\n#....#...#.#....\n.#A#.##.........\n......##..#..#..\n#..#....##....#.\n#.....#....##..#\nG.#.###..#.###..", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 1, "answer": "5 to 9 moves", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "efef131348ff144579286b3fbe452f425137d8278852662e5ef738d117f77d58"}332{"id": "reachable_within-000331", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,1), goal at (11,19).\n########################\n#.............#.......##\n#.#.###.#.###.#.#####.##\n#.#.#...#.#.#.#.#.#.#.##\n#.###.#.#.#.#.#.#.#.#.##\n#.#...#.#.#...#.#.#...##\n#.#.#.#...#.###.#.###.##\n#...#.....#...#.....#.##\n#.###.###.#.###.###.#.##\n#A#...#...#.....#...#.##\n#.#.#.#.###.###.#.###.##\n#.....#...#.....#.#G#.##\n#####.###.###.#...#.#.##\n#.....#.........#.#...##\n#.###.#.###.###.#.###.##\n#.....#.........#...#.##\n#.###.#.#.#########.#.##\n#...#...#...........#.##\n###.#.########.######.##\n#.....#...........#...##\n#.#.#.#.#########.#.####\n#...#...........#.....##\n########################\n########################", "question": "Can the agent reach the goal in at most 52 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "d3126d99301a66c51751204875dd93f00143b20d592ab6d1152b8b287195f1da"}333{"id": "reachable_within-000332", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (4,1), goal at (7,9).\n############\n#.........##\n#.#######.##\n#.#...#.#.##\n#A#.#.#.####\n#...#.....##\n#.#######.##\n#.....#..G##\n#.#.#.#.#.##\n#.#.......##\n############\n############", "question": "Can the agent reach the goal in at most 16 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "69ae0c4da01725b98fe82e0dbae8ca3ace872de917def5f816d5b4e1b63bb06a"}334{"id": "snake_food-000333", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,4), travelling south, length 3.\n.......*\n........\n........\n........\n...oo...\n....H...\n........\n........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "092a5adc17002f3d8d00aaf05ff4b3f0d431479fe327d91aa01559005e961548"}335{"id": "first_move-000334", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (19,34), goal at (6,0).\n##..#..##.#..##.................####\n###.#..##..#....#.....#..#.#...#####\n.#...............#..#......#..######\n........####...###..##..#.##########\n.#......####.....###......###.##..##\n...##..######....######..#.#......##\nG####...####..#..######....#.#......\n#..#...#....#.....####......###.....\n.....#...#..#.#....#..#....#.#...#.#\n..##.......#.............#...#.#....\n......##...#.##..........#..#.......\n..###......##...##.......#.##..#.#..\n.#..#.#......####.....#..#...##..#..\n.....#....#.#.#....#.#....#....#.##.\n.#.......#.......#..##.#......####..\n.......#..#..#..##..#......#..####.#\n...#.#..##.......#....#...###.####..\n.#..#....#..##...#.#..##......#####.\n..#....#.....##.###......###..##.##.\n###..#.#..#.#.#..#.##..##.###.....A.\n.##....#....#...#..#.........##..#..\n.......#..##.##..#.#..#.....#....#..\n.##.#....................#.#...#....\n..#.#..#####.......#.##.#.......#.#.\n...#..#.##.......#..##...#...#......\n......#.#.##..#...#...##............\n.##...#........##.........#....##.##\n...##.#.#..#......###..##..#.##....#\n...###...............#....#.#...#...\n..#.#.#.#####...#.....#.#.#..#..#..#\n.#....#..#...#.......#....##..####..\n.#..#...........#.#............###.#\n...#.#.#......##.....#...#.#....#...\n#.##..##..........#....#..###....#..\n##..#....##......######..###.....#..\n##...........#..#######.####...#....", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "df391d3b948a4d3fa975d4284d70f7504161e5e80c5c0b752ca8dae10a1d472f"}336{"id": "blocked_pair-000335", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[7,19],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"5,17\":\"open\",\"5,18\":\"open\",\"5,19\":\"open\",\"5,20\":\"open\",\"5,21\":\"open\",\"6,17\":\"wall\",\"6,18\":\"wall\",\"6,19\":\"wall\",\"6,20\":\"wall\",\"6,21\":\"open\",\"7,17\":\"open\",\"7,18\":\"wall\",\"7,19\":\"agent\",\"7,20\":\"wall\",\"7,21\":\"open\",\"8,17\":\"open\",\"8,18\":\"wall\",\"8,19\":\"open\",\"8,20\":\"wall\",\"8,21\":\"open\",\"9,17\":\"open\",\"9,18\":\"open\",\"9,19\":\"open\",\"9,20\":\"wall\",\"9,21\":\"open\"}}", "question": "Are BOTH the cell one step north and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "6b3d817e56e75a92c523af635a6d96e4773915f45d439b6749fae3cd823e2bf2"}337{"id": "stochastic-000336", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 1, column 4. Rows increase to the south and columns to the east.\nRow -3: column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow -2: column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow -1: column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow 0: column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall\nRow 1: column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is the agent, column 5 is open, column 6 is wall, column 7 is open, column 8 is open\nRow 2: column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall\nRow 3: column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 4: column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall\nRow 5: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is open", "question": "The actuator executes the intended move with probability 0.70; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.7999999999999999, 0.20000000000000004], "deterministic": false, "fingerprint": "5c177e82178d56b59c23dad94c87afb649753f01d4e023fc7dddb50afc3c9659"}338{"id": "goal_bearing-000337", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (33,3), goal at (33,11).\n####################################\n#.................#...#...........##\n#.###########.###.#.#.#####.#####.##\n#.........#...#.....#.#...#.#...#.##\n###.#####.#.#.#######.#.#.###.#.#.##\n#...#...#.#.#.#...#...#.#.....#.#.##\n#.###.#.###.###.#.###.#.#######.#.##\n#...#.#...#.#...#...#...#.#...#.#.##\n#####.###.#.#.#####.#####.#.#.#.#.##\n#.....#...#.#.#...#...#.....#...#.##\n#.#####.###.#.###.###.###.#######.##\n#.....#.....#...#...#...#.#.#.....##\n#####.#####.###.###.###.#.#.#.#.#.##\n#.....#.#.....#.....#...#.#...#.#.##\n#.#####.#.#####.#####.###.#.###.#.##\n#.....#...#...#.#...#...#.#.#...#.##\n#.###.#####.#.#.#.#.###.#.###.###.##\n#...#.....#.#.#.#.#.....#...#.#.#.##\n###.#####.#.#.#.#.#########.#.#.#.##\n#.#.#...#.#.#...#...#.#.....#.#...##\n#.#.###.#.#.#######.#.#.#####.###.##\n#.#.....#...#...#...#...#...#...#.##\n#.#####.#######.#.#####.#.#.###.#.##\n#.......#.......#.....#...#.....#.##\n#.###########.#####.#.###########.##\n#.#.......#...#...#.#...#.....#...##\n#.#.#####.#.###.#.#####.###.#.#.####\n#.#.#.......#...#.....#...#.#...#.##\n#.#.#.#######.#######.###.#.#####.##\n#.#.#.#.....#.....#.#.....#.#.....##\n#.#.#.#.###.#####.#.#######.###.#.##\n#...#.#...#.......#.......#.....#.##\n#.#######.#########.#####.#######.##\n#.#A......#G............#.........##\n####################################\n####################################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "d9cf917d73f0619989af06aa520967852306697a072884f929401abb4ae8e51a"}339{"id": "reachable_within-000338", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,10), goal at (3,3).\n################\n#.............##\n#.###########.##\n#.#G..........##\n#.##############\n#.............##\n#.##############\n#.#...#.......##\n#.#.###.#####.##\n#.#.....#...#.##\n#.#.#######.#.##\n#.#.........#.##\n#.###########.##\n#.........A...##\n################\n################", "question": "Can the agent reach the goal in at most 49 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "7a86e5d010faf10fe7ce3fe416b317246752dddb6102cd566c4e96edacd33bad"}340{"id": "move_two_step-000339", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 8, column 1. Rows increase to the south and columns to the east.\nRow 6: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open\nRow 7: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is open, column 3 is open\nRow 8: column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is open, column 3 is open\nRow 9: column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is wall, column 3 is open\nRow 10: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall", "question": "The agent plans to move east, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "c7c05e6dd5e59097e263aebdf3c02c3e733c1bb4dd789e94d7c42bf65594225a"}341{"id": "snake_food-000340", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,5), travelling west, length 3.\n............\n............\n............\n............\n.......*....\n............\n......o.....\n.....Ho.....\n............\n............\n............\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "be5e88a67d8d628ba161d57e8d21dcedef3538211a39162b71d591cf90be6186"}342{"id": "reachable_within-000341", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,23), goal at (14,0).\n######..#..#..#..#....##\n###.##.#.#..#..........#\n##.......##..#.#...#....\n#....##.....###....##.#A\n#....##...#####.#.#....#\n#####.....####......####\n######.#..###..#.#######\n#######.....###...#.####\n#######...#...#...#..###\n####..#.#..#####.#..####\n.##....##.#..........###\n....#.###.###..##..#..##\n...#...#......###.#..###\n.#####.#..##.####....###\nG####..#..########..####\n######...###############\n######...###############\n########.###############\n########################\n########################\n########################\n########################\n########################\n########################", "question": "Can the agent reach the goal in at most 43 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "438e43120d68da908f02c2e3102fe91670832f17533ee9e3f247b341290de3fd"}343{"id": "reachable_within-000342", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,4), goal at (7,7).\n############\n#...A...#.##\n#.#####.#.##\n#.#...#.#.##\n#.#.#####.##\n#.#.......##\n#.#.#####.##\n#.#....G#.##\n#.#######.##\n#.........##\n############\n############", "question": "Can the agent reach the goal in at most 35 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "756a9dee1a6eb0eef4a8797a120ebec700bc69c9b6a37c0710786a62b28acb9f"}344{"id": "distance_band-000343", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 40x40. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (37,14), goal at (1,37).\n########################################\n#...#.......#...............#........G##\n#.#.#.#.#.#.#########.#.#.#.#####.###.##\n#.#...#...#.#...#...#.....#.......#.#.##\n#.#.#.#.###.#.#.#.#.#.#########.###.#.##\n#...#.............#...#.....#.......#.##\n#####.#.#.#####.#######.###.#.##.##.#.##\n#.....#.......#.......#...#...#...#...##\n#.#.###.###.###.###.#####.#####.#.#.####\n#.....#.#.#...#...#.....#.....#.#.#...##\n###.....#.###.###.###.####.##.#.#.#.#.##\n#.#.#.#.....#...#...#.........#.#...#.##\n#.#...#.#.###.#.###...#.###.###.#####.##\n#.#.#.#.#...#...........#.....#...#...##\n#.###.#...#.####.#.######.###.##.##.#.##\n#.....#.#.#.............#...........#.##\n#.###.#.#.###.#######.#.#.#.#########.##\n#.#.#...#.#.....#...#.#.....#.........##\n#.#.###.#.#.#####.#.#.###.###.#.###.#.##\n#.....#.#.#.....#.....#...#...#.#...#.##\n#.###.#.#.###.#.#.#.###.#.#.###.#.###.##\n#...#...#.#.....#.#.....#...#...#...#.##\n#.#.#...#.#.#.###.#.#.#######.#.###.#.##\n#.....#.#...#.....#.....#.......#...#.##\n#.###.#.#.#.#.#.#.###.#.#.#####.#.###.##\n#.....#.....#.....#...#.......#.#.....##\n#####.###.#####.###.#.#######.#.#.#.#.##\n#.....#.......#.#...#...........#.#...##\n#.#####.##.##.#.#.#.#############.###.##\n#.......#...#.#.#...........#...#...#.##\n#...#.###.###...#.#.###.###.#.#.#.#.####\n#...#.....#...#.#.#.....#.#...#.#.#...##\n#.###.#.#.#.###.#.#####.#.#.###.#.###.##\n#.....#.#.#.#.#...#...#.#.....#.......##\n#####.#.#.#.#.#.#####...#.#.######.##.##\n#.....#.#.............#.#.#.........#.##\n#.###.#.###.##.##.###.#.#.#.#.#.#.#.#.##\n#...#.........A.........#...#.........##\n########################################\n########################################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 4, "answer": "40 moves or more", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "f15aeb7b454d264909dc29f021648b1f3a4bb0db8138be7f6778cbde7d09c351"}345{"id": "open_count-000344", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (0,26); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXX\nXXXXX\n..A#X\n#.##X\n.###X", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 1, "answer": "1", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3942dcc45cac90a3484cb46305858899160e438b6c31e5b489a1606ea66e7555"}346{"id": "open_count-000345", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[12,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"9,4\":\"open\",\"9,5\":\"open\",\"9,6\":\"open\",\"9,7\":\"open\",\"9,8\":\"open\",\"9,9\":\"open\",\"9,10\":\"open\",\"10,4\":\"wall\",\"10,5\":\"wall\",\"10,6\":\"wall\",\"10,7\":\"wall\",\"10,8\":\"wall\",\"10,9\":\"open\",\"10,10\":\"wall\",\"11,4\":\"wall\",\"11,5\":\"open\",\"11,6\":\"open\",\"11,7\":\"open\",\"11,8\":\"wall\",\"11,9\":\"open\",\"11,10\":\"wall\",\"12,4\":\"wall\",\"12,5\":\"open\",\"12,6\":\"wall\",\"12,7\":\"agent\",\"12,8\":\"wall\",\"12,9\":\"open\",\"12,10\":\"wall\",\"13,4\":\"open\",\"13,5\":\"open\",\"13,6\":\"wall\",\"13,7\":\"open\",\"13,8\":\"wall\",\"13,9\":\"open\",\"13,10\":\"open\",\"14,4\":\"wall\",\"14,5\":\"wall\",\"14,6\":\"wall\",\"14,7\":\"wall\",\"14,8\":\"wall\",\"14,9\":\"wall\",\"14,10\":\"wall\",\"15,4\":\"wall\",\"15,5\":\"wall\",\"15,6\":\"wall\",\"15,7\":\"wall\",\"15,8\":\"wall\",\"15,9\":\"wall\",\"15,10\":\"wall\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 2, "answer": "2", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "29e9579dee8ba5dd73ddb1f357299fcfe9029f7038a3a44888157050e43c3cf5"}347{"id": "first_move-000346", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (4,9), goal at (15,2).\n#.##..#....#...#\n...#..#.........\n..#....##.#....#\n........###....#\n#..#.....A......\n#...#.#....#.##.\n#......#......##\n.......#..##..##\n..#.....#....#.#\n........#..#.#..\n..#..#......#...\n...#........#.##\n#...###..#......\n#...####........\n...######..#..##\n..G#######.#..##", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "743cd2c5c72e93ceeee5086f07578b0d2386a8115f85cde92ca456204b3c51a4"}348{"id": "distance_band-000347", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,7), goal at (13,9).\n################\n#...#...#.....##\n###.#.#.#.###.##\n#...#.#...#.#.##\n#.###.#####.#.##\n#...#.#...#...##\n###.#.#.#.###.##\n#...#.#.#...#.##\n#.###.#.###.#.##\n#.#.......#.#.##\n#.#########.#.##\n#.#...#.....#.##\n#.#.#.#.#####.##\n#...#..A#G....##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 4, "answer": "40 moves or more", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "65b86cfc7f957d75e8a92f43fec3da2d7d270e73243fb035006c5858be406520"}349{"id": "onward-000348", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 1. Rows increase to the south and columns to the east.\nRow 3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is wall, column 5 is open\nRow 4: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 5: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 6: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall\nRow 7: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 8: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 9: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 10: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 11: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze", "question": "Suppose the agent moves one cell north. From that new cell, would a further step east be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "8ccfb60ec7a2d9abc439b97be1e7e6ab272be59738d29c8a9e1e9bd132c79807"}350{"id": "snake_food-000349", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (10,8), travelling south, length 3.\n............\n............\n...........*\n............\n............\n............\n............\n............\n........o...\n........o...\n........H...\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "0a238bc6182d03205099bd0184a554c048b4dc1c160769538422364950f2aa0c"}351{"id": "first_move-000350", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (4,0), goal at (4,2).\n.#.###.#.\n.........\n##.#.##.#\n.#.#.#...\nA#G###.#.\n..#..#...\n.......#.\n.##.#.#..\n#......#.", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "920f5b09b2d433016631e3429528a72fdb7aae24508774df4000e726cbbd88ad"}352{"id": "open_count-000351", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 25, column 5. Rows increase to the south and columns to the east.\nRow 23: column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open\nRow 24: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open\nRow 25: column 3 is open, column 4 is wall, column 5 is the agent, column 6 is wall, column 7 is open\nRow 26: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 27: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 1, "answer": "1", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "567e21fd4e437c33b31a733d08e987326a86697d32f4812d5bf23aa303c19aee"}353{"id": "plan_survives-000352", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (5,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX#...\nX####\nX#A#.\nX#.#.\nX#.#.", "question": "The agent will attempt this plan in order: south, north, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "775abb7451d955ab6e33c74b1a84a710f186a51781f430c9198a7478549e6752"}354{"id": "cell_kind-000353", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[26,27],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"24,25\":\"open\",\"24,26\":\"open\",\"24,27\":\"open\",\"24,28\":\"outside\",\"24,29\":\"outside\",\"25,25\":\"wall\",\"25,26\":\"open\",\"25,27\":\"wall\",\"25,28\":\"outside\",\"25,29\":\"outside\",\"26,25\":\"open\",\"26,26\":\"open\",\"26,27\":\"agent\",\"26,28\":\"outside\",\"26,29\":\"outside\",\"27,25\":\"wall\",\"27,26\":\"wall\",\"27,27\":\"wall\",\"27,28\":\"outside\",\"27,29\":\"outside\",\"28,25\":\"outside\",\"28,26\":\"outside\",\"28,27\":\"outside\",\"28,28\":\"outside\",\"28,29\":\"outside\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "d85349637d5acbd93ff0fef4079e5401d8cb07b675a2ace72c0fa99af6299d6e"}355{"id": "distance_band-000354", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,23), goal at (8,20).\n.#....###.#..........###\n.#...#.........#..#....#\n..#..#.#.##.##....###...\n.##.......##..##..#.###.\n...#....##..#.......#...\n#.....#........#..##...A\n......#.....#####.###.#.\n.....#...#.#...#...#...#\n....#......#..##...#G###\n..#.##..##.....#....#.##\n..#....#........###...##\n....##..#.........#.#..#\n...#..............#.#...\n..#......###...#.....#..\n..#......#.##..######...\n.##.##..#......###......\n...#....###...###..##.#.\n....#..###...#.....#.#.#\n......###......#..#....#\n##..##.#....#.#...#.....\n...#.....#.#..#.#.......\n##.#....#..##...#.#..#..\n##....#...##.#.#####.##.\n#....#..#....######.....", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 1, "answer": "5 to 9 moves", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "ab9646e3999b8cf26d343478a63b3becf8809ce30a9e354ea6239ab65fc0d905"}356{"id": "snake_food-000355", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,8), travelling east, length 3.\n..........\n..........\n..........\n..........\n..........\n.......o..\n.......oH.\n..........\n........*.\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 1, "answer": "south", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "15ae6206bc40fb0d782ba7c4b330c132d5411525721134680472f20ca43b9dc7"}357{"id": "move_two_step-000356", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[25,26],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"23,24\":\"open\",\"23,25\":\"wall\",\"23,26\":\"open\",\"23,27\":\"open\",\"23,28\":\"outside\",\"24,24\":\"open\",\"24,25\":\"open\",\"24,26\":\"open\",\"24,27\":\"open\",\"24,28\":\"outside\",\"25,24\":\"open\",\"25,25\":\"open\",\"25,26\":\"agent\",\"25,27\":\"open\",\"25,28\":\"outside\",\"26,24\":\"open\",\"26,25\":\"wall\",\"26,26\":\"open\",\"26,27\":\"open\",\"26,28\":\"outside\",\"27,24\":\"open\",\"27,25\":\"open\",\"27,26\":\"open\",\"27,27\":\"open\",\"27,28\":\"outside\"}}", "question": "The agent plans to move east, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "008fed98f4d9efe98998340d9f499b508da6d4e993ff5f610d49a075ae09c582"}358{"id": "snake_safe-000357", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,6), travelling south, length 3.\n..............\n..............\n..............\n..........*...\n..............\n..............\n......o.......\n......o.......\n......H.......\n..............\n..............\n..............\n..............\n..............", "question": "If the snake moves north on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "0554db125e32a458bee57d4f509bbf97b64f529ac5f55d58071795a0fb8e1490"}359{"id": "only_way-000358", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 13, column 15. Rows increase to the south and columns to the east.\nRow 11: column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open\nRow 12: column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open\nRow 13: column 13 is open, column 14 is wall, column 15 is the agent, column 16 is wall, column 17 is open\nRow 14: column 13 is open, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is open\nRow 15: column 13 is open, column 14 is open, column 15 is open, column 16 is open, column 17 is open", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "532f3be4d489bca055969347cb68ba74951a29524b516dbd2ae171d8d101adfc"}360{"id": "open_count-000359", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 3, column 13. Rows increase to the south and columns to the east.\nRow 1: column 11 is open, column 12 is open, column 13 is open, column 14 is open, column 15 is open\nRow 2: column 11 is open, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall\nRow 3: column 11 is open, column 12 is wall, column 13 is the agent, column 14 is wall, column 15 is open\nRow 4: column 11 is wall, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open\nRow 5: column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is open", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 1, "answer": "1", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "f420b5d7c6b48aa99520c58417d9a0f16c63bcb16ce24598bab7d87fb48824dc"}361{"id": "first_move-000360", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (27,28), goal at (33,30).\n####################################\n#...........#...................#.##\n#.#.#.#.#.#.##########.######.#.#.##\n#.....#...#...#.........#.........##\n#.###.###.#.#.#.#######.#.#.#.#.#.##\n#.....#.....#.#...#.......#...#...##\n#.##.##.#.#.#.###.##.######.###.####\n#.......#...#.....#...........#.#.##\n#####.###.#.#.#####.#####..##.#.#.##\n#...#.......#.......#.......#.#...##\n#.#.#####.###.##.####.###.#.#.######\n#.#.......#...#.....#.............##\n#.#####.#.#.###.###.###.###.#####.##\n#.#...#...#.#.........#.#.......#.##\n#.#.#.#.###.####.####.#.#.###.###.##\n#.#...#.#.#...#.......#.#...#.....##\n#.....#.#.###.#.#####.#.#.#.#.##.###\n#.#...........#...#.....#.#.......##\n#.#.#####.#.#.###.#.#####.#######.##\n#...............#.#.#...#.......#.##\n#.#.#####.##.##.#.#.#.#.##.#.##.#.##\n#.#...........#.....#.#.......#...##\n#.##..###.#.#.###.#.#...#.###.###.##\n#.........#.....#.#...#...#.....#.##\n#.#.#.#.#.#.#.#.#.#######.#.#.###.##\n#.......#...............#.#.......##\n#.#####...###.#####.#.###.#.########\n#.#.....#.#.......#.#.#...#.A.....##\n#.#.###.#.#.###.#.#.#...#######.#.##\n#.#...#...#.....#...#...#.......#.##\n#.###.#..##.#.#.#####.#.#.#.###.#.##\n#.#...#...#...#.#.....#.#.#...#.#.##\n#.#.###.#.###.###.#.#.#.#.#.#.###.##\n#.#.....#.........#...#.....#.G...##\n####################################\n####################################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b3a165e1e38290491b6d9384306b2e67dd0e8c9bcc7de38a25ad674f5ee7b112"}362{"id": "cell_kind-000361", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (17,21); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#.#...#.\n.#.#####.\n.#.....#.\n.#.###.#.\n.#.#A#.#.\n.#.#.#.#.\n.#.#.#.#.\n.#.#.#.#.\n.#...#.#.", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "2cda0ffe12e5139e7eae93ec975684677c40329805117dbf504da41f1e3366dc"}363{"id": "snake_food-000362", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,3), travelling west, length 3.\n..........\n..........\n..........\n..........\n..........\n....o.....\n...Ho.....\n......*...\n..........\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 6, "answer": "south-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "c7ccde107750e1639b79c592ddb161f73576d6e58bce8f725140c8e51da1d0ed"}364{"id": "only_way-000363", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[9,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"open\",\"7,3\":\"open\",\"8,-1\":\"outside\",\"8,0\":\"wall\",\"8,1\":\"open\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"9,-1\":\"outside\",\"9,0\":\"wall\",\"9,1\":\"agent\",\"9,2\":\"wall\",\"9,3\":\"open\",\"10,-1\":\"outside\",\"10,0\":\"wall\",\"10,1\":\"wall\",\"10,2\":\"wall\",\"10,3\":\"open\",\"11,-1\":\"outside\",\"11,0\":\"wall\",\"11,1\":\"open\",\"11,2\":\"open\",\"11,3\":\"open\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "2a98821b508aa2031f1466760a581cd11da63a949d6b4643dd570bddf193c0bc"}365{"id": "first_move-000364", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,1), goal at (1,7).\n#########\n#......G#\n###.#####\n#A#.....#\n#.#.###.#\n#.#...#.#\n#.#####.#\n#.......#\n#########", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "d3c7a2328dccc257957dbb1e0778e90d57ebedec4f9c68f28f5e6b6f5132b321"}366{"id": "stochastic-000365", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (7,5); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n...#.\n.###.\n.#A..\n#####\nXXXXX", "question": "The actuator executes the intended move with probability 0.70; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9, 0.10000000000000002], "deterministic": false, "fingerprint": "f60e48d5165f9b5bd79d2b726604f99c98727660da5cdefb43e31019186fb984"}367{"id": "open_count-000366", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,34); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n....X\n....X\n..A.X\n....X\n#..#X", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 4, "answer": "4", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "01a134c8e79d2152b6a7c972c433869ce838f20c95444a3888c172dd766fd003"}368{"id": "open_count-000367", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXX\nXXXXXXX\n#####XX\n...A#XX\n#####XX\n....#XX\n###.#XX", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 1, "answer": "1", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "524288e0161ccb6b586e0ee953ce5937abc1bdc2071c349465975f6d01833136"}369{"id": "first_move-000368", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,17), goal at (7,3).\n############################\n#.....#.......#...#...#...##\n#.###.#.#.###.#.#.#.#.#.#.##\n#.#...#.#.#...#.#.#.#...#.##\n#.#.###.#.###.#.#.#.#####.##\n#.#...#.#...#...#.#...#...##\n#.###.#.###.#####.###.######\n#.#G#.#...#.#....A....#...##\n#.#.#.###.#.###########.#.##\n#...#.#...#.............#.##\n#####.#.#################.##\n#.....#.....#.......#...#.##\n#.#########.#.#####.#.#.#.##\n#.#.......#...#.....#.#...##\n#.#.#####.#######.###.######\n#...#...#.#.....#...#.....##\n#####.#.#.#.###.#########.##\n#...#.#...#.#.#...#.......##\n#.#.#.#####.#.###.#.#####.##\n#.#...#.....#.....#.....#.##\n#.#####.#.#####.#######.#.##\n#.#.....#.....#.#...#...#.##\n#.###########.#.#.#.#.###.##\n#.......#...#.#.#.#.#.#.#.##\n#######.#.#.#.#.#.#.#.#.#.##\n#.........#...#...#...#...##\n############################\n############################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "495d14b62e3b277a94221711af2b176345a1bed4fa943b6009d4449677d1fa60"}370{"id": "move_two_step-000369", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 15, column 6. Rows increase to the south and columns to the east.\nRow 12: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall\nRow 13: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is open\nRow 14: column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall\nRow 15: column 3 is open, column 4 is wall, column 5 is open, column 6 is the agent, column 7 is open, column 8 is open, column 9 is open\nRow 16: column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall\nRow 17: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is open\nRow 18: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall", "question": "The agent plans to move south, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "435a43bf01b7b809f832c59bb94989a05371e81cfea22fe6b49542dab36516a4"}371{"id": "snake_food-000370", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,7), travelling east, length 3.\n..........\n.......*..\n..........\n..........\n..........\n..........\n.....ooH..\n..........\n..........\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "e7df514a174d0ad9079687817a4cd21372e7f25c951c49cb5bf9b23887a955f5"}372{"id": "distance_band-000371", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 40x40. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,3), goal at (0,2).\n..G####......##########.......#.###.#..#\n..#######.....##..#####..###.....#..##..\n#...###....##..#.#........##..###.......\n#.#A.##..........##..#..##..#.#####.....\n###..##............#...#...#...######...\n..................##..#...#.#.#.#####...\n.#.#........##...####....#......######..\n.#......##.#.....#.....##.#..#.....#..#.\n.#...#...##...#..######....##.#..#.#....\n#..#....##.#.#........#.##........#...#.\n.#....#....#..##.#....##.#..##..###....#\n.....##..#....##.........###.....#..##..\n.####........###.....#............##..#.\n.###.##.######.###.#.#.#......#.###...#.\n#.#.......#.........#..#..###..#..#..#..\n#......#..####.#.#.#.#....#.......#.#.#.\n....#.#...##.#..#..............#....#...\n#.#.......#.......##...##.......###....#\n....#.#.#..#..###.#...######...##.......\n.#.#...#......#....##..######...#.##..##\n.#..##..##...#.##...#.#.###....#...##...\n..##....####.#.....##...#..##...######.#\n.##...#..#...##.#.#.....#..##....#######\n.###..#.###.........#.#.#......#.#######\n.###..######...#.#.#.#..........#.###.##\n..###...#####.#......#.....##.......#..#\n....#....########..#....#......#.##...#.\n###..#...#.####........##.....#....#.##.\n.#..##.#.#..#..#.#....#..#..#...#.##..#.\n...#...#.#.......##.##.#.....#...#....#.\n.#..###....#...#..#.......###.#....#....\n.##..#..#.##..#.#......#.#....#..#.##...\n...#..##..#.#.......##...###....#..##...\n.#..#.#...#..#.#....##......#....#....#.\n..#.#.#...##..##..##.#....####.##.#.....\n#.....#......#.........#.##..##.....##.#\n##..##..###..#........#####..#..#..##...\n##......#.#.###...#.#.#####..#...##...#.\n#####......#..##.###########........#...\n######..##.....#############.##..#####..", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 1, "answer": "5 to 9 moves", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "acb517aceb6d7ab31cbf6f9ec882eb14ace17c2ab4fb5a1bb795f0d333557db8"}373{"id": "move_two_step-000372", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (6,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.#....\n#.###.#\n#.#.#.#\n#.#A#.#\n#.#.#.#\n#.#.#.#\n#.#.#.#", "question": "The agent plans to move west, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "84c7195a12e561934504d07d9ef875b6ef11c016c0dbe845d9ae8530868d9b99"}374{"id": "plan_survives-000373", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 5, column 19. Rows increase to the south and columns to the east.\nRow 3: column 17 is open, column 18 is wall, column 19 is open, column 20 is open, column 21 is open\nRow 4: column 17 is open, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is open\nRow 5: column 17 is open, column 18 is wall, column 19 is the agent, column 20 is open, column 21 is open\nRow 6: column 17 is open, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is wall\nRow 7: column 17 is open, column 18 is wall, column 19 is open, column 20 is open, column 21 is open", "question": "The agent will attempt this plan in order: north, west, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "035b64b0f5f44d3e2d981d90f5790411a1465f602f6f60501451d9c43e3a26a3"}375{"id": "goal_bearing-000374", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,15), goal at (19,15).\n############################\n#.........................##\n#.#.#####################.##\n#.#.#...#.....#.........#.##\n#.#.#.#.#.###.#.#######.#.##\n#.#.#.#.#.#.#.#.#.....#.#.##\n#.#.#.#.#.#.#.#.#####.#.#.##\n#.#.#.#.#.#.#.#.......#.#.##\n#.#.#.#.#.#.#.#######.#.#.##\n#.#.#.#.#.#.#.#A....#.#.#.##\n#.#.#.#.#.#.#.#####.#.#.#.##\n#.#.#.#.#.#.........#.#.#.##\n#.#.#.#.#.###########.#.#.##\n#.#.#.#.#.............#.#.##\n#.#.#.#.###############.#.##\n#.#.#.#.................#.##\n#.#.#.###################.##\n#.#.#.....................##\n#.#.#.######################\n#.#.#.#.......#G..........##\n#.#.#.#.###.#.###########.##\n#.#.#.#...#.#.#.........#.##\n#.#.#.#####.#.#########.#.##\n#.#.#.......#...........#.##\n#.#.#####################.##\n#.#.......................##\n############################\n############################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 1, "answer": "south", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "4fe00e96a8d734cfb0bddfc8e52eb04f42fe7ebeaa6f22a4e7d81483c1bbece1"}376{"id": "first_move-000375", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,7), goal at (0,3).\n...G#.#.###.\n#.##..#..#..\n...#.#......\n#.#....A...#\n...#..#...##\n#....##...##\n####.##..###\n####..#.####\n#####..#####\n####...#####\n##....######\n#####.######", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "b1ca781b840cf04052f3f52c7d27c1d57ea5c25523e298db1d581f7622307bca"}377{"id": "distance_band-000376", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (4,19), goal at (9,21).\n########################\n#.#.............#.....##\n#...###.#####.#...#.#.##\n#.#.#...#...#...#.#.#.##\n#.#.#.#.#.#.#####.#A#.##\n#...#.#.#.#.........#.##\n#.###.#.#.###.###.#.#.##\n#...#...............#.##\n###.###.###.###.#...#.##\n#.#.......#.#...#...#G##\n#.##.####.#.#.#.#...####\n#.........#.#.....#...##\n##.##.#.###.#...#####.##\n#...#.......#.#.#.....##\n#.#.#########.#.#.######\n#.........#...#.#.....##\n#.#.#.###.###...###.#.##\n#...#...#...#.#.......##\n#####.#.###.#.#.#.###.##\n#.........#.#...#...#.##\n#.##.##.###.#######.#.##\n#.....#.............#.##\n########################\n########################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "397d6b8ee30092a36d439266826b13452c04497d39ea66df3f68ba78b0618b91"}378{"id": "goal_bearing-000377", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (25,5), goal at (32,35).\n####..#..####...#.##.###...##...####\n#####.#...####.#..#....##..#.##.####\n####....#..#.....##.#...##....#.####\n##.....##.......##.....##....#...##.\n#.....#...#.#.####....#...#..###..#.\n##..##..##.....##..#..#.##.#.#......\n......#...#.....#..#.#.....##.##.#..\n....#.##.#....###....##.....#.......\n#.....#....#....#.#...#.#...#..#...#\n.#..#.#...#....#.....##.#..####..#..\n.#...#............#.#....#######....\n...#...###.#####.........######...#.\n..#.....##..#......####.#####..#...#\n..##....###.#......#......###.....##\n....#.#.....#.#......#.#...#...#.##.\n.##..#...#..#.#.####.....###...#..#.\n#......##..#......#...##.....#.#....\n....####.#.....#.....#.....#.####...\n......##.....##.......#.....#.##...#\n...#..#.##............##..#......###\n###...#.#....#.....##..#..###..#.#.#\n####..#...###..........#...#.#......\n##...#.....#....#...##..##..........\n####..#...#......#...#..#....##.#..#\n#####..#..##...#.#.#..##............\n####.A###....#......#..###.......##.\n######.##.###...#.###..#.......#....\n#####...##..##...##...##.#..........\n####....#..##..#..#...###...##...##.\n####...#.#.#...........###..........\n####.#..........#....#..###.......#.\n######.....#.###.#.#...#####.#..#...\n#########.........#..#.#.########..G\n##########.#.#.....#......#..#######\n###########............#.....#######\n############.....#.#.#.###..########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 6, "answer": "south-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "f52800e0cb0f00235e8f219c693a0094deebd4b5110bd305e3413ec4450d63db"}379{"id": "open_count-000378", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 13, column 34. Rows increase to the south and columns to the east.\nRow 11: column 32 is wall, column 33 is open, column 34 is open, column 35 is open, column 36 is outside the maze\nRow 12: column 32 is open, column 33 is wall, column 34 is wall, column 35 is open, column 36 is outside the maze\nRow 13: column 32 is open, column 33 is open, column 34 is the agent, column 35 is open, column 36 is outside the maze\nRow 14: column 32 is wall, column 33 is wall, column 34 is open, column 35 is open, column 36 is outside the maze\nRow 15: column 32 is wall, column 33 is wall, column 34 is wall, column 35 is open, column 36 is outside the maze", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 3, "answer": "3", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "1fffc0d7dc8495bd112f13989cf0b1e2399e9f37a1c8251ef1be18bfa9cd011a"}380{"id": "snake_safe-000379", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (10,3), travelling west, length 3.\n............\n............\n.........*..\n............\n............\n............\n............\n............\n............\n............\n...Hoo......\n............", "question": "If the snake moves east on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "b6169fa33cd3c471afe41a77d94084ffad48dbfd4ddccf8ed9361bd225fa807e"}381{"id": "distance_band-000380", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,6), goal at (15,15).\n..#.............\n..##.#..#.#.#..#\n...#..#.........\n.......#.....#.#\n#.....###.#..###\n###..#......##..\n..#.###...#.....\n.....#....###.#.\n.#...#.....#....\n.#...#A.#.#.....\n.##....#........\n.....##.........\n....#...#.....#.\n....##....#...#.\n..#.##...#..#.#.\n#...........###G", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "54e9fba745b0f3927c8a5dcabe8fceef41f10eab5f24c7b5fd7689375e138877"}382{"id": "first_move-000381", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,7), goal at (7,23).\n####################################\n#...#...#.....#.....#.....#.......##\n#.#.###.#.#.###.#.###.###.#####.####\n#.#...#...#.#...#.....#...#...#...##\n#.###.#.###.#.#########.###.#.###.##\n#.#...#.#...#.#.......#.#...#...#.##\n#.#.###.#.#.#.#####.#.#.#.#####.#.##\n#.#.#..A#.#.#.....#.#.#G#.#...#...##\n#.#.#####.#######.#.#.###.#.#.###.##\n#.#.#...#.....#...#.#...#.#.#...#.##\n#.#.#.#.#####.#.#####.#.#.#.###.#.##\n#.#...#.......#.......#...#...#.#.##\n#.###########################.#.#.##\n#.........#...........#.......#.#.##\n#########.#.#####.#####.###.###.#.##\n#.........#.#...#.....#.#.#...#.#.##\n#.#.#######.#.#######.#.#.###.###.##\n#.#.#.....#.#.#.....#...#...#...#.##\n#.#.#.###.#.#.#.###.#######.###.#.##\n#.#.#.#.#...#...#...#.......#...#.##\n#.#.#.#.#######.#.#.#.#.###.#.###.##\n#.#.#...#.....#.#.#.#.#.#...#.....##\n#.#####.#.###.###.###.#.#.##########\n#.#...#...#.#...#.....#.#.#...#...##\n#.#.#.#####.###.#######.#.#.#.###.##\n#.#.#.....#.....#.......#...#.#...##\n#.#.#####.#.#####.###########.#.####\n#...#.#...#.......#.#.......#.#...##\n#.###.#.#.#########.#.#####.#.#.#.##\n#.#...#.#.#.#.....#.....#...#.#.#.##\n#.#.###.#.#.#.#.#.#.#####.#.#.###.##\n#...#...#...#.#.#.#.#.....#.#.#...##\n###.#.#######.#.###.#.#######.#.#.##\n#...#.........#.....#...........#.##\n####################################\n####################################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "d9b01eb3d879db3800e55a22af8531a012a79e0af261f04daf2de02f3b06500c"}383{"id": "snake_safe-000382", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,4), travelling west, length 3.\n..........\n..........\n..........\n..........\n..........\n..........\n....Hoo...\n..........\n..........\n.........*", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "8fbdbc8aabf69373983ad35e24abcedaccc713c6a574594d9d3f167fc7fe787e"}384{"id": "stochastic-000383", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 2, column 1. Rows increase to the south and columns to the east.\nRow 0: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall\nRow 1: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open\nRow 2: column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is wall, column 3 is open\nRow 3: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open\nRow 4: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall", "question": "The actuator executes the intended move with probability 0.75; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.8333333333333334, 0.16666666666666666], "deterministic": false, "fingerprint": "6890828cba54f11a9e37c3df766c75c0734c08bea013749315369b07c2a2d2f9"}385{"id": "cell_kind-000384", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (2,25); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXX\n######X\n..#.##X\n#.#A##X\n#.#.##X\n#.#.##X\n#.#.##X", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "0570505935d4b8f67f05ed649824f6b68662f80f163d0657134b1450b685af53"}386{"id": "move_two_step-000385", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 5. Rows increase to the south and columns to the east.\nRow -2: column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow -1: column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow 0: column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall\nRow 1: column 2 is open, column 3 is open, column 4 is open, column 5 is the agent, column 6 is wall, column 7 is open, column 8 is wall\nRow 2: column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 3: column 2 is wall, column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 4: column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall", "question": "The agent plans to move north, then east. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "ebb980fe994a053806eef71fc91098507b76947b5f9dd580ad6e6a834cc2fcae"}387{"id": "distance_band-000386", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (10,9), goal at (3,11).\n################\n#.............##\n#.###########.##\n#..........G#.##\n#############.##\n#.............##\n#.###########.##\n#.#.........#.##\n#.###.###.#.#.##\n#...#...#.#.#.##\n#########A#.#.##\n#.........#.#.##\n#.###########.##\n#.............##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 4, "answer": "40 moves or more", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "9744cf30199ccb7b98a21fbe076470a5b100a46c79959e164e3e39c039672228"}388{"id": "reachable_within-000387", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,22), goal at (3,23).\n################################\n#.....#...#.........#.........##\n#.#.#.###.#.#####.#.#.#######.##\n#.#.#...#.....#...#...#G....#.##\n###.###.#######.###########.#.##\n#...#.........#.....#.....#...##\n#.###########.#.###.#####.######\n#...........#.#...#.#.A.#.#...##\n#.#########.#.#####.#.#.#.#.#.##\n#...#.....#.#.......#.#...#.#.##\n###.###.#.#.#######.#.#####.#.##\n#.#.....#.#.#...#...#...#...#.##\n#.#.#######.#.#.#.#####.#.###.##\n#...#.......#.#.......#...#...##\n#.###.#####.###############.####\n#...#.#...#.#...........#.#.#.##\n#####.#.#.#.#.#######.#.#.#.#.##\n#.....#.#...#...#.....#.#.#...##\n#.#####.#######.#.#####.#.###.##\n#.#...#.#.#...#.#.....#...#...##\n#.#.#.#.#.#.#.#.#####.###.#.####\n#.#.#...#...#.#.#.#...#...#.#.##\n#.#.#########.#.#.#.#######.#.##\n#.#.#.........#...#...#...#...##\n#.#.###.#.#######.###.#.#.###.##\n#.#.....#.#.....#...#.#.#.....##\n#.#########.#.###.###.#.########\n#...#...#...#.....#...#.#...#.##\n#.#.#.#.#.#########.###.#.#.#.##\n#.#...#...........#.......#...##\n################################\n################################", "question": "Can the agent reach the goal in at most 258 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "ccd4da6cde8cf77bd166ee4d96a566bb13bb2a75871200eb9fb80eae697cb063"}389{"id": "move_two_step-000388", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,33); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n######X\n....##X\n###.##X\n..#A##X\n###.##X\n....##X\n######X", "question": "The agent plans to move south, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "631fd7699f944db15a665842cdd59f309c214332199e9145bf533509fefe0d8d"}390{"id": "only_way-000389", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 17, column 7. Rows increase to the south and columns to the east.\nRow 15: column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open\nRow 16: column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open\nRow 17: column 5 is open, column 6 is wall, column 7 is the agent, column 8 is wall, column 9 is open\nRow 18: column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open\nRow 19: column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is open", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3278a8bb0ec281f4724f687665e5b8d3e5c57924940a9149da1d5ee8196da319"}391{"id": "only_way-000390", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 15, column 3. Rows increase to the south and columns to the east.\nRow 13: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 14: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 15: column 1 is open, column 2 is wall, column 3 is the agent, column 4 is wall, column 5 is open\nRow 16: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 17: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 5, "answer": "more than one of them", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "e179795ff896ec4f024ad741314d29ac804099165d3c3b7c84e19259f1f541d2"}392{"id": "snake_safe-000391", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,3), travelling west, length 3.\n..........\n..........\n..........\n..........\n..........\n..........\n...Hoo....\n...*......\n..........\n..........", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "b839df00ee4e3910e10d72b7ced332721cc39441e4b996a6107563058c8cad41"}393{"id": "cell_kind-000392", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 26, column 24. Rows increase to the south and columns to the east.\nRow 24: column 22 is wall, column 23 is open, column 24 is open, column 25 is wall, column 26 is open\nRow 25: column 22 is wall, column 23 is wall, column 24 is open, column 25 is wall, column 26 is open\nRow 26: column 22 is wall, column 23 is open, column 24 is the agent, column 25 is open, column 26 is open\nRow 27: column 22 is open, column 23 is open, column 24 is wall, column 25 is wall, column 26 is wall\nRow 28: column 22 is outside the maze, column 23 is outside the maze, column 24 is outside the maze, column 25 is outside the maze, column 26 is outside the maze", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "3a06ac16816091ef193c34f873a02de5d2aa51986ea8f3bd1990e585043004e4"}394{"id": "onward-000393", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (19,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.#.#.#\n#.#.#..\n#.#.###\n#..A...\n#######\nXXXXXXX\nXXXXXXX", "question": "Suppose the agent moves one cell east. From that new cell, would a further step north be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "99a95599f661b305156e744325eb5b99a9f343312b3f022a7fbec1fbb1bb049e"}395{"id": "only_way-000394", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"agent\",\"1,2\":\"wall\",\"1,3\":\"open\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"open\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "eac8ee437e48037662183fdea0aff1d0e54944ab0137af7e9740f9a7ed97cfe6"}396{"id": "move_two_step-000395", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (13,19); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#.#...#.\n##.###.#.\n.....#.#.\n######.#.\n.#..A..#.\n.#######.\n.#.....#.\n##.#.#.#.\n...#.#.#.", "question": "The agent plans to move west, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "518550b23e5f736bc48a9d754d1302a9b2851d64eecd6bf17e17028fde6567c6"}397{"id": "cell_kind-000396", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[0,25],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-3,22\":\"outside\",\"-3,23\":\"outside\",\"-3,24\":\"outside\",\"-3,25\":\"outside\",\"-3,26\":\"outside\",\"-3,27\":\"outside\",\"-3,28\":\"outside\",\"-2,22\":\"outside\",\"-2,23\":\"outside\",\"-2,24\":\"outside\",\"-2,25\":\"outside\",\"-2,26\":\"outside\",\"-2,27\":\"outside\",\"-2,28\":\"outside\",\"-1,22\":\"outside\",\"-1,23\":\"outside\",\"-1,24\":\"outside\",\"-1,25\":\"outside\",\"-1,26\":\"outside\",\"-1,27\":\"outside\",\"-1,28\":\"outside\",\"0,22\":\"open\",\"0,23\":\"open\",\"0,24\":\"open\",\"0,25\":\"agent\",\"0,26\":\"open\",\"0,27\":\"open\",\"0,28\":\"outside\",\"1,22\":\"open\",\"1,23\":\"open\",\"1,24\":\"open\",\"1,25\":\"open\",\"1,26\":\"open\",\"1,27\":\"open\",\"1,28\":\"outside\",\"2,22\":\"open\",\"2,23\":\"open\",\"2,24\":\"open\",\"2,25\":\"open\",\"2,26\":\"open\",\"2,27\":\"open\",\"2,28\":\"outside\",\"3,22\":\"open\",\"3,23\":\"open\",\"3,24\":\"open\",\"3,25\":\"open\",\"3,26\":\"wall\",\"3,27\":\"wall\",\"3,28\":\"outside\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "f71a9618367d549b6d89c59a9b49130a211e46e0e7780705e387fde4dc9f111f"}398{"id": "only_way-000397", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[17,13],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"13,9\":\"open\",\"13,10\":\"open\",\"13,11\":\"open\",\"13,12\":\"open\",\"13,13\":\"open\",\"13,14\":\"open\",\"13,15\":\"open\",\"13,16\":\"open\",\"13,17\":\"open\",\"14,9\":\"wall\",\"14,10\":\"wall\",\"14,11\":\"wall\",\"14,12\":\"wall\",\"14,13\":\"wall\",\"14,14\":\"wall\",\"14,15\":\"wall\",\"14,16\":\"wall\",\"14,17\":\"wall\",\"15,9\":\"open\",\"15,10\":\"wall\",\"15,11\":\"open\",\"15,12\":\"open\",\"15,13\":\"open\",\"15,14\":\"open\",\"15,15\":\"open\",\"15,16\":\"open\",\"15,17\":\"open\",\"16,9\":\"open\",\"16,10\":\"wall\",\"16,11\":\"open\",\"16,12\":\"wall\",\"16,13\":\"wall\",\"16,14\":\"wall\",\"16,15\":\"wall\",\"16,16\":\"wall\",\"16,17\":\"open\",\"17,9\":\"open\",\"17,10\":\"wall\",\"17,11\":\"open\",\"17,12\":\"wall\",\"17,13\":\"agent\",\"17,14\":\"open\",\"17,15\":\"open\",\"17,16\":\"wall\",\"17,17\":\"open\",\"18,9\":\"open\",\"18,10\":\"wall\",\"18,11\":\"open\",\"18,12\":\"wall\",\"18,13\":\"wall\",\"18,14\":\"wall\",\"18,15\":\"open\",\"18,16\":\"wall\",\"18,17\":\"open\",\"19,9\":\"open\",\"19,10\":\"wall\",\"19,11\":\"open\",\"19,12\":\"open\",\"19,13\":\"open\",\"19,14\":\"open\",\"19,15\":\"open\",\"19,16\":\"wall\",\"19,17\":\"open\",\"20,9\":\"wall\",\"20,10\":\"wall\",\"20,11\":\"wall\",\"20,12\":\"wall\",\"20,13\":\"wall\",\"20,14\":\"wall\",\"20,15\":\"wall\",\"20,16\":\"wall\",\"20,17\":\"wall\",\"21,9\":\"outside\",\"21,10\":\"outside\",\"21,11\":\"outside\",\"21,12\":\"outside\",\"21,13\":\"outside\",\"21,14\":\"outside\",\"21,15\":\"outside\",\"21,16\":\"outside\",\"21,17\":\"outside\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "603fc8ab09ba5face6ae52c78f7ebe093bcdf6e37379eea53c22267506480627"}399{"id": "stochastic-000398", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (19,11); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n...#.\n.###.\n.#A#.\n##.#.\n...#.", "question": "The actuator executes the intended move with probability 0.75; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.25, 0.75], "deterministic": false, "fingerprint": "3804e2901de65d51a1b2cd78693a1a2cdf06fa33c4d8cef52c73150c911a20a3"}400{"id": "plan_survives-000399", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[8,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"open\",\"4,8\":\"wall\",\"4,9\":\"open\",\"4,10\":\"wall\",\"4,11\":\"open\",\"4,12\":\"wall\",\"4,13\":\"open\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"wall\",\"5,9\":\"open\",\"5,10\":\"wall\",\"5,11\":\"open\",\"5,12\":\"wall\",\"5,13\":\"open\",\"6,5\":\"open\",\"6,6\":\"wall\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"open\",\"6,10\":\"wall\",\"6,11\":\"open\",\"6,12\":\"wall\",\"6,13\":\"open\",\"7,5\":\"open\",\"7,6\":\"wall\",\"7,7\":\"open\",\"7,8\":\"wall\",\"7,9\":\"open\",\"7,10\":\"wall\",\"7,11\":\"open\",\"7,12\":\"wall\",\"7,13\":\"open\",\"8,5\":\"wall\",\"8,6\":\"wall\",\"8,7\":\"open\",\"8,8\":\"wall\",\"8,9\":\"agent\",\"8,10\":\"wall\",\"8,11\":\"open\",\"8,12\":\"wall\",\"8,13\":\"open\",\"9,5\":\"open\",\"9,6\":\"wall\",\"9,7\":\"open\",\"9,8\":\"wall\",\"9,9\":\"open\",\"9,10\":\"wall\",\"9,11\":\"open\",\"9,12\":\"wall\",\"9,13\":\"open\",\"10,5\":\"open\",\"10,6\":\"wall\",\"10,7\":\"open\",\"10,8\":\"wall\",\"10,9\":\"wall\",\"10,10\":\"wall\",\"10,11\":\"open\",\"10,12\":\"wall\",\"10,13\":\"open\",\"11,5\":\"open\",\"11,6\":\"open\",\"11,7\":\"open\",\"11,8\":\"open\",\"11,9\":\"open\",\"11,10\":\"open\",\"11,11\":\"open\",\"11,12\":\"wall\",\"11,13\":\"open\",\"12,5\":\"wall\",\"12,6\":\"wall\",\"12,7\":\"wall\",\"12,8\":\"wall\",\"12,9\":\"wall\",\"12,10\":\"wall\",\"12,11\":\"wall\",\"12,12\":\"wall\",\"12,13\":\"open\"}}", "question": "The agent will attempt this plan in order: west, north, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "d9571d185eb6147096274346ecf8532487f7480e302169c789093c271b2818e4"}401{"id": "move_two_step-000400", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[5,14],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"2,11\":\"wall\",\"2,12\":\"wall\",\"2,13\":\"wall\",\"2,14\":\"wall\",\"2,15\":\"wall\",\"2,16\":\"wall\",\"2,17\":\"wall\",\"3,11\":\"open\",\"3,12\":\"open\",\"3,13\":\"open\",\"3,14\":\"open\",\"3,15\":\"open\",\"3,16\":\"open\",\"3,17\":\"open\",\"4,11\":\"wall\",\"4,12\":\"wall\",\"4,13\":\"wall\",\"4,14\":\"wall\",\"4,15\":\"wall\",\"4,16\":\"wall\",\"4,17\":\"open\",\"5,11\":\"open\",\"5,12\":\"open\",\"5,13\":\"open\",\"5,14\":\"agent\",\"5,15\":\"open\",\"5,16\":\"wall\",\"5,17\":\"open\",\"6,11\":\"wall\",\"6,12\":\"wall\",\"6,13\":\"wall\",\"6,14\":\"wall\",\"6,15\":\"wall\",\"6,16\":\"wall\",\"6,17\":\"open\",\"7,11\":\"open\",\"7,12\":\"open\",\"7,13\":\"open\",\"7,14\":\"open\",\"7,15\":\"open\",\"7,16\":\"open\",\"7,17\":\"open\",\"8,11\":\"wall\",\"8,12\":\"wall\",\"8,13\":\"wall\",\"8,14\":\"wall\",\"8,15\":\"wall\",\"8,16\":\"wall\",\"8,17\":\"wall\"}}", "question": "The agent plans to move north, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "bf8d1cafc2c62cec31086eb141ff7bafd3c00323c6023bfef5bf6337fdb40017"}402{"id": "snake_food-000401", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,7), travelling east, length 3.\n............\n............\n............\n............\n............\n............\n............\n............\n.....ooH....\n............\n............\n.......*....", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 1, "answer": "south", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "0096d5d3b4d5b53d959d883824aa47623a42d4380c2850da82c1c0247e75cdad"}403{"id": "blocked_pair-000402", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[9,13],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"5,9\":\"open\",\"5,10\":\"wall\",\"5,11\":\"open\",\"5,12\":\"wall\",\"5,13\":\"open\",\"5,14\":\"wall\",\"5,15\":\"wall\",\"5,16\":\"outside\",\"5,17\":\"outside\",\"6,9\":\"open\",\"6,10\":\"wall\",\"6,11\":\"wall\",\"6,12\":\"wall\",\"6,13\":\"open\",\"6,14\":\"wall\",\"6,15\":\"wall\",\"6,16\":\"outside\",\"6,17\":\"outside\",\"7,9\":\"open\",\"7,10\":\"open\",\"7,11\":\"open\",\"7,12\":\"open\",\"7,13\":\"open\",\"7,14\":\"wall\",\"7,15\":\"wall\",\"7,16\":\"outside\",\"7,17\":\"outside\",\"8,9\":\"wall\",\"8,10\":\"open\",\"8,11\":\"wall\",\"8,12\":\"wall\",\"8,13\":\"open\",\"8,14\":\"wall\",\"8,15\":\"wall\",\"8,16\":\"outside\",\"8,17\":\"outside\",\"9,9\":\"open\",\"9,10\":\"open\",\"9,11\":\"open\",\"9,12\":\"wall\",\"9,13\":\"agent\",\"9,14\":\"wall\",\"9,15\":\"wall\",\"9,16\":\"outside\",\"9,17\":\"outside\",\"10,9\":\"wall\",\"10,10\":\"wall\",\"10,11\":\"open\",\"10,12\":\"wall\",\"10,13\":\"open\",\"10,14\":\"wall\",\"10,15\":\"wall\",\"10,16\":\"outside\",\"10,17\":\"outside\",\"11,9\":\"open\",\"11,10\":\"wall\",\"11,11\":\"open\",\"11,12\":\"wall\",\"11,13\":\"open\",\"11,14\":\"wall\",\"11,15\":\"wall\",\"11,16\":\"outside\",\"11,17\":\"outside\",\"12,9\":\"open\",\"12,10\":\"wall\",\"12,11\":\"open\",\"12,12\":\"wall\",\"12,13\":\"open\",\"12,14\":\"wall\",\"12,15\":\"wall\",\"12,16\":\"outside\",\"12,17\":\"outside\",\"13,9\":\"open\",\"13,10\":\"wall\",\"13,11\":\"open\",\"13,12\":\"open\",\"13,13\":\"open\",\"13,14\":\"wall\",\"13,15\":\"wall\",\"13,16\":\"outside\",\"13,17\":\"outside\"}}", "question": "Are BOTH the cell one step south and the cell one step west blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "76ab320be7585a3deff7614218703d5e3dc08d16bdfa8a32a82bedbff90cf998"}404{"id": "goal_bearing-000403", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (15,14), goal at (7,13).\n#####################\n#...................#\n#.#################.#\n#.#...............#.#\n#.#.#############.#.#\n#.#.#...........#.#.#\n#.#.#.#########.#.#.#\n#.#.#.#......G#.#.#.#\n#.#.#.#.#######.#.#.#\n#.#.#.#.........#.#.#\n#.#.#.###########.#.#\n#.#.#.............#.#\n#.#.###############.#\n#.#.................#\n#.#.#################\n#.#.#.......#.A.....#\n#.#.#.#####.#######.#\n#.#.#.#...#.......#.#\n#.###.###.#.#######.#\n#.........#.........#\n#####################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "f58e86731c435aca901865d6dd36a474f54577f90e20ac96fbed4cf228a896ba"}405{"id": "reachable_within-000404", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,21), goal at (11,5).\n########################\n#...........#.........##\n#.###.#.###.#.###.#.####\n#...#.#.#.#.....#.....##\n#.#.###.#.###.#####.#.##\n#.#...............#...##\n#.#.#.###.#.#..##.#.#.##\n#.#...#...#.#...#.#.#.##\n#.#.###.###.###.#.#.#.##\n#.#...........#.....#A##\n#.#########.#.###.#.#.##\n#.#..G..#...#...#...#.##\n#.##.##.#.#.###.#.###.##\n#.......#.#.....#.....##\n#.#.###.#.#.#.###.######\n#.#.....#...#...#...#.##\n#.###.###.#.#.#.###.#.##\n#...#.......#.....#...##\n#.#.#.#.###.#.#.#.###.##\n#.#...#.....#.........##\n#.###.##.##.#.###.#.####\n#.........#.....#.....##\n########################\n########################", "question": "Can the agent reach the goal in at most 49 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "c7f0752493714afa6b676ee05f636a3a573a46e8eafbd14ce924e9bcdf860d85"}406{"id": "move_legal-000405", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 0, column 35. Rows increase to the south and columns to the east.\nRow -2: column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze, column 36 is outside the maze, column 37 is outside the maze\nRow -1: column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze, column 36 is outside the maze, column 37 is outside the maze\nRow 0: column 33 is open, column 34 is open, column 35 is the agent, column 36 is outside the maze, column 37 is outside the maze\nRow 1: column 33 is open, column 34 is open, column 35 is open, column 36 is outside the maze, column 37 is outside the maze\nRow 2: column 33 is open, column 34 is open, column 35 is open, column 36 is outside the maze, column 37 is outside the maze", "question": "Is the cell one step east of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "6bd84d5d0c3167788770f2183c0480f8e5a901e64f939bfd77b3fec049d8a7be"}407{"id": "blocked_pair-000406", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (5,9); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.........\n.#######.\n.#.......\n####.####\n...#A#...\n.#.###.#.\n.#.#...#.\n##.#####.\n.#...#...", "question": "Are BOTH the cell one step south and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "180f224a0adc43944f8addca4c628c92f53891ccd7e207147eb24d67c940a5c0"}408{"id": "snake_safe-000407", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,4), travelling south, length 3.\n........\n........\n........\n........\n...oo...\n....H...\n....*...\n........", "question": "If the snake moves north on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "485bb72c7372abaf8f1656d80eb0ada38021509a05da5d8faa3e4afe89d3fbf5"}409{"id": "distance_band-000408", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,6), goal at (5,5).\n################\n#.#...#.#.....##\n#.#.#.#.#.###.##\n#...#.#...#.#.##\n#.###.#.###.#.##\n#.#.#G#.#...#.##\n#.#.###.#.###.##\n#...#.#...#...##\n###.#.###.#.#.##\n#.#.#.A...#.#.##\n#.#.#.#####.#.##\n#.#.#.#.....#.##\n#.#.###.#####.##\n#.......#.....##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 4, "answer": "40 moves or more", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "216c3ae62f4c18c46f8f0cbf5ccd06ef95fcf6515cca34f9f0c5373e8074ebee"}410{"id": "distance_band-000409", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,9), goal at (1,7).\n################\n#......G#A....##\n###.#####.###.##\n#.#.......#.#.##\n#.#####.###.#.##\n#.....#.....#.##\n#####.#.#####.##\n#...#.#.#.....##\n###.#.###.###.##\n#...#.....#...##\n#.#.#######.####\n#.#.#.....#...##\n#.###.###.###.##\n#.......#.....##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a728882931373542b8c24a5c3018d0463f77acf62ed64546c95a931c5da768c6"}411{"id": "move_two_step-000410", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 13. Rows increase to the south and columns to the east.\nRow -3: column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze\nRow -2: column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze\nRow -1: column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze\nRow 0: column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 1: column 9 is open, column 10 is wall, column 11 is open, column 12 is open, column 13 is the agent, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 2: column 9 is open, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 3: column 9 is open, column 10 is wall, column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 4: column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 5: column 9 is open, column 10 is wall, column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze", "question": "The agent plans to move west, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "2879942e89c427067fc5c644b472590b41f7959108eff058fbaa82cd35b31410"}412{"id": "move_two_step-000411", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[21,17],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"19,15\":\"open\",\"19,16\":\"open\",\"19,17\":\"open\",\"19,18\":\"open\",\"19,19\":\"open\",\"20,15\":\"open\",\"20,16\":\"wall\",\"20,17\":\"wall\",\"20,18\":\"wall\",\"20,19\":\"open\",\"21,15\":\"open\",\"21,16\":\"wall\",\"21,17\":\"agent\",\"21,18\":\"wall\",\"21,19\":\"open\",\"22,15\":\"open\",\"22,16\":\"wall\",\"22,17\":\"open\",\"22,18\":\"wall\",\"22,19\":\"open\",\"23,15\":\"open\",\"23,16\":\"wall\",\"23,17\":\"open\",\"23,18\":\"wall\",\"23,19\":\"open\"}}", "question": "The agent plans to move south, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "3256fb6b4abba064ff536da08904bf3f8f75d19c85d89c6005454cc9394e9b46"}413{"id": "cell_kind-000412", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[0,0],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-3,-3\":\"outside\",\"-3,-2\":\"outside\",\"-3,-1\":\"outside\",\"-3,0\":\"outside\",\"-3,1\":\"outside\",\"-3,2\":\"outside\",\"-3,3\":\"outside\",\"-2,-3\":\"outside\",\"-2,-2\":\"outside\",\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-1,-3\":\"outside\",\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"0,-3\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"agent\",\"0,1\":\"open\",\"0,2\":\"open\",\"0,3\":\"open\",\"1,-3\":\"outside\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"wall\",\"1,2\":\"open\",\"1,3\":\"open\",\"2,-3\":\"outside\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"open\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"open\",\"3,-3\":\"outside\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"open\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"open\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "db35c6e3509d9f17dfa4ffe290a6f4ba34ebdce19ca4fcb0c47e0a613ebf5bed"}414{"id": "only_way-000413", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,33); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.....X\n..#...X\n.#....X\n.##A#.X\n...##.X\n#.....X\n....#.X", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "d0cb84dd2a34763498cc96e93c3e0fc54dd03bc772b0ee849ca1ee6ba151569d"}415{"id": "cell_kind-000414", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX\nXXX......\nXXX.A..#.\nXXX#.....\nXXX......\nXXX......\nXXX...##.", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "729642b44ecdddd816086f5fd54491959e54b7d378c26f21045c3f6b214c974b"}416{"id": "onward-000415", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[5,6],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"1,6\":\"open\",\"1,7\":\"open\",\"1,8\":\"open\",\"1,9\":\"open\",\"1,10\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"2,6\":\"wall\",\"2,7\":\"wall\",\"2,8\":\"wall\",\"2,9\":\"wall\",\"2,10\":\"wall\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"3,6\":\"open\",\"3,7\":\"open\",\"3,8\":\"open\",\"3,9\":\"open\",\"3,10\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"wall\",\"4,8\":\"wall\",\"4,9\":\"wall\",\"4,10\":\"wall\",\"5,2\":\"wall\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"agent\",\"5,7\":\"open\",\"5,8\":\"open\",\"5,9\":\"open\",\"5,10\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"6,7\":\"wall\",\"6,8\":\"wall\",\"6,9\":\"wall\",\"6,10\":\"wall\",\"7,2\":\"wall\",\"7,3\":\"open\",\"7,4\":\"open\",\"7,5\":\"open\",\"7,6\":\"open\",\"7,7\":\"open\",\"7,8\":\"open\",\"7,9\":\"open\",\"7,10\":\"open\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"wall\",\"8,6\":\"wall\",\"8,7\":\"wall\",\"8,8\":\"wall\",\"8,9\":\"wall\",\"8,10\":\"wall\",\"9,2\":\"open\",\"9,3\":\"open\",\"9,4\":\"open\",\"9,5\":\"open\",\"9,6\":\"open\",\"9,7\":\"open\",\"9,8\":\"open\",\"9,9\":\"open\",\"9,10\":\"open\"}}", "question": "Suppose the agent moves one cell east. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "4a37768654ef700e4dedbe6f1799120cfafc1a43c08e188d10b3791424d119e2"}417{"id": "move_two_step-000416", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[7,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"agent\",\"7,2\":\"wall\",\"7,3\":\"open\",\"8,-1\":\"outside\",\"8,0\":\"wall\",\"8,1\":\"wall\",\"8,2\":\"wall\",\"8,3\":\"open\",\"9,-1\":\"outside\",\"9,0\":\"wall\",\"9,1\":\"open\",\"9,2\":\"wall\",\"9,3\":\"open\"}}", "question": "The agent plans to move north, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "ab6433f0b8a6d06c7cf4b10fce01e045a94ce37d717d5f6b31d751234aeb9de4"}418{"id": "snake_food-000417", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (11,6), travelling south, length 3.\n........*.....\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n......o.......\n......o.......\n......H.......\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "ca8358936ce89c1db8b96c0b23b8a685fd2dfa55c96471fc4f3c8b727174e604"}419{"id": "move_two_step-000418", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXX\n####X\n..A#X\n##.#X\n.#.#X", "question": "The agent plans to move east, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "34861146c38049b7460e659ec29e6054d8ef4cdd841a5ee00172db7878e147be"}420{"id": "snake_food-000419", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (2,4), travelling west, length 3.\n........\n........\n.*..Ho..\n.....o..\n........\n........\n........\n........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "78f67e126b3ac4c0e29c83529716dba380fa19801af5be5ea17bcc71ee1499e2"}421{"id": "move_two_step-000420", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[10,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"6,5\":\"open\",\"6,6\":\"wall\",\"6,7\":\"wall\",\"6,8\":\"wall\",\"6,9\":\"open\",\"6,10\":\"wall\",\"6,11\":\"open\",\"6,12\":\"wall\",\"6,13\":\"open\",\"7,5\":\"open\",\"7,6\":\"wall\",\"7,7\":\"open\",\"7,8\":\"open\",\"7,9\":\"open\",\"7,10\":\"wall\",\"7,11\":\"open\",\"7,12\":\"wall\",\"7,13\":\"open\",\"8,5\":\"wall\",\"8,6\":\"wall\",\"8,7\":\"open\",\"8,8\":\"wall\",\"8,9\":\"wall\",\"8,10\":\"wall\",\"8,11\":\"open\",\"8,12\":\"wall\",\"8,13\":\"wall\",\"9,5\":\"open\",\"9,6\":\"wall\",\"9,7\":\"open\",\"9,8\":\"open\",\"9,9\":\"open\",\"9,10\":\"wall\",\"9,11\":\"open\",\"9,12\":\"wall\",\"9,13\":\"open\",\"10,5\":\"open\",\"10,6\":\"wall\",\"10,7\":\"wall\",\"10,8\":\"wall\",\"10,9\":\"agent\",\"10,10\":\"wall\",\"10,11\":\"open\",\"10,12\":\"wall\",\"10,13\":\"open\",\"11,5\":\"open\",\"11,6\":\"open\",\"11,7\":\"open\",\"11,8\":\"wall\",\"11,9\":\"open\",\"11,10\":\"wall\",\"11,11\":\"open\",\"11,12\":\"wall\",\"11,13\":\"open\",\"12,5\":\"wall\",\"12,6\":\"wall\",\"12,7\":\"wall\",\"12,8\":\"wall\",\"12,9\":\"wall\",\"12,10\":\"wall\",\"12,11\":\"open\",\"12,12\":\"wall\",\"12,13\":\"open\",\"13,5\":\"open\",\"13,6\":\"wall\",\"13,7\":\"open\",\"13,8\":\"open\",\"13,9\":\"open\",\"13,10\":\"open\",\"13,11\":\"open\",\"13,12\":\"wall\",\"13,13\":\"open\",\"14,5\":\"open\",\"14,6\":\"wall\",\"14,7\":\"open\",\"14,8\":\"wall\",\"14,9\":\"wall\",\"14,10\":\"wall\",\"14,11\":\"wall\",\"14,12\":\"wall\",\"14,13\":\"wall\"}}", "question": "The agent plans to move west, then east. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "6779f1b8dd33fa0bad022fc85c01393f63a26137e58e8bcb2058d4ce8520cced"}422{"id": "reachable_within-000421", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,12), goal at (13,1).\n################\n#.............##\n#.#.#########.##\n#.#.#.......#.##\n#.#.#.#####.#.##\n#.#.#.#...#.#.##\n#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.##\n#.#.#.#.###.#.##\n#.#.#.#.....#.##\n#.#.#.#######.##\n#G#.#.......A.##\n################\n################", "question": "Can the agent reach the goal in at most 38 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "c7f6e0b456bf97825c7cffcdfb737e04f8e46d49f2d1d212d14270fed40b738c"}423{"id": "snake_safe-000422", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (10,7), travelling south, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n.....*.o......\n.......o......\n.......H......\n..............\n..............\n..............", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "9a9c60b576ab041aeaee715e0cccd5319e4201be84ea37506b77a0442919e2b1"}424{"id": "snake_food-000423", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (3,6), travelling east, length 3.\n........\n........\n........\n....ooH*\n........\n........\n........\n........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "8a754970df3035b946dae96abbdc47d7831e98f79deca9456d0a88d68dd861c6"}425{"id": "blocked_pair-000424", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 3, column 10. Rows increase to the south and columns to the east.\nRow -1: column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze, column 14 is outside the maze\nRow 0: column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall\nRow 1: column 6 is open, column 7 is open, column 8 is wall, column 9 is open, column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is wall\nRow 2: column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open, column 14 is wall\nRow 3: column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is the agent, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall\nRow 4: column 6 is wall, column 7 is open, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open, column 14 is wall\nRow 5: column 6 is wall, column 7 is open, column 8 is open, column 9 is open, column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is wall\nRow 6: column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall\nRow 7: column 6 is wall, column 7 is open, column 8 is open, column 9 is open, column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is wall", "question": "Are BOTH the cell one step south and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "ab55c5169be2429213b441d9f3fb1c86e5341d2d437e3aa7130785bfc321d5f5"}426{"id": "snake_food-000425", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (3,4), travelling north, length 3.\n........\n........\n........\n....H*..\n...oo...\n........\n........\n........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a66f3ca48e146bd6842305f67286ff756d28fca9efa1828ad2925271679030a7"}427{"id": "snake_food-000426", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,2), travelling south, length 3.\n..........\n..........\n..........\n..........\n..........\n..........\n..........\n..oo....*.\n..H.......\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "af564b2aaaf797ede4bf9b620244c5962d565e300a26baf0a56cd1b799411315"}428{"id": "onward-000427", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[7,19],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"5,17\":\"open\",\"5,18\":\"open\",\"5,19\":\"open\",\"5,20\":\"open\",\"5,21\":\"open\",\"6,17\":\"open\",\"6,18\":\"wall\",\"6,19\":\"wall\",\"6,20\":\"wall\",\"6,21\":\"open\",\"7,17\":\"open\",\"7,18\":\"open\",\"7,19\":\"agent\",\"7,20\":\"wall\",\"7,21\":\"open\",\"8,17\":\"wall\",\"8,18\":\"wall\",\"8,19\":\"wall\",\"8,20\":\"wall\",\"8,21\":\"open\",\"9,17\":\"open\",\"9,18\":\"open\",\"9,19\":\"open\",\"9,20\":\"open\",\"9,21\":\"open\"}}", "question": "Suppose the agent moves one cell west. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "60e4710ffb046452740d2bbd61e7885665780909e11cb87915e4fc24a99c44f9"}429{"id": "snake_food-000428", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,7), travelling east, length 3.\n....*.......\n............\n............\n............\n............\n............\n.....ooH....\n............\n............\n............\n............\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "98c4f9ecb340f4936f5f9d775b47796663b0ec091512ab4628e0f324e16c5743"}430{"id": "goal_bearing-000429", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (8,3), goal at (11,3).\n################\n#.............##\n#.###########.##\n#.#.........#.##\n#.#.###.###.#.##\n#.#...#.#.#.#.##\n#.###.#.#.#.#.##\n#.#.#.#.#.#.#.##\n#.#A#.#.#.#.#.##\n#...#.#.#.#.#.##\n#####.#.#.#.#.##\n#..G#.#.#...#.##\n#.###.#.#####.##\n#.....#.......##\n################\n################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 1, "answer": "south", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "15b6e5dc27922f178e90c8ec95994de32d589c73e1e1ac16ab7fb9a341776d42"}431{"id": "move_two_step-000430", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 1. Rows increase to the south and columns to the east.\nRow 4: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is open, column 4 is wall\nRow 5: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is wall\nRow 6: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall\nRow 7: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is open, column 3 is open, column 4 is wall\nRow 8: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall\nRow 9: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open\nRow 10: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall", "question": "The agent plans to move west, then east. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "b61f5c61b4f3e893a653e6d7bc7fa3a4a8e43fab7b6b447851ed86449c57e066"}432{"id": "snake_safe-000431", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,9), travelling north, length 3.\n........*.....\n..............\n..............\n..............\n..............\n.........H....\n........oo....\n..............\n..............\n..............\n..............\n..............\n..............\n..............", "question": "If the snake moves east on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "9784a83b3c50ecaab38b60bb86f07afe93b0ce2d86a35e984bd5ab490d18142a"}433{"id": "onward-000432", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 6, column 6. Rows increase to the south and columns to the east.\nRow 3: column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is wall\nRow 4: column 3 is open, column 4 is wall, column 5 is wall, column 6 is open, column 7 is wall, column 8 is open, column 9 is wall\nRow 5: column 3 is wall, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open\nRow 6: column 3 is open, column 4 is wall, column 5 is wall, column 6 is the agent, column 7 is wall, column 8 is wall, column 9 is open\nRow 7: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is open, column 7 is wall, column 8 is wall, column 9 is wall\nRow 8: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall\nRow 9: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open", "question": "Suppose the agent moves one cell north. From that new cell, would a further step north be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "dd4b1af59924988744610ddd237d161d06b231509d63f12df9f6b32996090e3e"}434{"id": "goal_bearing-000433", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (16,3), goal at (17,13).\n#####################\n#...............#...#\n#.#############.#.#.#\n#.#...........#.#.#.#\n#.#.#########.#.#.#.#\n#.#.#.......#.#.#.#.#\n#.#.#.#####.#.#.#.###\n#.#.#.....#.#.#.#...#\n#.#.#######.#.#.#.#.#\n#.#.........#.#.#.#.#\n#.#########.#.#.#.#.#\n#...#.....#.#.#.#.#.#\n###.#.###.#.#.#.#.#.#\n#.#.#.#...#.#.#.#.#.#\n#.#.#.#.###.#.#.#.#.#\n#.#.#.#.#.#.#...#.#.#\n#.#A#.#.#.#.#####.#.#\n#.#.#.#...#.#G....#.#\n#.###.#####.#######.#\n#...................#\n#####################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 6, "answer": "south-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "ef51a678fda0d5c8bee06b33983f4490198a48750baa1c8f788094a8c91274c3"}435{"id": "distance_band-000434", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,7), goal at (3,7).\n########################\n#...#.....#.......#...##\n#.###.###.#.###.###.#.##\n#.....#G#.......#...#.##\n#.#.###.###.###.#.###.##\n#...#..A#.......#.#...##\n#.###.#.#.#.#####.#.####\n#...#.....#.#.....#...##\n###.#.#####.#.#######.##\n#...#...#.#.#.......#.##\n#.#####.#.#.#.#.#.#.#.##\n#.#.............#.#...##\n#.#######.#######.#.#.##\n#.......#.#...#...#...##\n###.#.#.###.#.#.#.###.##\n#.....#.....#.#.#.....##\n#.##.########.#.###.#.##\n#...........#.#.....#.##\n#.###.#.#.###.#.###.####\n#...#...#.#...#...#...##\n#.#.#.#.###.#########.##\n#.#...................##\n########################\n########################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "8b6bdca2b96b37b2143b9049ea678fc360f9a35261e3c46e09c9b0483e63ffd2"}436{"id": "distance_band-000435", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,5), goal at (0,0).\nG.##.#...#..#...\n..##...##...#.#.\n...#.#...#...#..\n.#.....#.....#..\n..#.#...#...#...\n....#A.......#..\n.#..........#...\n#...##....#..#.#\n.......#..##....\n#...#.###....#..\n##..#..#..###...\n##.#..#..#.##.##\n.#..###.........\n........#..#....\n..#...#.......#.\n......#......#..", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "22a1fdf1120bedee3c9e1a84568c0a900dee6dd5f844c39a9c8c95fb13af8ae4"}437{"id": "first_move-000436", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (10,25), goal at (23,1).\n############################\n#...#.......#.............##\n#.###.###.#.###.#.#####.#.##\n#.........#.#.....#...#...##\n#.#.###.#.#.#.###.#.#.###.##\n#.........#.#.#...#.#.....##\n###.#.##.##.#.#####.#####.##\n#...#.#.......#.....#.....##\n#####.###.#.###.###.#.###.##\n#.....#...#...#...#...#...##\n#.##.##.#.#.#.###.#.#####A##\n#.#.....#.#.#.....#.....#.##\n#.#.#.###.#.#########.#.####\n#...#...#.....#.#.........##\n#.#.###.#..##.#...#.#.###.##\n#...#...#...#.....#.#...#.##\n###.#.###.#.###...#.#####.##\n#.#...#...#.#...#.#.......##\n#.#.#.###.#.#####.#######.##\n#...#.......#.....#.....#.##\n#.###.###.#.#.#.###.#.#...##\n#.#...#.....#...........#.##\n#.###.#.###.#.#####.#...####\n#G#...#.#.....#...#...#...##\n#.#.#.#.#.#.###.#.#######.##\n#...#.....#.....#.........##\n############################\n############################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "9ee33b7e2aa82d0916996fee0c238b7d3e2d709b087dd52e2e8b43d2e0604e8b"}438{"id": "move_legal-000437", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[2,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"0,8\":\"wall\",\"0,9\":\"outside\",\"1,5\":\"open\",\"1,6\":\"wall\",\"1,7\":\"open\",\"1,8\":\"wall\",\"1,9\":\"outside\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"agent\",\"2,8\":\"wall\",\"2,9\":\"outside\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"3,8\":\"wall\",\"3,9\":\"outside\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"open\",\"4,8\":\"wall\",\"4,9\":\"outside\"}}", "question": "Is the cell one step west of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "74ecad1d2b20e7ef436a9d7e94538b8bd79a0249fde890427a738a860f6fd40a"}439{"id": "onward-000438", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (8,4); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n###..\n##.#.\n##A..\nXXXXX\nXXXXX", "question": "Suppose the agent moves one cell east. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "7f2a9891b30a653e157fe2a9d3688a36e9c8236fd2f6f642ffbec4f9de2de2f1"}440{"id": "goal_bearing-000439", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (12,0), goal at (1,1).\n....##...#######\n.G#....#########\n..#.....########\n.#...#.#########\n......##########\n.....###########\n#....###########\n..##.###########\n.....###########\n..#...##########\n#.##....########\n.#####.#########\nA.##....########\n....#..#########\n#.#...##########\n....#...########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "42029a4b290668363538b992059de7adbc87b37e32e404cd3113c43450526988"}441{"id": "distance_band-000440", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,7), goal at (9,11).\n################\n#.#.....#.....##\n#.#.#.#.###.#.##\n#...#.#.....#.##\n#.###.#######.##\n#...#.......#.##\n#.#########.#.##\n#.#...#.....#.##\n#.#.###.#####.##\n#.#...#...#G#.##\n#.###.###.#.#.##\n#...#...#.#.#.##\n###.###.#.#.#.##\n#.....#A..#...##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 4, "answer": "40 moves or more", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "fae7840e19c094d5d1a77ffd27fd104559f6c9870cd0fc59ad601454c8448d0a"}442{"id": "goal_bearing-000441", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,3), goal at (3,9).\n################\n#.............##\n#####.#####.####\n#..A#....G#...##\n#.###########.##\n#.#...........##\n#.#.#.#######.##\n#.#.#...#.....##\n#.#####.#####.##\n#.#...#.....#.##\n#.#.#.#####.#.##\n#...#.#.....#.##\n#####.#.#####.##\n#.......#.....##\n################\n################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a9554ec451a2e27a6acab9c78984088530c07a83ec42e979ccee9721d875c3ec"}443{"id": "plan_survives-000442", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 1. Rows increase to the south and columns to the east.\nRow 3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 4: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 5: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 6: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall\nRow 7: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is wall, column 3 is open, column 4 is open, column 5 is open\nRow 8: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall\nRow 9: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is wall, column 5 is open\nRow 10: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 11: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open", "question": "The agent will attempt this plan in order: south, east, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "98c34be7ad4e98d5d4d38ff28bfe0e922d144ab805fcd7c0824b3304dfca5e06"}444{"id": "plan_survives-000443", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (13,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#.#.##XX\n.#.#.##XX\n...#.##XX\n##.#.##XX\n...#A##XX\n#######XX\n#######XX\nXXXXXXXXX\nXXXXXXXXX", "question": "The agent will attempt this plan in order: south, west, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "76ac7d300ac91828e4e1f9ad2ba70b070a9b2228bd24787dcb11489251762a82"}445{"id": "cell_kind-000444", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (9,8); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#######\n.....##\n####.##\n.#.A.##\n#######\n#######\nXXXXXXX", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "ba33595b78350ba838feeb3b1f90b5657fc237288368f17e4c0ceaa4ea3dd3e4"}446{"id": "reachable_within-000445", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (21,3), goal at (1,21).\n########################\n#.............#......G##\n#.#.###.###.###.###.#.##\n#.#...#.#...#.....#.#.##\n#.###...##.##.##.##.#.##\n#...#...#.....#.....#.##\n#.#.#.#.#.#####.#.###.##\n#.#...#.#...#.#.......##\n###.#.#.###.#.###.#.#.##\n#.........#...#.#.#...##\n#.#####.#.###.#.#.###.##\n#.....#...#.....#...#.##\n#.###.##.##.#.#####.#.##\n#...#...............#.##\n###.#.####..#.#####.#.##\n#...#.......#.........##\n#.###.##.##.#.##.##.####\n#.....#...#...........##\n###.###.#.#.#.#.#.###.##\n#.......#...#.....#...##\n#.###.#######.#####.#.##\n#..A#...........#...#.##\n########################\n########################", "question": "Can the agent reach the goal in at most 45 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "4adc60c7bb1d954c4e0ef37ca5b3b831acba98385e734ac66a89ae23c9b98637"}447{"id": "snake_food-000446", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,7), travelling south, length 3.\n........\n.....*..\n........\n........\n........\n.......o\n.......o\n.......H", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3e8bdf16da86762ba0172bb07f33056581d8c9f0034e29d89b503c0d547dca8a"}448{"id": "cell_kind-000447", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,16],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,14\":\"outside\",\"-1,15\":\"outside\",\"-1,16\":\"outside\",\"-1,17\":\"outside\",\"-1,18\":\"outside\",\"0,14\":\"wall\",\"0,15\":\"wall\",\"0,16\":\"wall\",\"0,17\":\"wall\",\"0,18\":\"wall\",\"1,14\":\"wall\",\"1,15\":\"open\",\"1,16\":\"agent\",\"1,17\":\"open\",\"1,18\":\"open\",\"2,14\":\"wall\",\"2,15\":\"wall\",\"2,16\":\"wall\",\"2,17\":\"wall\",\"2,18\":\"wall\",\"3,14\":\"open\",\"3,15\":\"open\",\"3,16\":\"wall\",\"3,17\":\"open\",\"3,18\":\"open\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "060bef7e6733bd197e1d8af73a159885196d854fc077fba39dc6f076e30eab2f"}449{"id": "snake_food-000448", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (1,5), travelling north, length 3.\n.....*....\n.....H....\n.....o....\n.....o....\n..........\n..........\n..........\n..........\n..........\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "0e5396c1bf3c83e971a309fd2dd678314aff0bfe34ba599a2ebc0c93431e442b"}450{"id": "move_two_step-000449", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 27, column 26. Rows increase to the south and columns to the east.\nRow 25: column 24 is open, column 25 is open, column 26 is wall, column 27 is open, column 28 is outside the maze\nRow 26: column 24 is wall, column 25 is wall, column 26 is open, column 27 is open, column 28 is outside the maze\nRow 27: column 24 is open, column 25 is open, column 26 is the agent, column 27 is open, column 28 is outside the maze\nRow 28: column 24 is outside the maze, column 25 is outside the maze, column 26 is outside the maze, column 27 is outside the maze, column 28 is outside the maze\nRow 29: column 24 is outside the maze, column 25 is outside the maze, column 26 is outside the maze, column 27 is outside the maze, column 28 is outside the maze", "question": "The agent plans to move west, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "f4815c6d807adca53ccc32747c2e6a1dc7979c7691715e7c77f347dff0a2d3c0"}451{"id": "move_legal-000450", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[4,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,4\":\"wall\",\"1,5\":\"open\",\"1,6\":\"open\",\"1,7\":\"wall\",\"1,8\":\"open\",\"1,9\":\"outside\",\"1,10\":\"outside\",\"2,4\":\"open\",\"2,5\":\"wall\",\"2,6\":\"wall\",\"2,7\":\"wall\",\"2,8\":\"open\",\"2,9\":\"outside\",\"2,10\":\"outside\",\"3,4\":\"open\",\"3,5\":\"wall\",\"3,6\":\"open\",\"3,7\":\"open\",\"3,8\":\"open\",\"3,9\":\"outside\",\"3,10\":\"outside\",\"4,4\":\"open\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"agent\",\"4,8\":\"open\",\"4,9\":\"outside\",\"4,10\":\"outside\",\"5,4\":\"open\",\"5,5\":\"wall\",\"5,6\":\"open\",\"5,7\":\"open\",\"5,8\":\"wall\",\"5,9\":\"outside\",\"5,10\":\"outside\",\"6,4\":\"wall\",\"6,5\":\"open\",\"6,6\":\"open\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"outside\",\"6,10\":\"outside\",\"7,4\":\"open\",\"7,5\":\"open\",\"7,6\":\"open\",\"7,7\":\"open\",\"7,8\":\"open\",\"7,9\":\"outside\",\"7,10\":\"outside\"}}", "question": "If the agent attempts one cell east (col+1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "239e2391633b0959e6a483a20ca7a35ebadca4385e7be555da7041da1a9b7584"}452{"id": "plan_survives-000451", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,19); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX\n#########\n....A....\n##.###.#.\n...#.....\n########.\n.......#.", "question": "The agent will attempt this plan in order: north, north, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "1549b9826c598e466f2216a7907631b582f0d1743d2b958967e08c94a4e517cd"}453{"id": "only_way-000452", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[5,21],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"2,18\":\"wall\",\"2,19\":\"wall\",\"2,20\":\"wall\",\"2,21\":\"open\",\"2,22\":\"wall\",\"2,23\":\"open\",\"2,24\":\"wall\",\"3,18\":\"wall\",\"3,19\":\"open\",\"3,20\":\"wall\",\"3,21\":\"open\",\"3,22\":\"open\",\"3,23\":\"open\",\"3,24\":\"wall\",\"4,18\":\"wall\",\"4,19\":\"open\",\"4,20\":\"wall\",\"4,21\":\"wall\",\"4,22\":\"wall\",\"4,23\":\"wall\",\"4,24\":\"wall\",\"5,18\":\"wall\",\"5,19\":\"open\",\"5,20\":\"open\",\"5,21\":\"agent\",\"5,22\":\"wall\",\"5,23\":\"open\",\"5,24\":\"open\",\"6,18\":\"wall\",\"6,19\":\"open\",\"6,20\":\"wall\",\"6,21\":\"wall\",\"6,22\":\"wall\",\"6,23\":\"open\",\"6,24\":\"wall\",\"7,18\":\"wall\",\"7,19\":\"open\",\"7,20\":\"open\",\"7,21\":\"open\",\"7,22\":\"wall\",\"7,23\":\"open\",\"7,24\":\"open\",\"8,18\":\"wall\",\"8,19\":\"open\",\"8,20\":\"wall\",\"8,21\":\"open\",\"8,22\":\"wall\",\"8,23\":\"wall\",\"8,24\":\"wall\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a18e1d639dda1d1e2a5340a98d6a3bef81d05d639840423ec15231f5936bc259"}454{"id": "open_count-000453", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (25,25); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.#.#.#\n#.#...#\n#.###.#\n#.#A#.#\n#.#.#.#\n#.#.#.#\n#.#.###", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 1, "answer": "1", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a4b03c6e111cce88ec34e6839c998d283a878c414bca13fb2744fe14982add2c"}455{"id": "blocked_pair-000454", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[8,8],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"6,6\":\"wall\",\"6,7\":\"wall\",\"6,8\":\"wall\",\"6,9\":\"outside\",\"6,10\":\"outside\",\"7,6\":\"open\",\"7,7\":\"open\",\"7,8\":\"wall\",\"7,9\":\"outside\",\"7,10\":\"outside\",\"8,6\":\"wall\",\"8,7\":\"open\",\"8,8\":\"agent\",\"8,9\":\"outside\",\"8,10\":\"outside\",\"9,6\":\"outside\",\"9,7\":\"outside\",\"9,8\":\"outside\",\"9,9\":\"outside\",\"9,10\":\"outside\",\"10,6\":\"outside\",\"10,7\":\"outside\",\"10,8\":\"outside\",\"10,9\":\"outside\",\"10,10\":\"outside\"}}", "question": "Are BOTH the cell one step north and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "50a6648f50e183f110a909e9d2dc6c04e2dae3ffa0a04e8ca4f11d82980d7fc8"}456{"id": "stochastic-000455", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 2, column 3. Rows increase to the south and columns to the east.\nRow -1: column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze\nRow 0: column 0 is wall, column 1 is wall, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall\nRow 1: column 0 is wall, column 1 is wall, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall\nRow 2: column 0 is wall, column 1 is open, column 2 is open, column 3 is the agent, column 4 is wall, column 5 is wall, column 6 is wall\nRow 3: column 0 is wall, column 1 is wall, column 2 is open, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall\nRow 4: column 0 is wall, column 1 is wall, column 2 is open, column 3 is open, column 4 is open, column 5 is wall, column 6 is open\nRow 5: column 0 is open, column 1 is wall, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.2666666666666667, 0.7333333333333333], "deterministic": false, "fingerprint": "c74f66297c3a00ab1ea341b225fc953ef06c6c3e244a38755eba1ad54ce2fc4f"}457{"id": "goal_bearing-000456", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,12), goal at (13,13).\n################\n#...........A.##\n#######.########\n#.......#.....##\n#.#######.######\n#.#.......#...##\n#.#.#######.#.##\n#.#.#.......#.##\n#.#.#.#.#####.##\n#.#.#.#.#.....##\n#.#.###.#.######\n#.#.#...#.#...##\n#.#.#.###.#.#.##\n#.....#.....#G##\n################\n################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 6, "answer": "south-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "6efc78a28a5e6a30813eb4ca7b45a8018ca990134439e8606040bc6d0e608b3b"}458{"id": "move_legal-000457", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[2,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-2,5\":\"outside\",\"-2,6\":\"outside\",\"-2,7\":\"outside\",\"-2,8\":\"outside\",\"-2,9\":\"outside\",\"-2,10\":\"outside\",\"-2,11\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"-1,6\":\"outside\",\"-1,7\":\"outside\",\"-1,8\":\"outside\",\"-1,9\":\"outside\",\"-1,10\":\"outside\",\"-1,11\":\"outside\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"0,8\":\"wall\",\"0,9\":\"wall\",\"0,10\":\"wall\",\"0,11\":\"wall\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"1,6\":\"wall\",\"1,7\":\"open\",\"1,8\":\"wall\",\"1,9\":\"open\",\"1,10\":\"wall\",\"1,11\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"agent\",\"2,8\":\"wall\",\"2,9\":\"open\",\"2,10\":\"wall\",\"2,11\":\"wall\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"3,8\":\"open\",\"3,9\":\"open\",\"3,10\":\"wall\",\"3,11\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"wall\",\"4,8\":\"wall\",\"4,9\":\"open\",\"4,10\":\"wall\",\"4,11\":\"wall\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"open\",\"5,9\":\"open\",\"5,10\":\"wall\",\"5,11\":\"wall\",\"6,3\":\"wall\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"wall\",\"6,10\":\"wall\",\"6,11\":\"wall\"}}", "question": "Is the cell one step south of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "d65bd7c5b7b18f0bd1aa11ae0dca0decda2dba79858aeea0979860c5146be487"}459{"id": "plan_survives-000458", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,32); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXX\n#####\n..A.#\n#.###\n#...#", "question": "The agent will attempt this plan in order: west, east, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "44afbb692fc56a6bf0d3cc44be8ed301e80c12acdc23e7118fa4094a95b983f1"}460{"id": "distance_band-000459", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,7), goal at (1,13).\n################\n#............G##\n#.#####.########\n#......A......##\n###.#########.##\n#.#.......#...##\n#.#######.#.#.##\n#...........#.##\n#.###.###.###.##\n#...#.......#.##\n#.#.###.###.#.##\n#.........#.#.##\n#.#.#.###.###.##\n#...#...#.....##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 1, "answer": "5 to 9 moves", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "81486996b519414917d39baf1a9f6b27e3d5301cd83bc5bdf5072ceb81b2258f"}461{"id": "snake_safe-000460", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (4,3), travelling north, length 3.\n........\n........\n........\n........\n...H....\n...oo.*.\n........\n........", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "9bd4fff056874703e2dde58b3bc95eb52a7879d1469570c09df50cfa362f7227"}462{"id": "stochastic-000461", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 8, column 0. Rows increase to the south and columns to the east.\nRow 6: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is open\nRow 7: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open\nRow 8: column -2 is outside the maze, column -1 is outside the maze, column 0 is the agent, column 1 is open, column 2 is wall\nRow 9: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is open\nRow 10: column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open", "question": "The actuator executes the intended move with probability 0.70; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9, 0.10000000000000002], "deterministic": false, "fingerprint": "1759a3a90f05a953cd8aab19e830fdb5d66b0663ce1d63318cd53d55f83efe8c"}463{"id": "snake_safe-000462", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (1,4), travelling east, length 3.\n........\n...oH...\n...o....\n........\n........\n...*....\n........\n........", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "986f11f2f086d4c57cf167bbf519add670e8089e67f673e2805c695bf9b914f5"}464{"id": "move_two_step-000463", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,13],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,9\":\"outside\",\"-1,10\":\"outside\",\"-1,11\":\"outside\",\"-1,12\":\"outside\",\"-1,13\":\"outside\",\"-1,14\":\"outside\",\"-1,15\":\"outside\",\"-1,16\":\"outside\",\"-1,17\":\"outside\",\"0,9\":\"wall\",\"0,10\":\"wall\",\"0,11\":\"wall\",\"0,12\":\"wall\",\"0,13\":\"wall\",\"0,14\":\"wall\",\"0,15\":\"wall\",\"0,16\":\"outside\",\"0,17\":\"outside\",\"1,9\":\"open\",\"1,10\":\"open\",\"1,11\":\"open\",\"1,12\":\"open\",\"1,13\":\"open\",\"1,14\":\"wall\",\"1,15\":\"wall\",\"1,16\":\"outside\",\"1,17\":\"outside\",\"2,9\":\"wall\",\"2,10\":\"wall\",\"2,11\":\"open\",\"2,12\":\"wall\",\"2,13\":\"open\",\"2,14\":\"wall\",\"2,15\":\"wall\",\"2,16\":\"outside\",\"2,17\":\"outside\",\"3,9\":\"open\",\"3,10\":\"open\",\"3,11\":\"open\",\"3,12\":\"wall\",\"3,13\":\"agent\",\"3,14\":\"wall\",\"3,15\":\"wall\",\"3,16\":\"outside\",\"3,17\":\"outside\",\"4,9\":\"open\",\"4,10\":\"wall\",\"4,11\":\"wall\",\"4,12\":\"wall\",\"4,13\":\"open\",\"4,14\":\"wall\",\"4,15\":\"wall\",\"4,16\":\"outside\",\"4,17\":\"outside\",\"5,9\":\"open\",\"5,10\":\"open\",\"5,11\":\"open\",\"5,12\":\"wall\",\"5,13\":\"open\",\"5,14\":\"wall\",\"5,15\":\"wall\",\"5,16\":\"outside\",\"5,17\":\"outside\",\"6,9\":\"wall\",\"6,10\":\"wall\",\"6,11\":\"open\",\"6,12\":\"wall\",\"6,13\":\"open\",\"6,14\":\"wall\",\"6,15\":\"wall\",\"6,16\":\"outside\",\"6,17\":\"outside\",\"7,9\":\"open\",\"7,10\":\"wall\",\"7,11\":\"open\",\"7,12\":\"open\",\"7,13\":\"open\",\"7,14\":\"wall\",\"7,15\":\"wall\",\"7,16\":\"outside\",\"7,17\":\"outside\"}}", "question": "The agent plans to move east, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "e83f14b03b6d1800b35d80e3136d9216c755199e85c69c80745661f8d175b2ca"}465{"id": "cell_kind-000464", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 3, column 3. Rows increase to the south and columns to the east.\nRow 0: column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall\nRow 1: column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open\nRow 2: column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall\nRow 3: column 0 is wall, column 1 is open, column 2 is wall, column 3 is the agent, column 4 is open, column 5 is open, column 6 is open\nRow 4: column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall\nRow 5: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open, column 6 is open\nRow 6: column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "6bb5282ffb30f2066a8538a76114ed4efa705c4b6c478298320cfc4f278b876c"}466{"id": "reachable_within-000465", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (4,13), goal at (7,7).\n################\n#.............##\n#.###########.##\n#.#.........#.##\n#.#.#######.#A##\n#.#.#.....#.#.##\n#.#.#.###.#.#.##\n#.#.#.#G#.#.#.##\n#.#.#.#.#.#.#.##\n#.#.#...#.#.#.##\n#.#.#####.#.#.##\n#.#.......#.#.##\n#.#########.####\n#.............##\n################\n################", "question": "Can the agent reach the goal in at most 87 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "f20c97beb66773a4c1bad0e986d5eb3cbfb9be1330daf91fd7c28ff2d958cef1"}467{"id": "cell_kind-000466", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[5,5],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"1,6\":\"open\",\"1,7\":\"open\",\"1,8\":\"open\",\"1,9\":\"open\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"2,6\":\"wall\",\"2,7\":\"wall\",\"2,8\":\"wall\",\"2,9\":\"open\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"3,6\":\"open\",\"3,7\":\"open\",\"3,8\":\"wall\",\"3,9\":\"open\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"open\",\"4,8\":\"wall\",\"4,9\":\"open\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"agent\",\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"wall\",\"5,9\":\"open\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"open\",\"6,6\":\"wall\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"open\",\"7,1\":\"open\",\"7,2\":\"wall\",\"7,3\":\"open\",\"7,4\":\"wall\",\"7,5\":\"open\",\"7,6\":\"wall\",\"7,7\":\"open\",\"7,8\":\"wall\",\"7,9\":\"open\",\"8,1\":\"open\",\"8,2\":\"wall\",\"8,3\":\"open\",\"8,4\":\"wall\",\"8,5\":\"open\",\"8,6\":\"wall\",\"8,7\":\"open\",\"8,8\":\"wall\",\"8,9\":\"open\",\"9,1\":\"open\",\"9,2\":\"wall\",\"9,3\":\"open\",\"9,4\":\"wall\",\"9,5\":\"open\",\"9,6\":\"open\",\"9,7\":\"open\",\"9,8\":\"wall\",\"9,9\":\"open\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "24b1f9cee914e6302689573763cbfceaca669116a881bba9b3eced6c8aa8f665"}468{"id": "plan_survives-000467", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (34,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX....###.\nX.#..#..#\nX....#.#.\nX........\nX#.#A....\nX.....#.#\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX", "question": "The agent will attempt this plan in order: east, north, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "4064de7141df9804a63de1a85f3dff518c5c4dcc050f2161e0a198c50c4d5fda"}469{"id": "goal_bearing-000468", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,1), goal at (1,3).\n#########\n#A#G..#.#\n#.###.#.#\n#.#...#.#\n#.#.###.#\n#.#.....#\n#.#####.#\n#.......#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "4161f5530b7368ccf615b94d1937520dc7efbd30451e2f161ad8cd4ec13dc30a"}470{"id": "blocked_pair-000469", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 8, column 0. Rows increase to the south and columns to the east.\nRow 5: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is wall\nRow 6: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall\nRow 7: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is wall\nRow 8: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is the agent, column 1 is open, column 2 is open, column 3 is wall\nRow 9: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is open, column 3 is open\nRow 10: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is open, column 3 is wall\nRow 11: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall", "question": "Are BOTH the cell one step west and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "38190d56ce8b97f77362cf7e0c3eff134a559724a5c954aa688193638de9df6f"}471{"id": "move_legal-000470", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[13,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"9,-1\":\"outside\",\"9,0\":\"wall\",\"9,1\":\"open\",\"9,2\":\"wall\",\"9,3\":\"open\",\"9,4\":\"open\",\"9,5\":\"open\",\"9,6\":\"open\",\"9,7\":\"open\",\"10,-1\":\"outside\",\"10,0\":\"wall\",\"10,1\":\"open\",\"10,2\":\"wall\",\"10,3\":\"wall\",\"10,4\":\"wall\",\"10,5\":\"wall\",\"10,6\":\"wall\",\"10,7\":\"wall\",\"11,-1\":\"outside\",\"11,0\":\"wall\",\"11,1\":\"open\",\"11,2\":\"open\",\"11,3\":\"open\",\"11,4\":\"open\",\"11,5\":\"open\",\"11,6\":\"open\",\"11,7\":\"open\",\"12,-1\":\"outside\",\"12,0\":\"wall\",\"12,1\":\"wall\",\"12,2\":\"wall\",\"12,3\":\"wall\",\"12,4\":\"wall\",\"12,5\":\"wall\",\"12,6\":\"wall\",\"12,7\":\"wall\",\"13,-1\":\"outside\",\"13,0\":\"wall\",\"13,1\":\"open\",\"13,2\":\"open\",\"13,3\":\"agent\",\"13,4\":\"wall\",\"13,5\":\"open\",\"13,6\":\"open\",\"13,7\":\"open\",\"14,-1\":\"outside\",\"14,0\":\"wall\",\"14,1\":\"open\",\"14,2\":\"wall\",\"14,3\":\"wall\",\"14,4\":\"wall\",\"14,5\":\"wall\",\"14,6\":\"wall\",\"14,7\":\"wall\",\"15,-1\":\"outside\",\"15,0\":\"wall\",\"15,1\":\"open\",\"15,2\":\"wall\",\"15,3\":\"open\",\"15,4\":\"open\",\"15,5\":\"open\",\"15,6\":\"open\",\"15,7\":\"open\",\"16,-1\":\"outside\",\"16,0\":\"wall\",\"16,1\":\"open\",\"16,2\":\"wall\",\"16,3\":\"open\",\"16,4\":\"wall\",\"16,5\":\"wall\",\"16,6\":\"wall\",\"16,7\":\"wall\",\"17,-1\":\"outside\",\"17,0\":\"wall\",\"17,1\":\"open\",\"17,2\":\"wall\",\"17,3\":\"open\",\"17,4\":\"open\",\"17,5\":\"open\",\"17,6\":\"open\",\"17,7\":\"open\"}}", "question": "The agent wants to move south. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "7a72978aab5130d1c648f7743b2774f0fef6e1fcd85d29127cd0d0a122779d02"}472{"id": "plan_survives-000471", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[4,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"1,6\":\"wall\",\"1,7\":\"open\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"open\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"agent\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"open\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\",\"5,4\":\"open\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"wall\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"6,7\":\"open\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"open\",\"7,3\":\"open\",\"7,4\":\"open\",\"7,5\":\"open\",\"7,6\":\"open\",\"7,7\":\"open\",\"8,-1\":\"outside\",\"8,0\":\"wall\",\"8,1\":\"wall\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"wall\",\"8,6\":\"wall\",\"8,7\":\"wall\"}}", "question": "The agent will attempt this plan in order: south, north, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "18e0867ed0dddc542373a82df2f6cbb857659b17617065c6151284bef4783fd7"}473{"id": "first_move-000472", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,2), goal at (13,12).\n################\n#.A.....#.....##\n#.#.###.#.#.#.##\n#.#.......#...##\n#.###########.##\n#.#...........##\n#...###...###.##\n#.#...#.#...#.##\n#.###...#.#...##\n#.........#.#.##\n#.#######.#.#.##\n#.........#...##\n#.#.##.##.###.##\n#...........G.##\n################\n################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "8ab54eb0dba775ca6ff49ae22dac9243498bec89ee995395e4913fd46a6fc666"}474{"id": "snake_food-000473", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,6), travelling west, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n......Ho......\n.......o......\n..............\n..............\n........*.....\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 6, "answer": "south-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "9df0bcf0cd7545a86d5d555496bd2ca1f13e9ce7815849b741f3b748eec6ca4f"}475{"id": "plan_survives-000474", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 10, column 9. Rows increase to the south and columns to the east.\nRow 7: column 6 is wall, column 7 is open, column 8 is open, column 9 is open, column 10 is open, column 11 is open, column 12 is wall\nRow 8: column 6 is wall, column 7 is open, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is open, column 12 is wall\nRow 9: column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open, column 12 is wall\nRow 10: column 6 is wall, column 7 is open, column 8 is wall, column 9 is the agent, column 10 is wall, column 11 is open, column 12 is wall\nRow 11: column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is open, column 11 is open, column 12 is wall\nRow 12: column 6 is wall, column 7 is open, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall\nRow 13: column 6 is wall, column 7 is open, column 8 is open, column 9 is open, column 10 is open, column 11 is open, column 12 is open", "question": "The agent will attempt this plan in order: east, east, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b3cb238b9dbcf127bb0e28046bff6900d0b9cdbf9a852f7320861e725af25d54"}476{"id": "distance_band-000475", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,9), goal at (21,5).\n########################\n#.......#.............##\n#.#.###.#.#.#.###.###.##\n#.#.#...#...#...#.....##\n#.#.#####.###.#.#####.##\n#.#......A#...........##\n###.####.##.#.###.###.##\n#.....#.........#.....##\n#.###.#.#...#.####.##.##\n#...#...#...#.........##\n#.#.###.###.#...#.#.####\n#.#...#...#.#...#.#...##\n#.###.#####.#.#.###.#.##\n#.#.#.....#...#.....#.##\n#.#.#.###.###########.##\n#.....................##\n#.#.#.#.###.#####.#..###\n#...#.#...#.#.....#...##\n#.###.###.###.#.#.#.#.##\n#...#...#...#.#.....#.##\n#.#.###.#.#.#.#####.#.##\n#.#..G....#...........##\n########################\n########################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 3, "answer": "20 to 39 moves", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "2a4ab99e3c40b0a33306630443700a48caed8b14ded26217aa77c5c4337245eb"}477{"id": "goal_bearing-000476", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,5), goal at (3,5).\n################\n#.....#.......##\n#.###.#.###.#.##\n#.#.#G#.#.#.#.##\n#.#.###.#.#.#.##\n#...#A#...#.#.##\n###.#.###.#.####\n#.#.#.....#...##\n#.#.#########.##\n#.#...#...#...##\n#.###.#.#.#.#.##\n#...#.#.#.#.#.##\n#.#.#.#.#.#.#.##\n#.#.....#...#.##\n################\n################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "20d86180b5b9bd2f8e8b75dd7ab75b97ba83887f8bcd6a854089cec02a7ca339"}478{"id": "stochastic-000477", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[6,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"6,1\":\"wall\",\"6,2\":\"wall\",\"6,3\":\"agent\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"7,1\":\"open\",\"7,2\":\"wall\",\"7,3\":\"open\",\"7,4\":\"open\",\"7,5\":\"open\",\"8,1\":\"open\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"wall\"}}", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.7333333333333334, 0.26666666666666666], "deterministic": false, "fingerprint": "11b7971c72bfb24810327dc525af58d6922c80933741a855f27df91e89a727c4"}479{"id": "only_way-000478", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 9, column 11. Rows increase to the south and columns to the east.\nRow 7: column 9 is open, column 10 is open, column 11 is open, column 12 is wall, column 13 is open\nRow 8: column 9 is open, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall\nRow 9: column 9 is open, column 10 is wall, column 11 is the agent, column 12 is open, column 13 is open\nRow 10: column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open\nRow 11: column 9 is open, column 10 is open, column 11 is open, column 12 is open, column 13 is open", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "fc63e8899a915e399aa40a9ab06fdb7911a29984e97996e604b77f86a717a5c3"}480{"id": "distance_band-000479", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (23,17), goal at (23,18).\n########################\n########################\n.#######################\n..######################\n...#####################\n##...###################\n#....###################\n#.....##################\n#.....##################\n##.##..#..##############\n...##.#..#.#############\n.#..#.##.#.#############\n#.#....#....############\n.......#.....###########\n...###....#...##########\n...##....#.......#######\n..#...###...#.....######\n...#..#...........######\n.#.#..#....#...#..######\n####..#...#......#######\n###...#........#########\n...........##..#.#######\n.#....#...####....######\n........#####..#.AG#####", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "92acb9037983f2db6e7613f0e659ec42d7c27242829896cdf48d1ed629362c7d"}481{"id": "snake_safe-000480", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (4,4), travelling east, length 3.\n........\n........\n........\n........\n..ooH...\n........\n..*.....\n........", "question": "If the snake moves west on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "bdc5e8c9527f3f81a8ccf2f418dcfd6a8a5c33e42cb3180980e3a784b9b35fce"}482{"id": "reachable_within-000481", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (12,9), goal at (21,1).\n########################\n#.....................##\n#.#######.###########.##\n#.#.....#.#.........#.##\n#.#####.#.#.#######.#.##\n#.......#.#.#.....#.#.##\n#########.#.#.###.#.#.##\n#.........#.#.#.#.#.#.##\n#.#########.#.#.#.#.#.##\n#.#.......#.#.#...#.#.##\n#.#.#####.#.#.#####.#.##\n#.#.#...#.#.#.....#.#.##\n#.#.#.###A#.#.###.#.#.##\n#.#.#.....#.#...#.#.#.##\n#.#.#######.#####.#.#.##\n#.#...............#.#.##\n#.#################.#.##\n#.#.................#.##\n#.#.###############.#.##\n#.#.#...............#.##\n#.#.#################.##\n#G#...................##\n########################\n########################", "question": "Can the agent reach the goal in at most 154 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "5f0e0728169e7054ff99a25cba0d6c064a7351128aa1c22fad611726ba151cb0"}483{"id": "snake_safe-000482", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,5), travelling south, length 3.\n..............\n..............\n.*............\n..............\n..............\n..............\n..............\n.....oo.......\n.....H........\n..............\n..............\n..............\n..............\n..............", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "0df3a9459af4ca1d5c2b8b37bb68d17fd9bf61c14d492afea24c5577fdf70a68"}484{"id": "cell_kind-000483", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (23,5); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#####.#\n....#..\n#.#.###\n..#A#..\n#####.#\n....#..\n#.###.#", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "c21a307ea7a85d573aa48f7321754cc7ef64851562a4b1d9c8cf750bffe0ed73"}485{"id": "blocked_pair-000484", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[27,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"25,-1\":\"outside\",\"25,0\":\"wall\",\"25,1\":\"wall\",\"25,2\":\"open\",\"25,3\":\"wall\",\"26,-1\":\"outside\",\"26,0\":\"wall\",\"26,1\":\"open\",\"26,2\":\"open\",\"26,3\":\"wall\",\"27,-1\":\"outside\",\"27,0\":\"open\",\"27,1\":\"agent\",\"27,2\":\"open\",\"27,3\":\"wall\",\"28,-1\":\"outside\",\"28,0\":\"outside\",\"28,1\":\"outside\",\"28,2\":\"outside\",\"28,3\":\"outside\",\"29,-1\":\"outside\",\"29,0\":\"outside\",\"29,1\":\"outside\",\"29,2\":\"outside\",\"29,3\":\"outside\"}}", "question": "Are BOTH the cell one step south and the cell one step west blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "f843cb5ee16accd9cf9cc16f2f3fb35bf180f04bae5583a187d0ef752a3dc241"}486{"id": "distance_band-000485", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 40x40. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,31), goal at (1,35).\n########################################\n#...#.................#.#...#.....#G..##\n#.###.#######.#.#.###...#.#.#.#A#.###.##\n#.#...#...#.#.....#.#.........#.#...#.##\n#...###.#.#.#####...#.#.###.###.###.#.##\n#.#...#.#.......#.#.....#.......#.....##\n#.#.#.#.#####.###.#.###.###.#.#.########\n#.........#.......#...#.....#.#...#...##\n#.#####.#.#.###.#####.###.###.###.#.#.##\n#.#...#.#.....#...#.#...#...#.#.#...#.##\n#.#.#.###.###.###.#.#.#.#.###.#.###.#.##\n#.......#.#...#...#.#...#.........#...##\n###.###.#.###.#.###.#.####.######.#.#.##\n#.....#.#.....#.#.....#.....#.......#.##\n#.#####.#.###...#####.#.#.#.#####.#.#.##\n#.#...........#.#.........#...........##\n#.#.###.#.###.#.#.#.#####.##...######.##\n#...#.....#...#...#.................#.##\n#.###.#.###.#.#######.#.#.#####.#.#.#.##\n#.#...#.#.#.........#...#.....#...#.#.##\n#.###.#.#.#..##.#.###.#.#.###.#####.#.##\n#.........#...........#.#.#...#.....#.##\n#.#.#.###.#.##.####.#.#.#.#.#.###.###.##\n#.#.....#...#.....#.#...#.#...........##\n#.###.#.#####.#.#.###...###.############\n#.............#.......#...#...#.......##\n#########.#######.#####.#.#####.#####.##\n#.....#...#.....#.#.....#.....#.#.....##\n#.###...###.###.###.###.#####.#.#.#.####\n#...#.#.#...#.#.....#...#...#...#.....##\n#.#.###.#.###.#.#.#...#...#.###.###.#.##\n#...#.....#...#...#.#...#.#...#.......##\n#.#.#.#.###.#.#.###.#.#.#.#.#.#####.#.##\n#...#.#.#...#.#.#...#.#...#.....#...#.##\n###...#.#.#.###.#.###.#####.###...###.##\n#...#...#.......#...#...#...#...#.#...##\n#.#.#.#...#.#.#.###...###.#.#.###.#.#.##\n#.......#...#.......#...............#.##\n########################################\n########################################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "4ea42eee9e70d9f95dd38e77e1ccc011d511e46dc700311aa683ad4145885e90"}487{"id": "move_legal-000486", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (7,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.#####\n#......\n#######\n#..A...\n#######\nXXXXXXX\nXXXXXXX", "question": "If the agent attempts one cell north (row-1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "99048f108b65183ea56cedc9c630ab7d224e78bf9d18531cfd0f78c0226f4a40"}488{"id": "first_move-000487", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,29), goal at (23,15).\n####################################\n#...............#.................##\n#.#############.#.###############.##\n#.#.....#.....#.#.#.............#.##\n#.#.###.#.###.#.#.#.###########.#.##\n#.#.#.#.#.#.#.#.#.#.#.....#..A#.#.##\n#.#.#.#.#.#.#.#.#.#.#.###.###.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.#.......#.#.##\n#.#.#.#.#.#.#.#.#.#.#.#########.#.##\n#.#.#.#.#.#...#.#.#.#...........#.##\n#.#.#.#.#.#####.#.#.#############.##\n#.#.#.#.#.....#.#.#...............##\n#.#.#.#.#####.#.#.##################\n#.#.#.........#.#.................##\n#.#.#########.#.#.###############.##\n#.#.....#...#.#.#.#...#.........#.##\n#.#.###.#.#.#.#.#.#.###.#######.#.##\n#.#.#.#.#.#.#.#.#.#...........#.#.##\n#.#.#.#.#.###.#.#.#############.#.##\n#.#.#.#.#.....#.#...............#.##\n#.#.#.#.#.#####.#################.##\n#.#.#.#.#.#...#.#.................##\n#.#.#.#.#.###.#.#.###############.##\n#.#.#.#.#.....#G#.#...#.........#.##\n#.#.#.#.#########.#.#.#.#####.#.#.##\n#.#.#.#...........#.#.#.#...#.#.#.##\n#.#.#.###########.#.#.#.#.#.#.#.#.##\n#.#.#...........#.#.#.#.#.#.#.#.#.##\n#.#.#.#####.###.#.#.#.#.#.#.#.#.#.##\n#.#.#.....#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#######.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.........#...#.#.#.#.#.#.#.#.#.##\n#.###############.#.###.###.#.###.##\n#.................#.........#.....##\n####################################\n####################################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "1086477a254dd43bfb98d2656d3f72df60316cf21c7b51a3fd6b51c87182c922"}489{"id": "open_count-000488", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[9,2],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"6,-1\":\"outside\",\"6,0\":\"open\",\"6,1\":\"open\",\"6,2\":\"open\",\"6,3\":\"open\",\"6,4\":\"open\",\"6,5\":\"open\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"open\",\"7,3\":\"wall\",\"7,4\":\"wall\",\"7,5\":\"open\",\"8,-1\":\"outside\",\"8,0\":\"wall\",\"8,1\":\"wall\",\"8,2\":\"open\",\"8,3\":\"open\",\"8,4\":\"open\",\"8,5\":\"open\",\"9,-1\":\"outside\",\"9,0\":\"open\",\"9,1\":\"open\",\"9,2\":\"agent\",\"9,3\":\"open\",\"9,4\":\"wall\",\"9,5\":\"open\",\"10,-1\":\"outside\",\"10,0\":\"wall\",\"10,1\":\"open\",\"10,2\":\"open\",\"10,3\":\"wall\",\"10,4\":\"wall\",\"10,5\":\"open\",\"11,-1\":\"outside\",\"11,0\":\"open\",\"11,1\":\"open\",\"11,2\":\"open\",\"11,3\":\"open\",\"11,4\":\"wall\",\"11,5\":\"wall\",\"12,-1\":\"outside\",\"12,0\":\"outside\",\"12,1\":\"outside\",\"12,2\":\"outside\",\"12,3\":\"outside\",\"12,4\":\"outside\",\"12,5\":\"outside\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 4, "answer": "4", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "8dc4314b56ac02d936e0f618f1a041565c6883f7b29aea18c447796ddeb7b576"}490{"id": "move_legal-000489", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 7, column 10. Rows increase to the south and columns to the east.\nRow 5: column 8 is wall, column 9 is open, column 10 is open, column 11 is open, column 12 is wall\nRow 6: column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall\nRow 7: column 8 is open, column 9 is open, column 10 is the agent, column 11 is open, column 12 is open\nRow 8: column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall\nRow 9: column 8 is wall, column 9 is open, column 10 is open, column 11 is open, column 12 is open", "question": "The agent wants to move north. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "22e5bb54f461f5e2118da60888f389ac80a2d1001076fce21549b9e6e8218cdc"}491{"id": "only_way-000490", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (2,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#####\n...##\n.#A##\n.#.##\n.####", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 5, "answer": "more than one of them", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "8706765eefb8d958e70a7433321ce48a27557a680d3e9f5fdaa47eba507856d3"}492{"id": "blocked_pair-000491", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (31,5); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n...#.\n####.\n.#A..\n#####\n.....", "question": "Are BOTH the cell one step south and the cell one step west blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "9aad9e992953edcf67d7330188fa91c054f3fc34b92f93eec459b66f0d8d9447"}493{"id": "onward-000492", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[14,15],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"10,11\":\"wall\",\"10,12\":\"wall\",\"10,13\":\"wall\",\"10,14\":\"wall\",\"10,15\":\"wall\",\"10,16\":\"wall\",\"10,17\":\"wall\",\"10,18\":\"wall\",\"10,19\":\"open\",\"11,11\":\"open\",\"11,12\":\"wall\",\"11,13\":\"open\",\"11,14\":\"open\",\"11,15\":\"open\",\"11,16\":\"open\",\"11,17\":\"open\",\"11,18\":\"wall\",\"11,19\":\"open\",\"12,11\":\"open\",\"12,12\":\"wall\",\"12,13\":\"open\",\"12,14\":\"wall\",\"12,15\":\"wall\",\"12,16\":\"wall\",\"12,17\":\"open\",\"12,18\":\"wall\",\"12,19\":\"open\",\"13,11\":\"open\",\"13,12\":\"wall\",\"13,13\":\"open\",\"13,14\":\"wall\",\"13,15\":\"open\",\"13,16\":\"wall\",\"13,17\":\"open\",\"13,18\":\"wall\",\"13,19\":\"open\",\"14,11\":\"wall\",\"14,12\":\"wall\",\"14,13\":\"open\",\"14,14\":\"wall\",\"14,15\":\"agent\",\"14,16\":\"wall\",\"14,17\":\"open\",\"14,18\":\"wall\",\"14,19\":\"open\",\"15,11\":\"open\",\"15,12\":\"open\",\"15,13\":\"open\",\"15,14\":\"open\",\"15,15\":\"open\",\"15,16\":\"wall\",\"15,17\":\"open\",\"15,18\":\"wall\",\"15,19\":\"open\",\"16,11\":\"wall\",\"16,12\":\"wall\",\"16,13\":\"wall\",\"16,14\":\"wall\",\"16,15\":\"wall\",\"16,16\":\"wall\",\"16,17\":\"open\",\"16,18\":\"wall\",\"16,19\":\"open\",\"17,11\":\"open\",\"17,12\":\"open\",\"17,13\":\"open\",\"17,14\":\"open\",\"17,15\":\"open\",\"17,16\":\"open\",\"17,17\":\"open\",\"17,18\":\"wall\",\"17,19\":\"open\",\"18,11\":\"wall\",\"18,12\":\"wall\",\"18,13\":\"wall\",\"18,14\":\"wall\",\"18,15\":\"wall\",\"18,16\":\"wall\",\"18,17\":\"wall\",\"18,18\":\"wall\",\"18,19\":\"open\"}}", "question": "Suppose the agent moves one cell north. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "ec00cb68696c95d4fd39a3651c8019422e66118eed165532e2dd459cf9bbf93d"}494{"id": "open_count-000493", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 18. Rows increase to the south and columns to the east.\nRow -2: column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze, column 20 is outside the maze, column 21 is outside the maze\nRow -1: column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze, column 20 is outside the maze, column 21 is outside the maze\nRow 0: column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is wall, column 20 is open, column 21 is outside the maze\nRow 1: column 15 is open, column 16 is open, column 17 is open, column 18 is the agent, column 19 is open, column 20 is open, column 21 is outside the maze\nRow 2: column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is wall, column 21 is outside the maze\nRow 3: column 15 is open, column 16 is wall, column 17 is open, column 18 is open, column 19 is open, column 20 is open, column 21 is outside the maze\nRow 4: column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is open, column 21 is outside the maze", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 4, "answer": "4", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "6c8c1a7ac4824cdc06350ebbcbee88e154cee7d6ae8f1efce7abd36fc5f5b1b3"}495{"id": "first_move-000494", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,1), goal at (5,3).\n################\n#.............##\n#.###.###.###.##\n#...#.......#.##\n#.#.#####.#.#.##\n#..G#...#.#...##\n#####.###.#.#.##\n#.........#.#.##\n#.###.#####.####\n#.........#...##\n#########.###.##\n#...#...#...#.##\n#.#.#.#.###.#.##\n#A#...#.......##\n################\n################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "8d73b25d4e122d7e0de1bd38fd87bc9e8043227ba9d7b6652fa53c24d1861bfc"}496{"id": "snake_safe-000495", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (1,3), travelling north, length 3.\n..........\n...H......\n...o......\n...o......\n........*.\n..........\n..........\n..........\n..........\n..........", "question": "If the snake moves north on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "56ff69cbd2ee32ccda8402b8addd24eedd055cd7b45b04643c58270e3eb051fa"}497{"id": "reachable_within-000496", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,14), goal at (9,3).\n########################\n#...........#.....#...##\n#.#######.###.#.#.###.##\n#.#.....#.....#.#.#...##\n#.#####.#######.#.#.####\n#.....#...#.....#.#...##\n#####.###.#.#####.###.##\n#...#.....#.#.....#...##\n#.#.#####.#.#.#####.#.##\n#.#G#.....#.#.A.#...#.##\n#.#########.###.#.###.##\n#.#.......#.#.#.#.#...##\n#.#.#####.#.#.#.#.#.####\n#.#.....#...#.#...#...##\n#.#.###.#####.#######.##\n#.#...#.#.....#.....#.##\n#.#####.#.#.#.#.###.#.##\n#.....#.#.#.#.#...#...##\n#####.#.#.#.#####.######\n#...#.#.#.#.....#.#...##\n#.#.#.#.#.###.###.#.#.##\n#.#.....#...#.......#.##\n########################\n########################", "question": "Can the agent reach the goal in at most 78 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "2e77c48cc20f6f174f382c80db7a1e014183e7b64e5426d6fccd43f6983bfc8b"}498{"id": "first_move-000497", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (11,18), goal at (21,3).\n####################################\n#...#.....................#.......##\n#.#.###########.#.#######.###.###.##\n#.#...........#.#.#.....#...#.#...##\n#.###########.###.###.#.###.#.#.#.##\n#.#.....#...#...#...#.#.....#.#.#.##\n#.#####.#.#.###.###.#.#######.#.#.##\n#.....#...#...#.....#.......#.#.#.##\n#.###.#####.###############.#.#.#.##\n#.#...#...#.......#.......#.#.#.#.##\n###.###.#.#####.#.#.#.###.#.###.####\n#...#...#.....#.#.A.#.#...#...#...##\n#.###.#######.#######.#######.###.##\n#.#.....#.....#.......#.....#...#.##\n#.###.#.#####.#.#######.#.#.###.#.##\n#...#.#.....#.#.......#.#.#...#...##\n###.#.#####.#.###.###.#.#.#######.##\n#...#.#.....#...#...#.#.#.....#...##\n#.###.#.#######.#####.#.#####.#.####\n#.#...#...#...#.#.....#...#.#.#...##\n#.###.###.#.###.#.#.#####.#.#.###.##\n#.#G#.#...#.......#.#.....#.#.....##\n#.#.#.#.###.#########.#####.########\n#...#.#.#...#.........#...........##\n#######.#####.#########.#.#######.##\n#.....#.....#.#.........#.#.....#.##\n#.###.#####.#.###########.#.###.#.##\n#...#.......#.............#...#.#.##\n#.#.#########################.#.#.##\n#.#.......#.............#...#.#...##\n#.#######.#.#.#########.#.#.#.######\n#.#.....#...#.#.......#.#.#.#.....##\n#.#.###.#####.#.#####.###.#.#####.##\n#...#.......#.......#.....#.......##\n####################################\n####################################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "ea5ea8748315d38c6f00381dbf6f846b6c0de9e998f4b9ba0953c589b26d9da5"}499{"id": "cell_kind-000498", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[27,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"23,-1\":\"outside\",\"23,0\":\"open\",\"23,1\":\"open\",\"23,2\":\"open\",\"23,3\":\"open\",\"23,4\":\"open\",\"23,5\":\"open\",\"23,6\":\"open\",\"23,7\":\"open\",\"24,-1\":\"outside\",\"24,0\":\"wall\",\"24,1\":\"wall\",\"24,2\":\"open\",\"24,3\":\"wall\",\"24,4\":\"open\",\"24,5\":\"open\",\"24,6\":\"open\",\"24,7\":\"open\",\"25,-1\":\"outside\",\"25,0\":\"wall\",\"25,1\":\"open\",\"25,2\":\"open\",\"25,3\":\"open\",\"25,4\":\"wall\",\"25,5\":\"open\",\"25,6\":\"open\",\"25,7\":\"open\",\"26,-1\":\"outside\",\"26,0\":\"open\",\"26,1\":\"wall\",\"26,2\":\"open\",\"26,3\":\"open\",\"26,4\":\"wall\",\"26,5\":\"open\",\"26,6\":\"wall\",\"26,7\":\"wall\",\"27,-1\":\"outside\",\"27,0\":\"open\",\"27,1\":\"open\",\"27,2\":\"wall\",\"27,3\":\"agent\",\"27,4\":\"wall\",\"27,5\":\"open\",\"27,6\":\"open\",\"27,7\":\"wall\",\"28,-1\":\"outside\",\"28,0\":\"open\",\"28,1\":\"open\",\"28,2\":\"open\",\"28,3\":\"wall\",\"28,4\":\"wall\",\"28,5\":\"wall\",\"28,6\":\"wall\",\"28,7\":\"open\",\"29,-1\":\"outside\",\"29,0\":\"wall\",\"29,1\":\"open\",\"29,2\":\"wall\",\"29,3\":\"open\",\"29,4\":\"wall\",\"29,5\":\"open\",\"29,6\":\"open\",\"29,7\":\"open\",\"30,-1\":\"outside\",\"30,0\":\"open\",\"30,1\":\"open\",\"30,2\":\"open\",\"30,3\":\"open\",\"30,4\":\"wall\",\"30,5\":\"open\",\"30,6\":\"open\",\"30,7\":\"wall\",\"31,-1\":\"outside\",\"31,0\":\"open\",\"31,1\":\"open\",\"31,2\":\"open\",\"31,3\":\"wall\",\"31,4\":\"wall\",\"31,5\":\"open\",\"31,6\":\"wall\",\"31,7\":\"wall\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "92e0c7a62cfb69208290e3aeef48b01d711ca819ad270d218232c35e470419ff"}500{"id": "move_legal-000499", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 27, column 14. Rows increase to the south and columns to the east.\nRow 23: column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is open, column 15 is open, column 16 is wall, column 17 is open, column 18 is open\nRow 24: column 10 is wall, column 11 is open, column 12 is open, column 13 is wall, column 14 is wall, column 15 is open, column 16 is wall, column 17 is wall, column 18 is open\nRow 25: column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is wall, column 16 is open, column 17 is wall, column 18 is wall\nRow 26: column 10 is open, column 11 is open, column 12 is wall, column 13 is open, column 14 is open, column 15 is wall, column 16 is open, column 17 is open, column 18 is open\nRow 27: column 10 is wall, column 11 is open, column 12 is open, column 13 is wall, column 14 is the agent, column 15 is wall, column 16 is wall, column 17 is open, column 18 is open\nRow 28: column 10 is open, column 11 is open, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is open, column 16 is open, column 17 is open, column 18 is wall\nRow 29: column 10 is open, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is open, column 15 is open, column 16 is open, column 17 is wall, column 18 is wall\nRow 30: column 10 is open, column 11 is wall, column 12 is open, column 13 is open, column 14 is wall, column 15 is open, column 16 is open, column 17 is open, column 18 is open\nRow 31: column 10 is open, column 11 is open, column 12 is wall, column 13 is open, column 14 is open, column 15 is wall, column 16 is open, column 17 is open, column 18 is open", "question": "Is the cell one step east of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "8cb79428d9d6677c994579a55d010a72dc0aa8dab4fcfbda3a379e3127f87961"}501{"id": "distance_band-000500", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,12), goal at (0,10).\n##########G.###.\n###########...#.\n############A...\n############.#.#\n###########..#..\n##############..\n##############..\n###########....#\n###########..#.#\n###.######....##\n##....##..#..###\n####...#....####\n###..#..#...####\n####.##...######\n######..########\n######..########", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "54f1c8f4ffe263d86d59434a046392c965148d9df9ffb1318e612df2fe79da34"}502{"id": "only_way-000501", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (13,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX#...\nX####\nX#A..\nX####\nX####", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "485a213a71de413aff6829c5755e1b4103bd5c136bec1858ca61ebb084357eba"}503{"id": "cell_kind-000502", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,2); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXX\nXXXXXXX\nX######\nX#.A...\nX#.####\nX#...#.\nX###...", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "8bea473b2e94e0c5eac256ce1cb3e7e45769e38dced42911d0a174bbc6078d2a"}504{"id": "plan_survives-000503", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 23, column 3. Rows increase to the south and columns to the east.\nRow 19: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open\nRow 20: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall\nRow 21: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is open\nRow 22: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 23: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is the agent, column 4 is open, column 5 is open, column 6 is wall, column 7 is open\nRow 24: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open\nRow 25: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open\nRow 26: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 27: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall", "question": "The agent will attempt this plan in order: west, north, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "fc2817bfdbb9613a87eb5b7dca0600a2720d9c0524595ca3240fc81c7d681981"}505{"id": "first_move-000504", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,19), goal at (7,15).\n#####################\n#...................#\n#.#####...#######.#.#\n#.#.....#...#.....#.#\n#.#.###...#.#.#######\n#.#.........#.......#\n#.#.#.#.#.#.#######.#\n#...#.....#...#G....#\n#.#.#..####.#.#.###.#\n#.#.........#.#.....#\n#.#########.#.#####.#\n#.........#...#.#...#\n#.#.#.#...##.##.#.###\n#.#.....#.......#..A#\n#.#####.#.#.###.###.#\n#...#...#.#...#.....#\n#.###.#####.#.#.###.#\n#.....#.......#.....#\n#.#.#####.#.#.#####.#\n#.........#.......#.#\n#####################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "9981f4ffae9ba68498ef60eeafbe0a0d7811905ee148e823beb3bc44276582ae"}506{"id": "first_move-000505", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,9), goal at (1,11).\n################\n#.........#G..##\n#.##.##.#.###.##\n#.......#.....##\n#.#.###.#.###.##\n#...#...#.....##\n###.#.#.#####.##\n#.............##\n#.####.####.#.##\n#...#....A..#.##\n###...#.#.#.#.##\n#...#.#.#...#.##\n#.#####...###.##\n#.......#.....##\n################\n################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "1cf147727c8f227450a71d42aff63f22ff96763b31b99cfc3117d648af22693a"}507{"id": "reachable_within-000506", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,9), goal at (3,7).\n############\n#.........##\n##.####.####\n#.....#G..##\n#.###.###.##\n#...#....A##\n#.#.###.#.##\n#.........##\n#.#####.#.##\n#.#.....#.##\n############\n############", "question": "Can the agent reach the goal in at most 4 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "dcbb733853e9707947b90cb1737adfbef7123e06ccc12f387059a65bcbf6c9f6"}508{"id": "plan_survives-000507", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[35,35],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"32,32\":\"open\",\"32,33\":\"wall\",\"32,34\":\"open\",\"32,35\":\"open\",\"32,36\":\"outside\",\"32,37\":\"outside\",\"32,38\":\"outside\",\"33,32\":\"wall\",\"33,33\":\"open\",\"33,34\":\"open\",\"33,35\":\"wall\",\"33,36\":\"outside\",\"33,37\":\"outside\",\"33,38\":\"outside\",\"34,32\":\"open\",\"34,33\":\"open\",\"34,34\":\"open\",\"34,35\":\"open\",\"34,36\":\"outside\",\"34,37\":\"outside\",\"34,38\":\"outside\",\"35,32\":\"open\",\"35,33\":\"open\",\"35,34\":\"wall\",\"35,35\":\"agent\",\"35,36\":\"outside\",\"35,37\":\"outside\",\"35,38\":\"outside\",\"36,32\":\"outside\",\"36,33\":\"outside\",\"36,34\":\"outside\",\"36,35\":\"outside\",\"36,36\":\"outside\",\"36,37\":\"outside\",\"36,38\":\"outside\",\"37,32\":\"outside\",\"37,33\":\"outside\",\"37,34\":\"outside\",\"37,35\":\"outside\",\"37,36\":\"outside\",\"37,37\":\"outside\",\"37,38\":\"outside\",\"38,32\":\"outside\",\"38,33\":\"outside\",\"38,34\":\"outside\",\"38,35\":\"outside\",\"38,36\":\"outside\",\"38,37\":\"outside\",\"38,38\":\"outside\"}}", "question": "The agent will attempt this plan in order: south, south, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "87cb514934bbc45560913923a00cfe6dd5a1c8f49be9bf9edf602e5dba791a1e"}509{"id": "onward-000508", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 14, column 15. Rows increase to the south and columns to the east.\nRow 11: column 12 is open, column 13 is open, column 14 is open, column 15 is open, column 16 is open, column 17 is open, column 18 is wall\nRow 12: column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall\nRow 13: column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is open, column 17 is open, column 18 is wall\nRow 14: column 12 is wall, column 13 is wall, column 14 is wall, column 15 is the agent, column 16 is wall, column 17 is open, column 18 is wall\nRow 15: column 12 is open, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open, column 18 is open\nRow 16: column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall\nRow 17: column 12 is wall, column 13 is open, column 14 is open, column 15 is open, column 16 is wall, column 17 is open, column 18 is open", "question": "Suppose the agent moves one cell south. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "90c81a679ec8b587b149bc7724f9df3c944dde2d5039116349c821a94f858858"}510{"id": "goal_bearing-000509", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (27,1), goal at (0,27).\n...#...............#.#....#G\n#............#....#.........\n##....#.#.....##........#...\n...#..#.#.#...#.#..##.#...#.\n....#.#.##..#.#...#.#...#...\n......#........#.#...#......\n.##..................#......\n..#...#.....................\n#....#.##.......##..#.......\n....#.....##.#...#..........\n.#.....#...##.....#.##......\n##......##............#..#..\n.....#...#..#........#......\n.#.#...#..#.......#........#\n...........##...#..#.#...#..\n...#.#.............#.....#..\n.#..####...###.#........#...\n#....###.......##..........#\n....###..........##...#...##\n.....#........##.##.#....#..\n...#....#.......#.....#.....\n#.#..#.#...#...........#.#..\n......#..##...#..........#..\n....##.....#.#....##........\n....#....#####......##.##...\n......#..###...#.....#......\n#..#....#.#....###...#....#.\n.A#..#.....#................", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "fccd65204ef13cba510a62660dbd9c5a85bea4432764f698f5fca3358560756f"}511{"id": "plan_survives-000510", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[0,19],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,17\":\"outside\",\"-2,18\":\"outside\",\"-2,19\":\"outside\",\"-2,20\":\"outside\",\"-2,21\":\"outside\",\"-1,17\":\"outside\",\"-1,18\":\"outside\",\"-1,19\":\"outside\",\"-1,20\":\"outside\",\"-1,21\":\"outside\",\"0,17\":\"open\",\"0,18\":\"open\",\"0,19\":\"agent\",\"0,20\":\"open\",\"0,21\":\"outside\",\"1,17\":\"open\",\"1,18\":\"open\",\"1,19\":\"open\",\"1,20\":\"wall\",\"1,21\":\"outside\",\"2,17\":\"open\",\"2,18\":\"open\",\"2,19\":\"open\",\"2,20\":\"open\",\"2,21\":\"outside\"}}", "question": "The agent will attempt this plan in order: south, east, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "e2ecd65ad0912ed0fd311abd1fa050cf794454db0414035a40a282a4b6e0802e"}512{"id": "snake_safe-000511", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (3,1), travelling west, length 3.\n........\n........\n........\n.Hoo....\n........\n........\n.......*\n........", "question": "If the snake moves east on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "8c96f0de3baddee9d150bb44f39fe0b565d2e3c01dced470bf35ce84b64ff73c"}513{"id": "move_two_step-000512", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 3, column 7. Rows increase to the south and columns to the east.\nRow 1: column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is open\nRow 2: column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall\nRow 3: column 5 is open, column 6 is open, column 7 is the agent, column 8 is open, column 9 is open\nRow 4: column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall\nRow 5: column 5 is open, column 6 is wall, column 7 is open, column 8 is open, column 9 is open", "question": "The agent plans to move south, then east. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "bac8d2c6759a9b7969c7c96f00b4daa19d4175c7fc10368208771e31fbb28d0b"}514{"id": "onward-000513", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 9, column 0. Rows increase to the south and columns to the east.\nRow 6: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is wall\nRow 7: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is wall, column 3 is open\nRow 8: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is wall, column 3 is open\nRow 9: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is the agent, column 1 is open, column 2 is wall, column 3 is open\nRow 10: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is wall, column 3 is open\nRow 11: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall\nRow 12: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze", "question": "Suppose the agent moves one cell south. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "1323edf53c5255548b41fff971da2970fa4b216c9846a387b42e933d93ec94c8"}515{"id": "distance_band-000514", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,2), goal at (13,3).\n################\n#.....#.......##\n#.###.###.#.#.##\n#.........#...##\n#.#.#####.#.####\n#.#...........##\n###.#.#.#####.##\n#...#.......#.##\n#.###.###.###.##\n#.........#...##\n#.###.###.#.####\n#.....#...#.#.##\n#.###.#####.#.##\n#.AG#.........##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "e17380755718527bedcb5d0614cceeb9101de9255aac43b36ff3ad7f8d6ada9f"}516{"id": "move_two_step-000515", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 6, column 9. Rows increase to the south and columns to the east.\nRow 2: column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze\nRow 3: column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze\nRow 4: column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze\nRow 5: column 5 is open, column 6 is wall, column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze\nRow 6: column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is the agent, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze\nRow 7: column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze\nRow 8: column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze\nRow 9: column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze\nRow 10: column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze", "question": "The agent plans to move east, then east. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "5b633d8806eebe77d11716d8531729f6660d8489203903dab079ae78fa446386"}517{"id": "distance_band-000516", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,5), goal at (13,9).\n################\n#.....#.......##\n#.###.#.###.#.##\n#.#.#...#.#.#.##\n#.#.#####.#.#.##\n#.....#...#.#.##\n#####.#.###.#.##\n#...#...#...#.##\n#.#.#####.###.##\n#.#.......#...##\n#.#.#######.####\n#.#.#.....#...##\n#.###.###.###.##\n#....A..#G#...##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 1, "answer": "5 to 9 moves", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "e34b9b39df01d87bbdc964ae26fc1182aa40cf647753bb9b3abe4f7c6b1c5d71"}518{"id": "distance_band-000517", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 40x40. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,1), goal at (1,1).\n########################################\n#G..............#.......#.............##\n#.##.####.#.###.#.#####.#.#####.#.###.##\n#A#.....#.#...#.......#.......#.......##\n#.#.###.#.###.#..####...###.#.#.###.#.##\n#.#...#...........#...#...#...#...#...##\n#.###.#####.#.#.###.#####.###.#.#.###.##\n#.#...#.........#...........#.#...#...##\n#.#.#.###.##..#.#.#.#.#...#.#.###...####\n#...#...#.......#.....#.#.#.#.........##\n###.###...#.#.#.#.###.#.#.#.#####.###.##\n#.......#.....#.........#...#.....#...##\n#.##.##.#.###.####..#.###.###.##.##.#.##\n#.#.....#...#.............#...#.....#.##\n#.#.#.##########..#####.###.#####.#.#.##\n#.#.#.................................##\n#.#.#.#.#.#######.#.###.#.#.#.##.##.####\n#.#.#.#.#.......#.#...........#.......##\n#.#...#.##..###.#.##.##.#.#####.#.###.##\n#.#.#...........#.....#.#.......#.....##\n#...###.#.###.#.#.#.#.#.#...#.#.###.#.##\n#...#...#.#.....#...#.....#.#.......#.##\n###.#.###.#####.###.#.###.#.#.#.###.#.##\n#...#...#...........#.#.....#.#...#.#.##\n#.#.###.#.#.##.##.#.###.#########.#.#.##\n#.#...#...#.....#...#.....#.......#...##\n#.#.#.###.#.###...#.#.#.#.#.#.###.###.##\n#...#...#.#.....#.....#.......#.......##\n#.#.###.#.#.#####.#.#.#.#.###.#.#.###.##\n#.#.#.#...#.....#...........#.#...#...##\n#.#.#.#####.#.#.#####.##..###...#.#.####\n#.#...#.....#.................#.#...#.##\n#.###.#.#####.##.#.##.#.#.#.#.#.#.###.##\n#...#.................#.#.....#...#...##\n#.#.#.############.##.#.#####...#####.##\n#.#.#...#.............#...............##\n#.#.###.#.###########.#.#.######.#..#.##\n#.#.......#...........................##\n########################################\n########################################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "c7e89237e647bd5359a5fea4b9d63cbef5969103b28338782c120ecb24a93a5f"}519{"id": "snake_safe-000518", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,5), travelling south, length 3.\n........\n........\n........\n........\n........\n....oo..\n.....H*.\n........", "question": "If the snake moves north on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "5f4961d87b3665ccb3290a4b3048cb5af53c11d674615a0431f158004538d87a"}520{"id": "distance_band-000519", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,7), goal at (1,2).\n################\n#.G...#.......##\n#.###.###.#.#.##\n#.#.#..A#.#.#.##\n#.#.#.#.###.#.##\n#...........#.##\n#.#.#####.#.#.##\n#...#.......#.##\n#.#.#.#.##.##.##\n#.....#.....#.##\n#.#########.#.##\n#.#.........#.##\n#.###.#.#####.##\n#.............##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 1, "answer": "5 to 9 moves", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "032dfc7601753f2e569fb964d67cf22fc924c010d969d6a0627bcdc2d6c904ad"}521{"id": "plan_survives-000520", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 5, column 4. Rows increase to the south and columns to the east.\nRow 2: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall, column 6 is open, column 7 is wall\nRow 3: column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open\nRow 4: column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open\nRow 5: column 1 is open, column 2 is open, column 3 is open, column 4 is the agent, column 5 is open, column 6 is wall, column 7 is open\nRow 6: column 1 is wall, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall\nRow 7: column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open\nRow 8: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall", "question": "The agent will attempt this plan in order: west, east, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3d8e13d10efca0e6bb0b772534bfed0930e945252da7dff8ae804978572ba177"}522{"id": "snake_safe-000521", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (9,7), travelling south, length 3.\n............\n............\n............\n............\n............\n............\n............\n.......o....\n.......o....\n.*.....H....\n............\n............", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "7e8826e2ccdaad115ab3c2228add4bceba9ae274eec136f58cf6d51fe47f5190"}523{"id": "reachable_within-000522", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,18), goal at (1,7).\n########################\n#......G#...........#.##\n#.###########.###.#.#.##\n#...........#.#...#...##\n###########.###.#####.##\n#...#.....#...#...#...##\n#.#.#.#.#####.#.#.######\n#.#...#...#...#.#.....##\n#.#######.#.###.#####.##\n#.#.....#...#.#.#.A...##\n#.###.#######.#.#.###.##\n#...#.....#...#.#.#...##\n###.###.#.#.#.#.#.###.##\n#.#...#.#...#...#...#.##\n#.###.#.#####.#####.#.##\n#.....#.#...#.#.#...#.##\n#.#######.#.#.#.#.###.##\n#...#.....#.#...#...#.##\n###.#.#####.#######.#.##\n#.#.#.#...#.......#.#.##\n#.#.#.#.#.#######.#.#.##\n#.....#.#...........#.##\n########################\n########################", "question": "Can the agent reach the goal in at most 123 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "d517cad05ba03a563e82896d164ec92e367de9179a2f63ede06f19c7268be545"}524{"id": "only_way-000523", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[25,0],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"22,-3\":\"outside\",\"22,-2\":\"outside\",\"22,-1\":\"outside\",\"22,0\":\"wall\",\"22,1\":\"wall\",\"22,2\":\"wall\",\"22,3\":\"wall\",\"23,-3\":\"outside\",\"23,-2\":\"outside\",\"23,-1\":\"outside\",\"23,0\":\"wall\",\"23,1\":\"wall\",\"23,2\":\"wall\",\"23,3\":\"wall\",\"24,-3\":\"outside\",\"24,-2\":\"outside\",\"24,-1\":\"outside\",\"24,0\":\"open\",\"24,1\":\"wall\",\"24,2\":\"open\",\"24,3\":\"wall\",\"25,-3\":\"outside\",\"25,-2\":\"outside\",\"25,-1\":\"outside\",\"25,0\":\"agent\",\"25,1\":\"open\",\"25,2\":\"open\",\"25,3\":\"wall\",\"26,-3\":\"outside\",\"26,-2\":\"outside\",\"26,-1\":\"outside\",\"26,0\":\"wall\",\"26,1\":\"open\",\"26,2\":\"open\",\"26,3\":\"open\",\"27,-3\":\"outside\",\"27,-2\":\"outside\",\"27,-1\":\"outside\",\"27,0\":\"open\",\"27,1\":\"open\",\"27,2\":\"open\",\"27,3\":\"open\",\"28,-3\":\"outside\",\"28,-2\":\"outside\",\"28,-1\":\"outside\",\"28,0\":\"open\",\"28,1\":\"wall\",\"28,2\":\"open\",\"28,3\":\"open\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 5, "answer": "more than one of them", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "23842b3178b07376721ee554dd42eecdf6a5aff152ade3cbdca13612b9e260ec"}525{"id": "first_move-000524", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (6,2), goal at (0,9).\n#........G##\n..###..##.##\n.....#....##\n.#..#..#..##\n...#.......#\n#....#......\n..A#........\n....#....#.#\n....#.#.....\n..#.........\n....#...#...\n.....#.###..", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "fb2f169672250ad90b367808867ee849463e5661c6ad26d711c7cb93e03a8c66"}526{"id": "reachable_within-000525", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (11,8), goal at (15,9).\n...#.......#...#\n#.....#..#.#....\n..##..#.#.......\n#..#....#.#..#.#\n###.........#...\n##......##.#...#\n###...##.#..#.##\n##..#.....#.##..\n...######......#\n#.......#####...\n.##..#.#.#.###..\n...#.##.A..###..\n.......##...#...\n.....#...##.#..#\n#..#.#..##......\n.....#...G###..#", "question": "Can the agent reach the goal in at most 49 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "d854a9a3eea5ad13ee97206fe684a066b40a556f1576d84977098684a5d630bb"}527{"id": "plan_survives-000526", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 3, column 4. Rows increase to the south and columns to the east.\nRow -1: column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow 0: column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall\nRow 1: column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is open\nRow 2: column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 3: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is the agent, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 4: column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall\nRow 5: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 6: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 7: column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall", "question": "The agent will attempt this plan in order: east, west, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "d6b168248cebf98b76668d933edbea34cd9f02ec25b8874312004178101c5cc8"}528{"id": "snake_food-000527", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,7), travelling east, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n.....ooH....*.\n..............\n..............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b43323c48e089c68aa72691e7cde1f4ce3bd765c6f1d4790402725433cabb77d"}529{"id": "stochastic-000528", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 12, column 12. Rows increase to the south and columns to the east.\nRow 8: column 8 is open, column 9 is open, column 10 is wall, column 11 is wall, column 12 is open, column 13 is open, column 14 is wall, column 15 is open, column 16 is outside the maze\nRow 9: column 8 is open, column 9 is open, column 10 is open, column 11 is open, column 12 is wall, column 13 is open, column 14 is open, column 15 is open, column 16 is outside the maze\nRow 10: column 8 is open, column 9 is open, column 10 is wall, column 11 is open, column 12 is open, column 13 is open, column 14 is open, column 15 is open, column 16 is outside the maze\nRow 11: column 8 is open, column 9 is open, column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is open, column 15 is open, column 16 is outside the maze\nRow 12: column 8 is open, column 9 is open, column 10 is wall, column 11 is open, column 12 is the agent, column 13 is wall, column 14 is open, column 15 is wall, column 16 is outside the maze\nRow 13: column 8 is open, column 9 is open, column 10 is open, column 11 is wall, column 12 is open, column 13 is open, column 14 is open, column 15 is open, column 16 is outside the maze\nRow 14: column 8 is wall, column 9 is open, column 10 is open, column 11 is wall, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is outside the maze\nRow 15: column 8 is open, column 9 is wall, column 10 is open, column 11 is open, column 12 is wall, column 13 is open, column 14 is open, column 15 is wall, column 16 is outside the maze\nRow 16: column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.1333333333333333, 0.8666666666666667], "deterministic": false, "fingerprint": "34293ee7b3628811e5fa3d6a8af90f5d87fa737aa1a03d97e705f4ae2705d0fe"}530{"id": "reachable_within-000529", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (8,0), goal at (11,11).\n.........###\n##.......###\n.......#####\n.....#.#..##\n...#.....###\n...#..#...##\n.#....#.####\n........####\nA#..#....#.#\n..####......\n..#####.....\n.######...#G", "question": "Can the agent reach the goal in at most 20 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "552c7a1d938369fe9e9f5951faf7414541141799a4dccb7793aa8127b560d613"}531{"id": "goal_bearing-000530", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,4), goal at (7,1).\n#########\n#.......#\n#.#####.#\n#.....#.#\n###.###.#\n#...#...#\n#.###.#.#\n#G#.A.#.#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "991408df210dbec48aee682cad4cd578af637a246f9af5f3527414da0d2b9dc8"}532{"id": "first_move-000531", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,9), goal at (5,5).\n############\n#.........##\n#.#######A##\n#.#.....#.##\n#.#.###.#.##\n#.#.#G#.#.##\n#.#.#.#.#.##\n#.#...#.#.##\n#.#####.#.##\n#.......#.##\n############\n############", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "f6bf20ef5e9801f6259f2ba01005172ea0f9e18a4e8266f94b223a52d6714f6a"}533{"id": "cell_kind-000532", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (25,0); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXX...##\nXXXX.....\nXXXX.##..\nXXXX....#\nXXXXA....\nXXXX...#.\nXXXX.#...\nXXXXXXXXX\nXXXXXXXXX", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "884165654d00547879eb6976f33ac2d53cf36adfa55a705f52bd3aa1b6740d06"}534{"id": "distance_band-000533", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (15,15), goal at (0,1).\n#G#..#.......#..\n#.#.........#...\n#..#..........##\n..#....#.....#..\n........#...#...\n.###..##...#..#.\n##.....##.....#.\n....#....#......\n....#..#.......#\n......#.....#.##\n###..#.....#..#.\n#..#............\n#.............#.\n.#....#...#.....\n.#.#...#.###...#\n.............#.A", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 3, "answer": "20 to 39 moves", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "30c6d8d612a1b50bd6248aed374f2432e00a2ce735fbd8cc468a06573a8945be"}535{"id": "move_two_step-000534", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (33,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n....#\n#..##\n#.A..\n####.\n####.", "question": "The agent plans to move south, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "2794dd669c8de9fef395eacc3d0b82d461b3ff3dd0b91ef054941643283846fb"}536{"id": "cell_kind-000535", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 11, column 5. Rows increase to the south and columns to the east.\nRow 8: column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall\nRow 9: column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is open\nRow 10: column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall\nRow 11: column 2 is open, column 3 is open, column 4 is open, column 5 is the agent, column 6 is wall, column 7 is open, column 8 is open\nRow 12: column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall\nRow 13: column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is open\nRow 14: column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "d5ef45aa60ec22c7a08391db13a7dd40ed5b991353564a83e4183e3d6fc13e4e"}537{"id": "goal_bearing-000536", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,7), goal at (7,1).\n#########\n#.#.....#\n#.#.###.#\n#.......#\n#####.#.#\n#.....#.#\n#.#.###.#\n#G..#..A#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "be18c898c5a399a16ea415df3f04dfe31654c7694c778b7ca932d95888a85fdb"}538{"id": "onward-000537", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,15); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXX\nXXXXXXX\n#..#XXX\n##.AXXX\n##..XXX\n....XXX\n#.##XXX", "question": "Suppose the agent moves one cell south. From that new cell, would a further step east be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "cf3868e5c1f6b762bbafd65f0a0a3d21a5e2fecfaa57b4628dd5c4730ad701b2"}539{"id": "snake_safe-000538", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,8), travelling east, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n...*..........\n......ooH.....\n..............\n..............\n..............\n..............\n..............\n..............", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "db266f0ead0f9401bc54b87eeab33ba4e5ac06311d822b193844e8a37fa71d7c"}540{"id": "only_way-000539", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXX\nX####\nX#A..\nX#.##\nX#.#.", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 5, "answer": "more than one of them", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "4dfd88d06138613b884177ac9b91ce82802c374508227b42db3da9382185b74d"}541{"id": "open_count-000540", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,10); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXX\n#####\n..A.#\n#####\n.....", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 2, "answer": "2", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "93e32f8bc812cbce90cf81a791c8a6ba9fe448d448c8063eee726bd98d0bd175"}542{"id": "move_two_step-000541", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (6,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.####\n.#...\n..A#.\n.#.#.\n.###.", "question": "The agent plans to move west, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "d1282b472d194d03be4b51baff3567c67c375c6ba0aa3cb9f2881d4c98092d44"}543{"id": "stochastic-000542", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[15,0],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"11,-4\":\"outside\",\"11,-3\":\"outside\",\"11,-2\":\"outside\",\"11,-1\":\"outside\",\"11,0\":\"open\",\"11,1\":\"open\",\"11,2\":\"wall\",\"11,3\":\"wall\",\"11,4\":\"wall\",\"12,-4\":\"outside\",\"12,-3\":\"outside\",\"12,-2\":\"outside\",\"12,-1\":\"outside\",\"12,0\":\"open\",\"12,1\":\"open\",\"12,2\":\"open\",\"12,3\":\"open\",\"12,4\":\"open\",\"13,-4\":\"outside\",\"13,-3\":\"outside\",\"13,-2\":\"outside\",\"13,-1\":\"outside\",\"13,0\":\"wall\",\"13,1\":\"open\",\"13,2\":\"open\",\"13,3\":\"open\",\"13,4\":\"wall\",\"14,-4\":\"outside\",\"14,-3\":\"outside\",\"14,-2\":\"outside\",\"14,-1\":\"outside\",\"14,0\":\"open\",\"14,1\":\"open\",\"14,2\":\"wall\",\"14,3\":\"open\",\"14,4\":\"open\",\"15,-4\":\"outside\",\"15,-3\":\"outside\",\"15,-2\":\"outside\",\"15,-1\":\"outside\",\"15,0\":\"agent\",\"15,1\":\"open\",\"15,2\":\"open\",\"15,3\":\"open\",\"15,4\":\"open\",\"16,-4\":\"outside\",\"16,-3\":\"outside\",\"16,-2\":\"outside\",\"16,-1\":\"outside\",\"16,0\":\"outside\",\"16,1\":\"outside\",\"16,2\":\"outside\",\"16,3\":\"outside\",\"16,4\":\"outside\",\"17,-4\":\"outside\",\"17,-3\":\"outside\",\"17,-2\":\"outside\",\"17,-1\":\"outside\",\"17,0\":\"outside\",\"17,1\":\"outside\",\"17,2\":\"outside\",\"17,3\":\"outside\",\"17,4\":\"outside\",\"18,-4\":\"outside\",\"18,-3\":\"outside\",\"18,-2\":\"outside\",\"18,-1\":\"outside\",\"18,0\":\"outside\",\"18,1\":\"outside\",\"18,2\":\"outside\",\"18,3\":\"outside\",\"18,4\":\"outside\",\"19,-4\":\"outside\",\"19,-3\":\"outside\",\"19,-2\":\"outside\",\"19,-1\":\"outside\",\"19,0\":\"outside\",\"19,1\":\"outside\",\"19,2\":\"outside\",\"19,3\":\"outside\",\"19,4\":\"outside\"}}", "question": "The actuator executes the intended move with probability 0.80; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.8666666666666667, 0.1333333333333333], "deterministic": false, "fingerprint": "d4e19a628f2fbb2b923592997881688ffce6095b64d52e13196458bd0c719bc4"}544{"id": "reachable_within-000543", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (29,12), goal at (5,29).\n################################\n#.....#.....#.........#.......##\n#.#.#.#.#.#.#.#.####..#.###.#.##\n#.#.....#.....#.#.............##\n#.#.#.###.##.##.#.#####.#.##.###\n#.........#...#.........#....G##\n#.#####.#.#.#.###.#.###.#.###.##\n#.......#...#...#.#...#.#...#.##\n###.#####..####...###.#.#.#.#.##\n#.............#.#...#.........##\n#####.#######...#.###.#.#.###.##\n#...#.........#.#...#.#.....#.##\n#.#.#.###.#.###.###.#.###.###.##\n#.#.....#.#.#...#...#...#.#...##\n#.#.##.##.###.#.#.#.###.#.#.#.##\n#.#...............#...#.#...#.##\n#.#####.###.#####.#.###.#.###.##\n#.....#.#...#.........#.....#.##\n#.###.###.###.###.#.#.#.#.#.#.##\n#...#.........#...#.....#.....##\n#.#.#.#.#.###.#.###.#..##.###.##\n#...#...#.....#.#.#.#...#...#.##\n#.###.###.#.#.#.#.#.#.#.###.#.##\n#.#...#.....#.....#...#.#.....##\n#.#.#.#.###.#.#########.########\n#.....#.#...#...#.....#...#...##\n###.#.#.###.#.###.###.###.#.#.##\n#.........#.....#.......#...#.##\n#.###.#########.###.###.#####.##\n#...........A.#...............##\n################################\n################################", "question": "Can the agent reach the goal in at most 60 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "7d5df224649ed239d9e0268eb9a6b7980ea49553a00d7a46d1deeed0e151d14d"}545{"id": "reachable_within-000544", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,3), goal at (5,7).\n############\n#.........##\n#.#.########\n#.#.#.....##\n#.###.###.##\n#..A#.#G..##\n#.#.#.######\n#.#...#...##\n#.#####.#.##\n#.......#.##\n############\n############", "question": "Can the agent reach the goal in at most 13 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "84ae5407065886d31fa63027cebe02e6cf8511cd903bce08713e4ae005f3f8a5"}546{"id": "move_two_step-000545", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 22, column 9. Rows increase to the south and columns to the east.\nRow 18: column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open\nRow 19: column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is open, column 10 is open, column 11 is open, column 12 is wall, column 13 is open\nRow 20: column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall\nRow 21: column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open, column 12 is open, column 13 is open\nRow 22: column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is the agent, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall\nRow 23: column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is open, column 11 is open, column 12 is open, column 13 is open\nRow 24: column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open\nRow 25: column 5 is open, column 6 is wall, column 7 is open, column 8 is open, column 9 is open, column 10 is open, column 11 is open, column 12 is wall, column 13 is open\nRow 26: column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open", "question": "The agent plans to move east, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "e78164902bb259c455b0c213bb79b91b5c925c9d7885b534a6d71079fccf5adf"}547{"id": "snake_safe-000546", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (10,10), travelling south, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n...*.....oo...\n..........H...\n..............\n..............\n..............", "question": "If the snake moves west on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "2473a5dae492388daba33e1dc645e7a85b8c4bdcedbf2d9a9badf4f1a42d12f7"}548{"id": "move_legal-000547", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[1,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,-2\":\"outside\",\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"agent\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"open\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"wall\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\"}}", "question": "If the agent attempts one cell east (col+1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "b379f5f54f90e3f9366d06f1614d5285e87bbc687ce3fafb4ac91a700b0ea113"}549{"id": "stochastic-000548", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (7,6); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n..#.#\n###.#\n..A.#\n#####\nXXXXX", "question": "The actuator executes the intended move with probability 0.75; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.16666666666666663, 0.8333333333333334], "deterministic": false, "fingerprint": "f45a318ab8103d9b6040046098147aa017a8a16451eb1d346d2c3fde9f118509"}550{"id": "snake_safe-000549", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,5), travelling south, length 3.\n..........\n..........\n..........\n..........\n..........\n.*...o....\n.....o....\n.....H....\n..........\n..........", "question": "If the snake moves north on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "bd0ecadc611063c3d45af0001dd1243874e9451f1685daef42d235280f644e13"}551{"id": "cell_kind-000550", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (31,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n...#...#.\n####.#.#.\n.......#.\n.#####.##\n.#..A#...\n.#.#####.\n...#.....\n#########\n#########", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "6d0d0b4beb814e09e296247d059624f184526fb99b738621855d27deea1b7e0c"}552{"id": "snake_safe-000551", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (4,5), travelling north, length 3.\n..........\n..........\n..........\n..........\n.....H....\n....oo....\n...*......\n..........\n..........\n..........", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "d195602f3739d1a558fc52f70ff53687efbe2a00df468a9a1badf62c0de61a1b"}553{"id": "first_move-000552", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (8,5), goal at (27,27).\n##..#......#############...#\n....#..#...#############....\n....#....##.############....\n...#..........#######.#.....\n....#..........#.####.##...#\n....#.....##...#..##........\n.#....#..#...##.#......#....\n......#.......#.#.##.....#.#\n...#.A........#.............\n.....#.##.......#.........#.\n..#.....#.............#.##..\n.###....#.#.#..#............\n#.............#............#\n.............#.#...........#\n........#.#..#..##..........\n..##.........#.......##.....\n...#...#.......#..#...#.....\n..........#.###.#.#..##.##..\n#..###......#...#.##.#..#...\n....####..#......#......#...\n.....#........#...#.....#..#\n##....#.#....###...#....#.#.\n###...........#...#####..#..\n..#..#.....##......#..#...#.\n..#..###.#.#.....#..#.##....\n..#......##...........#...#.\n........#...........#......#\n........#......#........#..G", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "de9d014f429e11786a6fb4768b4ab9886463fa7f2cbe7c17da7c16de991a66a2"}554{"id": "move_legal-000553", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[5,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"wall\",\"1,5\":\"open\",\"1,6\":\"wall\",\"1,7\":\"open\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"open\",\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"open\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"open\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"agent\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"open\",\"6,6\":\"wall\",\"6,7\":\"open\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"wall\",\"7,3\":\"open\",\"7,4\":\"wall\",\"7,5\":\"open\",\"7,6\":\"wall\",\"7,7\":\"open\",\"8,-1\":\"outside\",\"8,0\":\"wall\",\"8,1\":\"open\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"open\",\"8,6\":\"wall\",\"8,7\":\"wall\",\"9,-1\":\"outside\",\"9,0\":\"wall\",\"9,1\":\"open\",\"9,2\":\"open\",\"9,3\":\"open\",\"9,4\":\"open\",\"9,5\":\"open\",\"9,6\":\"open\",\"9,7\":\"open\"}}", "question": "Is the cell one step south of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "5937d3d8444e3bcfe3156896df7b33e91558359525b46bcbc980119fc79c7bf1"}555{"id": "only_way-000554", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 20, column 21. Rows increase to the south and columns to the east.\nRow 18: column 19 is open, column 20 is wall, column 21 is open, column 22 is wall, column 23 is open\nRow 19: column 19 is open, column 20 is wall, column 21 is open, column 22 is wall, column 23 is open\nRow 20: column 19 is open, column 20 is wall, column 21 is the agent, column 22 is wall, column 23 is open\nRow 21: column 19 is open, column 20 is wall, column 21 is open, column 22 is wall, column 23 is open\nRow 22: column 19 is open, column 20 is wall, column 21 is wall, column 22 is wall, column 23 is open", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 5, "answer": "more than one of them", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "496f017884959e07737ea34b4517888fd35dceaa310460c9e0e643a841fe28e6"}556{"id": "plan_survives-000555", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,11); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n###.XXX\n.##.XXX\n.##.XXX\n.#.AXXX\n.##.XXX\n#...XXX\n....XXX", "question": "The agent will attempt this plan in order: north, north, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "bc7e3347dd02a1ac6e997a3c28e945b08c8aff4b3ac80814e51bdf3f5bb2b47c"}557{"id": "open_count-000556", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 0, column 1. Rows increase to the south and columns to the east.\nRow -4: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow -3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow -2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow -1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 0: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is the agent, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 4: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is open, column 3 is wall, column 4 is open, column 5 is open", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 3, "answer": "3", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "030b88862b3165b62dc5daacbb8bf56898b72e1d5af319884cdaf3abffe21f9c"}558{"id": "cell_kind-000557", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[9,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"7,7\":\"open\",\"7,8\":\"wall\",\"7,9\":\"open\",\"7,10\":\"wall\",\"7,11\":\"wall\",\"8,7\":\"open\",\"8,8\":\"wall\",\"8,9\":\"open\",\"8,10\":\"wall\",\"8,11\":\"wall\",\"9,7\":\"open\",\"9,8\":\"wall\",\"9,9\":\"agent\",\"9,10\":\"wall\",\"9,11\":\"wall\",\"10,7\":\"wall\",\"10,8\":\"wall\",\"10,9\":\"wall\",\"10,10\":\"wall\",\"10,11\":\"wall\",\"11,7\":\"wall\",\"11,8\":\"wall\",\"11,9\":\"wall\",\"11,10\":\"wall\",\"11,11\":\"wall\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "e14a047a7de4fd3880830f28656c6762b9d3fc257202242aeb1d8921729ff698"}559{"id": "blocked_pair-000558", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-3,5\":\"outside\",\"-3,6\":\"outside\",\"-3,7\":\"outside\",\"-3,8\":\"outside\",\"-3,9\":\"outside\",\"-3,10\":\"outside\",\"-3,11\":\"outside\",\"-3,12\":\"outside\",\"-3,13\":\"outside\",\"-2,5\":\"outside\",\"-2,6\":\"outside\",\"-2,7\":\"outside\",\"-2,8\":\"outside\",\"-2,9\":\"outside\",\"-2,10\":\"outside\",\"-2,11\":\"outside\",\"-2,12\":\"outside\",\"-2,13\":\"outside\",\"-1,5\":\"outside\",\"-1,6\":\"outside\",\"-1,7\":\"outside\",\"-1,8\":\"outside\",\"-1,9\":\"outside\",\"-1,10\":\"outside\",\"-1,11\":\"outside\",\"-1,12\":\"outside\",\"-1,13\":\"outside\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"0,8\":\"wall\",\"0,9\":\"wall\",\"0,10\":\"wall\",\"0,11\":\"wall\",\"0,12\":\"outside\",\"0,13\":\"outside\",\"1,5\":\"open\",\"1,6\":\"wall\",\"1,7\":\"open\",\"1,8\":\"open\",\"1,9\":\"agent\",\"1,10\":\"wall\",\"1,11\":\"wall\",\"1,12\":\"outside\",\"1,13\":\"outside\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"open\",\"2,8\":\"wall\",\"2,9\":\"open\",\"2,10\":\"wall\",\"2,11\":\"wall\",\"2,12\":\"outside\",\"2,13\":\"outside\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"3,8\":\"wall\",\"3,9\":\"open\",\"3,10\":\"wall\",\"3,11\":\"wall\",\"3,12\":\"outside\",\"3,13\":\"outside\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"open\",\"4,8\":\"wall\",\"4,9\":\"wall\",\"4,10\":\"wall\",\"4,11\":\"wall\",\"4,12\":\"outside\",\"4,13\":\"outside\",\"5,5\":\"open\",\"5,6\":\"open\",\"5,7\":\"open\",\"5,8\":\"open\",\"5,9\":\"open\",\"5,10\":\"wall\",\"5,11\":\"wall\",\"5,12\":\"outside\",\"5,13\":\"outside\"}}", "question": "Are BOTH the cell one step north and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "308c66010a111ee8dc8c215a48418352a9f575d5ceadbc195e15b02cdb105c6d"}560{"id": "first_move-000559", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,5), goal at (1,3).\n#########\n#..G#...#\n#.###A#.#\n#.#...#.#\n#.#.#####\n#.#.#...#\n#.#.#.#.#\n#.....#.#\n#########", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "06b2b2c2ae70b1cd0f7d97994552677ca3bec90ed5c4a1ffab609aa298f10409"}561{"id": "move_two_step-000560", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,8); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX\n########X\n#...A.##X\n#.###.##X\n#.#.#.##X\n###.#.##X\n..#.#.##X", "question": "The agent plans to move east, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "1dd9419d823b026c6a691f1b440afaf17778c5c2c8c35da2d49136e2683cac6d"}562{"id": "first_move-000561", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (10,13), goal at (11,7).\n################\n#.........#...##\n#.###.###.#.#.##\n#.#...#.....#.##\n###.###.#####.##\n#...#...#...#.##\n#.#.#####.#.#.##\n#.#.#.....#.#.##\n#.###.#####.#.##\n#...#.....#.#.##\n#.#.#.#####.#A##\n#.#...#G..#.#.##\n#.#######.#.#.##\n#.......#...#.##\n################\n################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "8ebce4c5fd29eff733d9574c3bc33a65ccc40f1d12fecda5f49ceac917d285fc"}563{"id": "blocked_pair-000562", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[5,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"agent\",\"5,4\":\"wall\",\"5,5\":\"open\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"wall\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"7,1\":\"open\",\"7,2\":\"open\",\"7,3\":\"open\",\"7,4\":\"open\",\"7,5\":\"open\"}}", "question": "Are BOTH the cell one step west and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "86ff571eec78874f8bdca62730e68198b7a1197386bef668789aee1eede657d3"}564{"id": "snake_safe-000563", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,4), travelling west, length 3.\n..........\n..........\n..........\n..........\n..........\n.*...o....\n....Ho....\n..........\n..........\n..........", "question": "If the snake moves east on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "9d3ae0f2052abf94e2dc585998eae459ad624adc63c9fb78dd4813adc69f80e1"}565{"id": "plan_survives-000564", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (0,4); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXX\nXXXXXXX\nXXXXXXX\n...A...\n.......\n...##.#\n.......", "question": "The agent will attempt this plan in order: north, west, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "afc031a6f3b7b7092d99bb11f7cb3f6c807517f8da7bedd81635e637f394821c"}566{"id": "snake_safe-000565", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (1,5), travelling north, length 3.\n..........\n.....H..*.\n.....o....\n.....o....\n..........\n..........\n..........\n..........\n..........\n..........", "question": "If the snake moves north on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "04b44a05e9162a719d665ad50ff6d26e24c4468d5829a99863ff4d8240e16735"}567{"id": "plan_survives-000566", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,-3\":\"outside\",\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"0,-3\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"1,-3\":\"outside\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"2,-3\":\"outside\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"3,-3\":\"outside\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"agent\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"4,-3\":\"outside\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"wall\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"5,-3\":\"outside\",\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"6,-3\":\"outside\",\"6,-2\":\"outside\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"open\",\"7,-3\":\"outside\",\"7,-2\":\"outside\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"wall\",\"7,3\":\"open\",\"7,4\":\"wall\",\"7,5\":\"open\"}}", "question": "The agent will attempt this plan in order: west, east, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "7a40fa62ec39f3701aeceb2ca0a6ccb0843c9fcbf0543d7811af627e1b836af6"}568{"id": "only_way-000567", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[7,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"3,-3\":\"outside\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"wall\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"4,-3\":\"outside\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"open\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\",\"5,-3\":\"outside\",\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"open\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"open\",\"5,5\":\"open\",\"6,-3\":\"outside\",\"6,-2\":\"outside\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"wall\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"open\",\"7,-3\":\"outside\",\"7,-2\":\"outside\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"agent\",\"7,2\":\"open\",\"7,3\":\"wall\",\"7,4\":\"open\",\"7,5\":\"open\",\"8,-3\":\"outside\",\"8,-2\":\"outside\",\"8,-1\":\"outside\",\"8,0\":\"wall\",\"8,1\":\"wall\",\"8,2\":\"open\",\"8,3\":\"open\",\"8,4\":\"open\",\"8,5\":\"open\",\"9,-3\":\"outside\",\"9,-2\":\"outside\",\"9,-1\":\"outside\",\"9,0\":\"outside\",\"9,1\":\"outside\",\"9,2\":\"outside\",\"9,3\":\"outside\",\"9,4\":\"outside\",\"9,5\":\"outside\",\"10,-3\":\"outside\",\"10,-2\":\"outside\",\"10,-1\":\"outside\",\"10,0\":\"outside\",\"10,1\":\"outside\",\"10,2\":\"outside\",\"10,3\":\"outside\",\"10,4\":\"outside\",\"10,5\":\"outside\",\"11,-3\":\"outside\",\"11,-2\":\"outside\",\"11,-1\":\"outside\",\"11,0\":\"outside\",\"11,1\":\"outside\",\"11,2\":\"outside\",\"11,3\":\"outside\",\"11,4\":\"outside\",\"11,5\":\"outside\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "94c477c8eeee412f099c265e00f62d40f78f57fe8de0bc6c6a232a175361eeeb"}569{"id": "first_move-000568", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,19), goal at (5,5).\n############################\n#.........#...#....A......##\n#.###.###.#.#.#.############\n#.#.#...#.#.#...#.....#...##\n#.#.###.#.#.#####.###.#.#.##\n#...#G#.#.#...#...#.#...#.##\n#####.#.#.###.#.###.#####.##\n#.....#.#...#.#.#.......#.##\n#.#####.###.#.#.#######.#.##\n#.#.....#...#.#.#.......#.##\n#.###.#####.#.#.#.###.###.##\n#...#.....#...#.#.#...#...##\n###.#####.#####.#.#####.#.##\n#.#.#...#.....#...#.....#.##\n#.#.#.#.#####.###.#.#####.##\n#.#...#.....#.....#.#.....##\n#.#########.#.#####.#.######\n#...#.......#...#...#.#...##\n#.###.###########.###.###.##\n#.....#.....#.....#.#.....##\n#.#####.###.#.#####.#####.##\n#.....#...#...#.....#.....##\n#####.#.#.#######.###.######\n#.....#.#...#.....#...#...##\n#.#########.#.#####.###.#.##\n#...........#...........#.##\n############################\n############################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "67841dc78cc0c7549963337245e4323a086dd3d102df09189740f8534c188150"}570{"id": "first_move-000569", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,27), goal at (33,11).\n####################################\n#.................................##\n#.#######################.##########\n#.#.....................#.........##\n#.#.###################.#########.##\n#.#...............#...#.#..A....#.##\n#.#.#############.#.#.#.#.#####.#.##\n#.#.#.............#.#.#.#.....#.#.##\n#.#.#.#############.###.#######.#.##\n#.#.#.#.........................#.##\n#.#.#.###########################.##\n#.#.#.#.........................#.##\n#.#.#.#.#######################.#.##\n#.#.#.#.#.....................#.#.##\n#.#.#.#.#.###.###############.#.#.##\n#.#.#.#.#...#.#.............#.#.#.##\n#.#.#.#.#####.#.#####.#######.#.#.##\n#.#.#.#.......#.....#.........#.#.##\n#.#.#.#.#######################.#.##\n#.#.#.#.#.......................#.##\n#.#.#.#.#.#####################.#.##\n#.#.#.#.#.#.....#.........#...#.#.##\n#.#.#.#.#.#.###.#.###.###.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#...#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#######.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.........#...#.#.#.#.#.#.#.#.#.#.##\n###############.#.#.#.#.#.###.#.#.##\n#...............#.#.#.#.......#.#.##\n#.#.#############.#.#.#########.#.##\n#.#.................#...........#.##\n#.###############################.##\n#.........#G......................##\n####################################\n####################################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "fe44a63b1137e0cabdad338b7d8071686b1aed822338c1fa09da49a3c31b76b6"}571{"id": "move_legal-000570", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (3,0); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXX###\nXX###\nXXA##\nXX...\nXX...", "question": "Is the cell one step south of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "fe2c9292ceb7e46cee89a6e2a42c2856306d1a6c574a5da9aedb5976954fc1a3"}572{"id": "distance_band-000571", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (4,21), goal at (1,21).\n########################\n#.....#.....#.....#..G##\n###.###.#.#.#.#.#.#.#.##\n#...#...#.#...#.#.#.#.##\n#.###.###.#####.#.#.#A##\n#.....#.........#...#.##\n#.#.#.#.#...#.###.###.##\n#.....#.#.#...#.....#.##\n#.....#.....#.###.#.#.##\n#.#.#...#.#...........##\n#...###.#.#####.#.######\n#.......#...#.#.#.....##\n#.###.#####...#...###.##\n#...#.#.....#...#...#.##\n#.#.#.#.#.###.#######.##\n#.....#...#.........#.##\n#.###.###.#.###.#####.##\n#...#.#.......#.......##\n#.#.#.#.#####.####.#.###\n#...#.#.........#.....##\n#.#.#.#.###.#.#.###.#.##\n#.#.........#.........##\n########################\n########################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "eacbff00689098e6a856b93fe8bd422188fc424f050a9e15071037beb4fbe93d"}573{"id": "snake_food-000572", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,4), travelling north, length 3.\n............\n............\n............\n............\n............\n............\n............\n............\n....H...*...\n....oo......\n............\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b2bfd533489fe185e09074fa936372443510be8943a13e3f138626ed1d7b91f8"}574{"id": "snake_food-000573", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,5), travelling west, length 3.\n..............\n..............\n..............\n..............\n..............\n.....*........\n.....Ho.......\n......o.......\n..............\n..............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a3c665a873b28bfbcb1f71392c28c909f5c3a090856985791ab477c358c60a38"}575{"id": "plan_survives-000574", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[25,21],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"22,18\":\"wall\",\"22,19\":\"open\",\"22,20\":\"wall\",\"22,21\":\"wall\",\"22,22\":\"wall\",\"22,23\":\"open\",\"22,24\":\"wall\",\"23,18\":\"wall\",\"23,19\":\"open\",\"23,20\":\"open\",\"23,21\":\"open\",\"23,22\":\"wall\",\"23,23\":\"open\",\"23,24\":\"open\",\"24,18\":\"wall\",\"24,19\":\"wall\",\"24,20\":\"wall\",\"24,21\":\"open\",\"24,22\":\"wall\",\"24,23\":\"wall\",\"24,24\":\"wall\",\"25,18\":\"open\",\"25,19\":\"open\",\"25,20\":\"wall\",\"25,21\":\"agent\",\"25,22\":\"open\",\"25,23\":\"open\",\"25,24\":\"wall\",\"26,18\":\"wall\",\"26,19\":\"open\",\"26,20\":\"wall\",\"26,21\":\"wall\",\"26,22\":\"wall\",\"26,23\":\"wall\",\"26,24\":\"wall\",\"27,18\":\"open\",\"27,19\":\"open\",\"27,20\":\"wall\",\"27,21\":\"open\",\"27,22\":\"open\",\"27,23\":\"open\",\"27,24\":\"open\",\"28,18\":\"wall\",\"28,19\":\"open\",\"28,20\":\"wall\",\"28,21\":\"open\",\"28,22\":\"wall\",\"28,23\":\"wall\",\"28,24\":\"wall\"}}", "question": "The agent will attempt this plan in order: south, west, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3e04c396b48d709fd534360a59afa5ab919fbab670a97dab8f98f79ade96b4c0"}576{"id": "cell_kind-000575", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,10); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXX\nXXXXXXX\n..#.#XX\n...A.XX\n..#..XX\n.###.XX\n.####XX", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "24ec2ca8650d3c8e7a975a2d21f2749c9a6b3833e2f2ff9221790ab525874436"}577{"id": "move_two_step-000576", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (8,9); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#.##\n.#.##\n##A##\n.#.##\n#####", "question": "The agent plans to move south, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "e8817c81448155e8e574a0c797482eb4bbc2149711149cc4981630f0d1a1b3e4"}578{"id": "reachable_within-000577", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (14,29), goal at (9,9).\n################################\n#.#.......#...................##\n#.#.#.#####.###.#.#.#.###.#.#.##\n#.#.........#.....#...#.......##\n#.#.#.#.#####.###.#.###.####.###\n#...#...#...#.#...............##\n#.#.###.#.#.#.#.#####..###..#.##\n#.....#...#.#...........#...#.##\n#.#...#####.###########.#.###.##\n#.#.#...#G#.#...#.............##\n#.#.###.#.#.#.#.#..#####.##.####\n#...#...#...#.#.....#.....#...##\n#.###.#######.#####.#.#.#...#.##\n#...#.......#.....#.......#.#.##\n###...#.###.###.#.#...#.#####A##\n#...#...........#.#.....#.....##\n#.#.#.#.#.#.###...###.#####.####\n#...#...#...#...#...#.....#...##\n#.#.#####.###.#.###.##.##.###.##\n#.........#.............#...#.##\n###.###.#...###.###.###.###.#.##\n#...#...#.....................##\n#.###.###.#.###.#.###.###.#.####\n#.#...#.#.#.....#...#.........##\n#.#.#.#.#.#.#.#.#.#.###.#.#.####\n#.#...#.#.....#.........#.....##\n#.#.###.##..#.#.###.#.#.###.#.##\n#...#.........#...#.......#.#.##\n##.##.#.#######.#.###.###.#.#.##\n#.............................##\n################################\n################################", "question": "Can the agent reach the goal in at most 54 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "e3c014f7ff5377264929680ffadbc7d167e9deba6f95aa8b72173b182ef1f3a1"}579{"id": "open_count-000578", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 0, column 26. Rows increase to the south and columns to the east.\nRow -4: column 22 is outside the maze, column 23 is outside the maze, column 24 is outside the maze, column 25 is outside the maze, column 26 is outside the maze, column 27 is outside the maze, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze\nRow -3: column 22 is outside the maze, column 23 is outside the maze, column 24 is outside the maze, column 25 is outside the maze, column 26 is outside the maze, column 27 is outside the maze, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze\nRow -2: column 22 is outside the maze, column 23 is outside the maze, column 24 is outside the maze, column 25 is outside the maze, column 26 is outside the maze, column 27 is outside the maze, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze\nRow -1: column 22 is outside the maze, column 23 is outside the maze, column 24 is outside the maze, column 25 is outside the maze, column 26 is outside the maze, column 27 is outside the maze, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze\nRow 0: column 22 is open, column 23 is open, column 24 is open, column 25 is open, column 26 is the agent, column 27 is open, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze\nRow 1: column 22 is open, column 23 is open, column 24 is wall, column 25 is open, column 26 is open, column 27 is open, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze\nRow 2: column 22 is open, column 23 is open, column 24 is open, column 25 is wall, column 26 is open, column 27 is open, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze\nRow 3: column 22 is open, column 23 is open, column 24 is open, column 25 is open, column 26 is open, column 27 is open, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze\nRow 4: column 22 is open, column 23 is open, column 24 is open, column 25 is open, column 26 is open, column 27 is open, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 3, "answer": "3", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "fc1d041762a50c66766c22b1cfa2246344487a4dd0044fef53b886564c42b761"}580{"id": "stochastic-000579", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[4,2],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"2,0\":\"wall\",\"2,1\":\"wall\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"3,0\":\"open\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"wall\",\"3,4\":\"wall\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"agent\",\"4,3\":\"wall\",\"4,4\":\"open\",\"5,0\":\"open\",\"5,1\":\"wall\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"open\",\"6,0\":\"open\",\"6,1\":\"open\",\"6,2\":\"open\",\"6,3\":\"open\",\"6,4\":\"open\"}}", "question": "The actuator executes the intended move with probability 0.75; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.16666666666666663, 0.8333333333333334], "deterministic": false, "fingerprint": "463d848c1b4d5c248826bea469fe151fe18c310a741f90b8b824462dc0584588"}581{"id": "stochastic-000580", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[2,0],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"open\",\"0,1\":\"open\",\"0,2\":\"open\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"open\",\"1,1\":\"open\",\"1,2\":\"open\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"agent\",\"2,1\":\"open\",\"2,2\":\"open\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"open\",\"3,1\":\"open\",\"3,2\":\"wall\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"open\",\"4,1\":\"wall\",\"4,2\":\"wall\"}}", "question": "The actuator executes the intended move with probability 0.75; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.08333333333333337, 0.9166666666666666], "deterministic": false, "fingerprint": "b3289a130b5da5448570193d25111806866ed866a83da0735ab84843baeaf598"}582{"id": "stochastic-000581", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[17,16],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"15,14\":\"open\",\"15,15\":\"open\",\"15,16\":\"open\",\"15,17\":\"open\",\"15,18\":\"open\",\"16,14\":\"wall\",\"16,15\":\"wall\",\"16,16\":\"wall\",\"16,17\":\"wall\",\"16,18\":\"wall\",\"17,14\":\"open\",\"17,15\":\"open\",\"17,16\":\"agent\",\"17,17\":\"open\",\"17,18\":\"wall\",\"18,14\":\"wall\",\"18,15\":\"wall\",\"18,16\":\"wall\",\"18,17\":\"wall\",\"18,18\":\"wall\",\"19,14\":\"open\",\"19,15\":\"open\",\"19,16\":\"open\",\"19,17\":\"open\",\"19,18\":\"open\"}}", "question": "The actuator executes the intended move with probability 0.70; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.20000000000000007, 0.7999999999999999], "deterministic": false, "fingerprint": "6bec4938be0199609c54543bc49373bc2a40d8e834e2c30d1edbd3662146433b"}583{"id": "plan_survives-000582", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (25,16); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n##....#\n..##..#\n..#.#..\n##.A##.\n..#.#.#\n....#..\nXXXXXXX", "question": "The agent will attempt this plan in order: south, east, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "288ed55d68ef13fd6eb38b2702c92ffa0a8020046b71aad4479c96dec76bb848"}584{"id": "only_way-000583", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (7,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n...#.#XXX\n####.#XXX\n.....#XXX\n.#####XXX\n....A#XXX\n######XXX\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "28b6dd9819999c544a3d523ac0c27a25768f0f721108615653b792da537bba9b"}585{"id": "move_legal-000584", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[18,33],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"15,30\":\"open\",\"15,31\":\"open\",\"15,32\":\"open\",\"15,33\":\"open\",\"15,34\":\"wall\",\"15,35\":\"open\",\"15,36\":\"outside\",\"16,30\":\"open\",\"16,31\":\"wall\",\"16,32\":\"wall\",\"16,33\":\"wall\",\"16,34\":\"open\",\"16,35\":\"open\",\"16,36\":\"outside\",\"17,30\":\"open\",\"17,31\":\"wall\",\"17,32\":\"open\",\"17,33\":\"open\",\"17,34\":\"open\",\"17,35\":\"open\",\"17,36\":\"outside\",\"18,30\":\"open\",\"18,31\":\"wall\",\"18,32\":\"open\",\"18,33\":\"agent\",\"18,34\":\"open\",\"18,35\":\"open\",\"18,36\":\"outside\",\"19,30\":\"open\",\"19,31\":\"open\",\"19,32\":\"wall\",\"19,33\":\"wall\",\"19,34\":\"wall\",\"19,35\":\"open\",\"19,36\":\"outside\",\"20,30\":\"open\",\"20,31\":\"open\",\"20,32\":\"open\",\"20,33\":\"open\",\"20,34\":\"wall\",\"20,35\":\"open\",\"20,36\":\"outside\",\"21,30\":\"wall\",\"21,31\":\"open\",\"21,32\":\"open\",\"21,33\":\"wall\",\"21,34\":\"wall\",\"21,35\":\"open\",\"21,36\":\"outside\"}}", "question": "If the agent attempts one cell east (col+1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "7acf6dce259fd30adb1e81b59f770129c5355144c9da2cc8f57ededef421e708"}586{"id": "reachable_within-000585", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (6,3), goal at (5,9).\n############\n#.#.......##\n#.#.#####.##\n#.#.#...#.##\n#.#.#.#.#.##\n#.#.#.#.#G##\n#.#A#.######\n#.#.#.....##\n#.#.#####.##\n#.........##\n############\n############", "question": "Can the agent reach the goal in at most 19 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "94f806c8d4a32c41fb579bc8d781cd427bb59a7ecdc0a6b2257158842efa10e0"}587{"id": "goal_bearing-000586", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (18,23), goal at (15,21).\n####################################\n#.......................#.........##\n#.#####################.#.#######.##\n#.#.............#.....#.#.#.......##\n#.#.#####.#####.#.###.#.#.##########\n#.#.#...#.#...#.#.#.#.#.#.........##\n#.#.#.#.#.#.#.#.#.#.#.#.#.#######.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.#.....#.##\n#.#.###.#.#.#.#.#.#.#.#.#.#.###.#.##\n#.#.....#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#######.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.........#.#.#.#.#.#.#.#.#.#...#.##\n#############.#.#.#.#.#.#.#.#####.##\n#.............#.#.#...#.#.#.....#.##\n#.###########.#.#.#.###.#.#.###.#.##\n#.#.........#.#.#.#.#G#.#.#.#.#.#.##\n#.#.#####.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#...#.#.#.#.#.#.#.#.#.#...#.#.##\n#.#.#.#.#.#.#.#.#.#.#.#A#.###.#.#.##\n#.#.#.#.#.#.#.#.#.#...#.#.#.#.#.#.##\n#.#.###.#.###.#.#.#######.#.#.#.#.##\n#.#.....#.....#.#.......#.#...#.#.##\n#.#############.#.#####.#.#####.#.##\n#...............#.....#.........#.##\n#.###############################.##\n#.#...............................##\n#.#.###.#########################.##\n#.#.#.#.#.......................#.##\n#.#.#.#.#.#####.###.###########.#.##\n#.#...#.#.#...#.#.#.#.......#.#.#.##\n#.###.#.#.#.#.#.#.#.#.#####.#.#.#.##\n#.#...#.#.#.#.#.#...#.#...#.#...#.##\n#.#####.#.###.#.#####.###.#.#####.##\n#.......#.....#...........#.......##\n####################################\n####################################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "cc4bbf6f8f1daf4584f4785f2fe8f79cf27748509d5f1049495d497c4756e2be"}588{"id": "distance_band-000587", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (15,3), goal at (1,14).\n#..##..#......##\n..........#...G.\n...#..#....#..#.\n............#...\n....#.#.....#..#\n......##...##..#\n#.....#.###.##..\n......#.##..#...\n...........#....\n................\n................\n..###..#.###....\n#...#..#.#.#...#\n.....#..........\n##.#.....#..##..\n###A..#.#.......", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 3, "answer": "20 to 39 moves", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "467e8adbe66d61767ff899410a9136763f0124a7bf3dcfe905da367b3f724adc"}589{"id": "onward-000588", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (5,11); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#######\n......#\n#####.#\n...A#..\n#######\n..#...#\n#.#.#.#", "question": "Suppose the agent moves one cell west. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "9611d35c619d854560b3e98b7c1eabdc6be6aa40f49d2dc8e9b9467bc79930f2"}590{"id": "plan_survives-000589", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[0,0],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-4,-4\":\"outside\",\"-4,-3\":\"outside\",\"-4,-2\":\"outside\",\"-4,-1\":\"outside\",\"-4,0\":\"outside\",\"-4,1\":\"outside\",\"-4,2\":\"outside\",\"-4,3\":\"outside\",\"-4,4\":\"outside\",\"-3,-4\":\"outside\",\"-3,-3\":\"outside\",\"-3,-2\":\"outside\",\"-3,-1\":\"outside\",\"-3,0\":\"outside\",\"-3,1\":\"outside\",\"-3,2\":\"outside\",\"-3,3\":\"outside\",\"-3,4\":\"outside\",\"-2,-4\":\"outside\",\"-2,-3\":\"outside\",\"-2,-2\":\"outside\",\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-1,-4\":\"outside\",\"-1,-3\":\"outside\",\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"0,-4\":\"outside\",\"0,-3\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"agent\",\"0,1\":\"open\",\"0,2\":\"open\",\"0,3\":\"open\",\"0,4\":\"open\",\"1,-4\":\"outside\",\"1,-3\":\"outside\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"open\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"2,-4\":\"outside\",\"2,-3\":\"outside\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"open\",\"2,4\":\"open\",\"3,-4\":\"outside\",\"3,-3\":\"outside\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"open\",\"4,-4\":\"outside\",\"4,-3\":\"outside\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"open\",\"4,3\":\"open\",\"4,4\":\"open\"}}", "question": "The agent will attempt this plan in order: north, south, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "823702086a037a1036f365b090cedfd1759de70fc7b165a0730fed8f9da82467"}591{"id": "stochastic-000590", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 15, column 19. Rows increase to the south and columns to the east.\nRow 11: column 15 is open, column 16 is open, column 17 is wall, column 18 is open, column 19 is open, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 12: column 15 is wall, column 16 is wall, column 17 is open, column 18 is open, column 19 is open, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 13: column 15 is wall, column 16 is open, column 17 is open, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 14: column 15 is open, column 16 is wall, column 17 is open, column 18 is open, column 19 is open, column 20 is open, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 15: column 15 is open, column 16 is open, column 17 is wall, column 18 is open, column 19 is the agent, column 20 is open, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 16: column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 17: column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 18: column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 19: column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze", "question": "The actuator executes the intended move with probability 0.70; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.09999999999999998, 0.9], "deterministic": false, "fingerprint": "7bc2ec6dec03a7a57f0a3f75a3a5bc5279cd2532f1bc0c65b90a899e2ac6c9bf"}592{"id": "distance_band-000591", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (21,6), goal at (21,1).\n########################\n#.#.......#.....#.....##\n#.#.#.#.###.#.#.#.###.##\n#.#.#.#.....#.........##\n#.#.#.#.###.#######.#.##\n#.#.#.#...#.#.......#.##\n#.....#.#.#.#.###...#.##\n#.#.#...#.#.#.#.....#.##\n#.#.#####.#.#.#.#####.##\n#.#...#...#.#.#.....#.##\n#.###.#.#####.#####.#.##\n#...#.......#.....#...##\n#.###.#####.###.#.###.##\n#.........#...#.......##\n##.##.##.####.#####.#.##\n#...#.#.........#...#.##\n#.#.###.#.#####.#.###.##\n#.#.#.........#.#.#...##\n#.#.#.#####.#.#.#.#.#.##\n#.#...#.....#...#.....##\n#.#.#####.#.#########.##\n#G....A...#...........##\n########################\n########################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 1, "answer": "5 to 9 moves", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a895b7b2e83b8d41ad4af0333bf85778117abcfbd2a3414490df34d80f81e310"}593{"id": "reachable_within-000592", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,3), goal at (0,20).\n################...#G.##\n################...#..##\n.#######.######.......#.\n..#..##.....####..##....\n#...#...........#..#...#\n...A..##...#...#..##...#\n#...#...#...#...#..##...\n##...##.#....#.###......\n###...#...##.##..##..##.\n#####.#...##.###..#.....\n..#....#...#.#........#.\n..#..#.......###...#####\n#.....#...#..#....#...#.\n.#.##.........#..#..#...\n..........#.#..#........\n#...#.#.##...#.##...#.#.\n###.###..#........####..\n....#..#...#.#.######..#\n...#..######......####.#\n#.#..###..##...#..######\n.........#####...#######\n...#..#....#...#...#####\n#........#....#.##.#..##\n##..##.................#", "question": "Can the agent reach the goal in at most 45 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "252f420325b8d1a087355f811552aa01ab305c0b6556f32dda730bdf279c4324"}594{"id": "goal_bearing-000593", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (23,1), goal at (21,21).\n############################\n#.................#.......##\n#.###############.#.#####.##\n#.#.#...........#.#.....#.##\n#.#.#.#########.#.#######.##\n#.#...........#...........##\n#.##########################\n#.........................##\n#.#######################.##\n#.#.....................#.##\n#.#.###################.#.##\n#.#.#.................#.#.##\n#.#.#.#.###############.#.##\n#.#.#.#.#...............#.##\n#.#.#.#.#.#############.#.##\n#.#.#.#.#.#...........#.#.##\n#.#.#.#.#.#.#########.#.#.##\n#.#.#.#.#.#.#.......#.#.#.##\n#.#.###.#.#.#.#.#####.#.#.##\n#.#.....#.#.#.#.......#.#.##\n#.#######.#.#.#########.#.##\n#.........#.#........G#.#.##\n###########.###########.#.##\n#A......#...............#.##\n#######.#################.##\n#.........................##\n############################\n############################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "ed30696762cb9c9593fad65cec5f98bc80110c56af8ffc13b5a6eb5e59089e35"}595{"id": "goal_bearing-000594", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,3), goal at (13,25).\n####################################\n#.................................##\n#.#####.#########################.##\n#.#...#.#.......................#.##\n#.#.#.#.#.###################.#.#.##\n#.#.#.#.#.#.........#.......#.#.#.##\n#.#.#.#.#.#.#.#####.#.#####.#.#.#.##\n#.#.#.#.#.#.#.#...#.#.#...#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.###.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.#...#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#A#.#.#.#.#.#.#.#.#.#.#G#.#.#.#.##\n#.###.#.#.#.#.#.#.#.#.#.###.#.#.#.##\n#.....#.#.#.#.#.#.#.#.#...#.#.#.#.##\n#####.#.#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.###.#.#.#.#.###.#.#.#.#.##\n#.#...#.#...#.#.#.#.#.....#.#.#.#.##\n#.#####.###.#.#.###.#######.#.###.##\n#.......#.#.#.#.............#.....##\n#########.#.#.######################\n#.....#...#.#.....................##\n#.###.#.#.#.#.###################.##\n#.#.#.#.#.#.#.#.............#...#.##\n#.#.#.###.#.#.#.###########.#.#.#.##\n#.#.#.....#.#.#.#.........#.#.#.#.##\n#.#.#####.#.#.#.#########.#.#.#.#.##\n#.#.....#.#.#.#...........#...#.#.##\n#.#.#####.#.#.#################.#.##\n#.#.......#.#...................#.##\n#.#########.#####################.##\n#.................................##\n####################################\n####################################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "7a795e570032b5d1f673ad4028e08be4469643048b5a23af5773e0c690366e3d"}596{"id": "snake_safe-000595", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (10,7), travelling south, length 3.\n............\n........*...\n............\n............\n............\n............\n............\n............\n............\n.......oo...\n.......H....\n............", "question": "If the snake moves west on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "51f700d3b68f26d7869b66ca124a7d4dcd1c412fadb44c7797e702da8ef265ff"}597{"id": "goal_bearing-000596", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (8,1), goal at (5,1).\n############\n#.....#...##\n###.#...####\n#...#.....##\n#.###.#.#.##\n#G#.#.#.#.##\n###.#.#.#.##\n#.#...#...##\n#A#.###.#.##\n#.......#.##\n############\n############", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "125b1a8bb7ffc13f4a48a89606ff257983b016336480395ccc7c8edea7e3e4b7"}598{"id": "stochastic-000597", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (9,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX#.#.....\nX#.######\nX#.......\nX########\nX#..A#...\nX#.###.#.\nX#.#...#.\nX#.######\nX#.......", "question": "The actuator executes the intended move with probability 0.90; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9666666666666667, 0.033333333333333326], "deterministic": false, "fingerprint": "bfdb202d802a26d260428192c788ef480caed4a9448150e3a70180322e5b679c"}599{"id": "first_move-000598", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,11), goal at (9,11).\n################\n#...........#.##\n#.##..#..##.#.##\n#.#...........##\n#.#######.###.##\n#..........A..##\n#######..####.##\n#.....#.......##\n#.###.#.#####.##\n#.......#..G#.##\n#.##.####.###.##\n#.#...#.......##\n#.#.#.#.###.#.##\n#.........#...##\n################\n################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "13d4fa26c00c8c412ebd252f7afa695a277444f5899570830e4e5b05135e069f"}600{"id": "stochastic-000599", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 8, column 11. Rows increase to the south and columns to the east.\nRow 4: column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open\nRow 5: column 7 is open, column 8 is open, column 9 is open, column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is open\nRow 6: column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open\nRow 7: column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open\nRow 8: column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is the agent, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open\nRow 9: column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open\nRow 10: column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open\nRow 11: column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open\nRow 12: column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.7333333333333334, 0.26666666666666666], "deterministic": false, "fingerprint": "4d6083cd49b35ed73176d1e5a635e6ee0c687935dc09e4060511e3c86e7eac58"}601{"id": "move_legal-000600", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[27,27],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"25,25\":\"open\",\"25,26\":\"open\",\"25,27\":\"open\",\"25,28\":\"outside\",\"25,29\":\"outside\",\"26,25\":\"open\",\"26,26\":\"open\",\"26,27\":\"open\",\"26,28\":\"outside\",\"26,29\":\"outside\",\"27,25\":\"open\",\"27,26\":\"open\",\"27,27\":\"agent\",\"27,28\":\"outside\",\"27,29\":\"outside\",\"28,25\":\"outside\",\"28,26\":\"outside\",\"28,27\":\"outside\",\"28,28\":\"outside\",\"28,29\":\"outside\",\"29,25\":\"outside\",\"29,26\":\"outside\",\"29,27\":\"outside\",\"29,28\":\"outside\",\"29,29\":\"outside\"}}", "question": "If the agent attempts one cell north (row-1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "38d978e6cc5071e5392b0762368be7f1575677b64982b0297117fa1e198f4ffe"}602{"id": "move_legal-000601", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (12,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.....#\n#.###.#\n#.#.#.#\n#.#A#.#\n#.#.#.#\n#.#.#.#\n#.#.#.#", "question": "The agent wants to move south. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "216c524c05814550dd58ca586fa6ddfbb5c170f1a153d46e6f400f72c4e5d9f3"}603{"id": "cell_kind-000602", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 0, column 1. Rows increase to the south and columns to the east.\nRow -2: column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze\nRow -1: column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze\nRow 0: column -1 is outside the maze, column 0 is open, column 1 is the agent, column 2 is open, column 3 is wall\nRow 1: column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is wall\nRow 2: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is wall", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "733aadda5c75fd0490a462964c2a24898b8ef8939f7bf65e9a2607299c219a13"}604{"id": "move_legal-000603", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 14, column 19. Rows increase to the south and columns to the east.\nRow 10: column 15 is open, column 16 is open, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 11: column 15 is open, column 16 is open, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 12: column 15 is open, column 16 is wall, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 13: column 15 is open, column 16 is open, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 14: column 15 is open, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is the agent, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 15: column 15 is open, column 16 is open, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 16: column 15 is wall, column 16 is wall, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 17: column 15 is open, column 16 is wall, column 17 is open, column 18 is open, column 19 is open, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 18: column 15 is open, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze", "question": "Is the cell one step east of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "68eb94ee85cd0455142e73736b2fcdf58d5cfab25fb1ee6308bf10bd91e68f56"}605{"id": "move_two_step-000604", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 15, column 12. Rows increase to the south and columns to the east.\nRow 12: column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall\nRow 13: column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open\nRow 14: column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is open, column 15 is open\nRow 15: column 9 is wall, column 10 is wall, column 11 is open, column 12 is the agent, column 13 is open, column 14 is wall, column 15 is wall\nRow 16: column 9 is wall, column 10 is wall, column 11 is wall, column 12 is open, column 13 is open, column 14 is open, column 15 is wall\nRow 17: column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open\nRow 18: column 9 is wall, column 10 is wall, column 11 is wall, column 12 is open, column 13 is open, column 14 is wall, column 15 is wall", "question": "The agent plans to move south, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "0dcb3918dc653c651e8617de24d2bd8ebf14fe7c8cef5e1d93120e9923bd8276"}606{"id": "move_two_step-000605", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (4,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX########\nX#.......\nX#.###.#.\nX#.#.#.#.\nX#.#A#.#.\nX#.#.#.#.\nX#.#.#.#.\nX#...#.#.\nX########", "question": "The agent plans to move west, then east. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "0dbce464511cae88b059935aa682a9465f6759d5b28e48b798cd68a4577bec8d"}607{"id": "distance_band-000606", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (10,5), goal at (1,13).\n################\n#.....#......G##\n#.#.#.#.#.##.###\n#.#.#...#.....##\n#.#.#..##.#.####\n#.#.......#...##\n#.#######.#...##\n#.......#.#.#.##\n###.#.#...#.#.##\n#.#.#.#.....#.##\n#.#.#A#######.##\n#.#.......#...##\n#.#.###.#.###.##\n#.......#.....##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "bb854da36379e1c2982599e5743c062f06e6db6246919df8d6f8b4ffdb45c96a"}608{"id": "blocked_pair-000607", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (7,17); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.....\n####.\n.#A#.\n.#.#.\n.#.#.", "question": "Are BOTH the cell one step west and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "7cd4730cf0e692c4ff944284a55ce316d4ad231e299f176244f5713d860e905e"}609{"id": "plan_survives-000608", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[22,19],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"18,15\":\"wall\",\"18,16\":\"wall\",\"18,17\":\"open\",\"18,18\":\"wall\",\"18,19\":\"open\",\"18,20\":\"wall\",\"18,21\":\"open\",\"18,22\":\"wall\",\"18,23\":\"wall\",\"19,15\":\"open\",\"19,16\":\"wall\",\"19,17\":\"open\",\"19,18\":\"wall\",\"19,19\":\"open\",\"19,20\":\"open\",\"19,21\":\"open\",\"19,22\":\"open\",\"19,23\":\"open\",\"20,15\":\"wall\",\"20,16\":\"wall\",\"20,17\":\"open\",\"20,18\":\"wall\",\"20,19\":\"wall\",\"20,20\":\"wall\",\"20,21\":\"wall\",\"20,22\":\"wall\",\"20,23\":\"wall\",\"21,15\":\"open\",\"21,16\":\"open\",\"21,17\":\"open\",\"21,18\":\"wall\",\"21,19\":\"open\",\"21,20\":\"open\",\"21,21\":\"open\",\"21,22\":\"wall\",\"21,23\":\"open\",\"22,15\":\"wall\",\"22,16\":\"wall\",\"22,17\":\"wall\",\"22,18\":\"wall\",\"22,19\":\"agent\",\"22,20\":\"wall\",\"22,21\":\"wall\",\"22,22\":\"wall\",\"22,23\":\"wall\",\"23,15\":\"open\",\"23,16\":\"open\",\"23,17\":\"open\",\"23,18\":\"open\",\"23,19\":\"open\",\"23,20\":\"wall\",\"23,21\":\"open\",\"23,22\":\"open\",\"23,23\":\"open\",\"24,15\":\"wall\",\"24,16\":\"wall\",\"24,17\":\"wall\",\"24,18\":\"wall\",\"24,19\":\"wall\",\"24,20\":\"wall\",\"24,21\":\"open\",\"24,22\":\"wall\",\"24,23\":\"wall\",\"25,15\":\"open\",\"25,16\":\"open\",\"25,17\":\"open\",\"25,18\":\"open\",\"25,19\":\"open\",\"25,20\":\"open\",\"25,21\":\"open\",\"25,22\":\"wall\",\"25,23\":\"open\",\"26,15\":\"wall\",\"26,16\":\"wall\",\"26,17\":\"wall\",\"26,18\":\"wall\",\"26,19\":\"wall\",\"26,20\":\"wall\",\"26,21\":\"wall\",\"26,22\":\"wall\",\"26,23\":\"wall\"}}", "question": "The agent will attempt this plan in order: south, south, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "64028fb8d1ca2733e10873e3cd1955df161c7c859155b7ec137a3ee9dcf20937"}610{"id": "reachable_within-000609", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,1), goal at (2,15).\n#..#....#...#...\n...#.......##...\n..............#G\n...#.#.......#.#\n...#.#...#.#....\n...#..#........#\n##.............#\n.#...##........#\n...#.##......#..\n...#.#.......#..\n.....#.#......#.\n..###....#......\n#.......##.###..\n.A.#......###...\n.....#........##\n...#.#..#..##..#", "question": "Can the agent reach the goal in at most 24 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "83929e854536d71532cd647944fda6385d3f483c7b5db0555261b806b5060645"}611{"id": "open_count-000610", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (9,9); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#...##XX\n##.####XX\n.....##XX\n#######XX\n....A##XX\n#######XX\n#######XX\nXXXXXXXXX\nXXXXXXXXX", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 1, "answer": "1", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "beda3e81d1ade5c1c54a30df1be4c05a1f6e145c6a6b8b4948cf60ea3d849d9a"}612{"id": "onward-000611", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (27,26); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#....XXX\n#..#.#XXX\n#.....XXX\n...#..XXX\n.##.A.XXX\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX", "question": "Suppose the agent moves one cell north. From that new cell, would a further step east be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "4917b9f50e43e1fcc424864e7a3324602932569084e27e5e88a399008211ff70"}613{"id": "distance_band-000612", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,3), goal at (3,2).\n###....##..#....\n###..........#..\n#..A#.#...####..\n##G#.#.##.##..##\n..##...##...#...\n...#......#.....\n#..##.....#..#..\n#.###.....#..#..\n#.##.##...###.#.\n.....##.#...#...\n##.#..##..#.##..\n##.##..#..#.....\n...##....#...###\n.#..#.#.##.#..##\n#..##..#####..##\n..#...#######.##", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "832f953ce8a3cb681c3e1a05252b0791ccb5bba4bcb941dfe384cccbded6743a"}614{"id": "plan_survives-000613", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 5. Rows increase to the south and columns to the east.\nRow 3: column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze\nRow 4: column 1 is wall, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze\nRow 5: column 1 is open, column 2 is open, column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is outside the maze\nRow 6: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze\nRow 7: column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is the agent, column 6 is open, column 7 is open, column 8 is wall, column 9 is outside the maze\nRow 8: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is outside the maze\nRow 9: column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze\nRow 10: column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze\nRow 11: column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze", "question": "The agent will attempt this plan in order: north, north, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "764d003b0ab05bd78ff9cd75d3894f95239b341cc61275498f08c5df198c3963"}615{"id": "first_move-000614", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,15), goal at (7,7).\n#####################\n#...........#.#.....#\n#.#########.#.#.#####\n#.#.........#.#A....#\n#.#.#########.#.###.#\n#.#.#.......#.#.#.#.#\n#.#.#.#####.#.#.#.#.#\n#.#.#.#G..#.#.#.#.#.#\n#.#.#.###.#.#.#.#.#.#\n#.#.#.....#.#.#.#.#.#\n#.#.#######.#.#.#.#.#\n#.#.........#.#.#.#.#\n#.###########.#.#.#.#\n#.............#.#.#.#\n#.#############.#.#.#\n#.#...........#.#.#.#\n#.#.###########.#.#.#\n#.#...............#.#\n#.#################.#\n#...................#\n#####################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "95c6012909242582a595a17d2da2df76360f5a62db25bb334659d57dec800c05"}616{"id": "reachable_within-000615", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,9), goal at (9,3).\n############\n#.....#...##\n#.#.#.#.####\n#.#.#.#...##\n###.#.###.##\n#...#....A##\n#.###.###.##\n#.........##\n#.###.###.##\n#..G#.....##\n############\n############", "question": "Can the agent reach the goal in at most 11 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "4000a291d8e9591f7307c0f2b48445f3250e9905a851c134e465babcd6e7da46"}617{"id": "first_move-000616", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (6,7), goal at (0,8).\n..#...#.G\n#........\n#.####...\n#..#...#.\n.....####\n##..#..#.\n#......A.\n#......#.\n#......#.", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "52a2690764f7fa54dddfbea724f5de5a47a94e00dab45ed92b2b1c4100470f5c"}618{"id": "onward-000617", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (11,9); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.....#...\n######.##\n...#.....\n##.######\n.#..A#...\n.#####.#.\n.......#.\n####.###.\n.....#.#.", "question": "Suppose the agent moves one cell west. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "c59a3b81ea9ce6c21cdc22dc350383213b0f5d3fb1c631ffd444e88f9a49cd24"}619{"id": "cell_kind-000618", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 26, column 1. Rows increase to the south and columns to the east.\nRow 22: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 23: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is wall, column 4 is open, column 5 is open\nRow 24: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is open, column 5 is open\nRow 25: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 26: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is the agent, column 2 is open, column 3 is open, column 4 is open, column 5 is wall\nRow 27: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 28: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 29: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 30: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "4921ab51082aad28710d5ae054b772d59d35c76cc5fdda916724470b282a546d"}620{"id": "goal_bearing-000619", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,3), goal at (1,1).\n#########\n#G......#\n#######.#\n#.....#.#\n#.###.#.#\n#.#A..#.#\n#.#####.#\n#.......#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "6fc2c6722297c324a0ea02eb957a30416c848c898109800ea9240cbe88f31b8f"}621{"id": "open_count-000620", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 15, column 21. Rows increase to the south and columns to the east.\nRow 12: column 18 is wall, column 19 is wall, column 20 is wall, column 21 is wall, column 22 is wall, column 23 is wall, column 24 is wall\nRow 13: column 18 is open, column 19 is open, column 20 is open, column 21 is open, column 22 is open, column 23 is open, column 24 is wall\nRow 14: column 18 is wall, column 19 is wall, column 20 is wall, column 21 is wall, column 22 is wall, column 23 is open, column 24 is wall\nRow 15: column 18 is wall, column 19 is open, column 20 is open, column 21 is the agent, column 22 is wall, column 23 is open, column 24 is wall\nRow 16: column 18 is wall, column 19 is open, column 20 is wall, column 21 is wall, column 22 is wall, column 23 is open, column 24 is wall\nRow 17: column 18 is wall, column 19 is open, column 20 is open, column 21 is open, column 22 is open, column 23 is open, column 24 is wall\nRow 18: column 18 is wall, column 19 is wall, column 20 is wall, column 21 is wall, column 22 is wall, column 23 is wall, column 24 is wall", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 1, "answer": "1", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3b7ba7776d0e21fe0d64bdac1d9ab535bed02908402b269c938d5d09440c1257"}622{"id": "move_legal-000621", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (7,23); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.#.###\n..#.#.#\n###.#.#\n#.#A#.#\n#.#.#.#\n..#.#.#\n#####.#", "question": "If the agent attempts one cell north (row-1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "537057f9acce8525ac3e042dcfdd6bbedb00198ada3dd4871da18e9fe0bbee7e"}623{"id": "onward-000622", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 4. Rows increase to the south and columns to the east.\nRow -3: column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow -2: column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow -1: column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow 0: column 0 is wall, column 1 is wall, column 2 is open, column 3 is open, column 4 is open, column 5 is wall, column 6 is wall, column 7 is open, column 8 is open\nRow 1: column 0 is wall, column 1 is wall, column 2 is wall, column 3 is open, column 4 is the agent, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is open\nRow 2: column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is wall, column 6 is wall, column 7 is open, column 8 is open\nRow 3: column 0 is open, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall, column 6 is open, column 7 is open, column 8 is open\nRow 4: column 0 is open, column 1 is open, column 2 is open, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is open\nRow 5: column 0 is open, column 1 is open, column 2 is open, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is open, column 7 is open, column 8 is open", "question": "Suppose the agent moves one cell south. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "fd3102f755eebb197bfe36ed2ccadcd3a18ac88fdca4a3317b69441631d4426b"}624{"id": "first_move-000623", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (33,25), goal at (9,13).\n####################################\n#...#...........#.................##\n#.#.#.#########.#.#########.#####.##\n#.#.#.#.......#.#.#.......#.#...#.##\n#.#.#.#.#######.#.#.#####.#.#.#.#.##\n#.#.#.#.........#.#.#...#.#.#.#.#.##\n#.#.#.#.#########.#.###.#.#.#.###.##\n#.#.#.#.#.......#.#.....#.#.#.....##\n#.#.#.#.#.#####.#.#######.#.########\n#.#.#.#.#.#..G#.#.........#...#...##\n#.#.#.#.#.#.###.#.###########.#.#.##\n#.#.#.#.#.#.....#.#...........#.#.##\n#.#.#.#.#.#######.#.###########.#.##\n#.#.#.#.#.........#.............#.##\n#.#.#.#.#########################.##\n#.#.#.#...........................##\n#.#.#.##############################\n#.#.#.#...........................##\n#.#.#.#.#########################.##\n#.#.#.#.#.......................#.##\n#.#.#.#.#######################.#.##\n#.#.#...........................#.##\n#.#.###########################.#.##\n#.#.......................#...#.#.##\n#.#######################.#.###.#.##\n#...........#...........#.#.....#.##\n#######################.#.#.#####.##\n#.......................#.#.#...#.##\n#.#####################.#.#.#.#.#.##\n#.#...................#.#.#.#.#.#.##\n#.#.#########.#########.#.#.###.#.##\n#.#.........#...........#.#.....#.##\n#.#######################.#######.##\n#........................A........##\n####################################\n####################################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3c51026dfaac25cffd5371a8dbf218442dc9f36d13f9605dd400c5706d8ac460"}625{"id": "first_move-000624", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (15,14), goal at (0,0).\nG..#.#..#.#.....\n.#......#.......\n..........#.#..#\n#.##.#........##\n....#..........#\n#..##...........\n#....#......##..\n.........#..#..#\n#..###.......#.#\n..#...##......##\n......#........#\n.#....#.#....#.#\n#...............\n.....###....#...\n.#....#.#.....#.\n###...#.......A.", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "bb8bb0f4825622c7d67404f57f03f64d24a52e03286e8b7e7c21c56b1f712249"}626{"id": "blocked_pair-000625", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (13,8); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#...#.###\n...##...#\n.####.#.#\n##.####..\n....A#..#\n.....#.##\n.##....##\nXXXXXXXXX\nXXXXXXXXX", "question": "Are BOTH the cell one step north and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "5a820a635042f248c7cd70513d3b32724e1124cc904e953494e19e82343985b3"}627{"id": "distance_band-000626", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (10,10), goal at (2,23).\n###.##.####........#..##\n###..#...##.....#.....##\n###.#.....#.....#.###..G\n###.#.#..###..#....###..\n##....#.#.###.....##.#.#\n###...##...#............\n###...#.....##..#.#..###\n##.......##.#..#..#.####\n.......##.....##.##.####\n..#.#.##...#.#..###....#\n.#....#.#.A##...###..#..\n##....#....####..##....#\n#.#..#....#####..###....\n.......##..###.#..##....\n.#.........#.....#...##.\n###.#.#..###.....##...#.\n##....###...#..#....#.#.\n##.#....#...#....##..#..\n#.#..#.#.....#.........#\n#......#.#.......#....##\n##.#.#.#.......#....#.##\n.......##..#..#.##...#..\n...#.#........#..#......\n.#...###..###..#....#.##", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 3, "answer": "20 to 39 moves", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "5cc80c030c7193566a343192d39e5edad76bf1944edbac892d649755614e98e2"}628{"id": "cell_kind-000627", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (13,2); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXX#..####\nXX...##..\nXX.####..\nXX....#.#\nXX##A.#.#\nXX####...\nXX#######\nXXXXXXXXX\nXXXXXXXXX", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "9b5ada3fd410c55819890337c535a6d3ebc453fcbd4b8794827862be75e420ca"}629{"id": "snake_food-000628", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,5), travelling south, length 4.\n..........\n..........\n..........\n..........\n..........\n..........\n...*o.....\n....oo....\n.....H....\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "0218bd6e2eb4e893dbfdb83512c4c4580bf8d337ed3b4fce5d08cd84eecda054"}630{"id": "move_legal-000629", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[33,2],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"29,-2\":\"outside\",\"29,-1\":\"outside\",\"29,0\":\"wall\",\"29,1\":\"open\",\"29,2\":\"wall\",\"29,3\":\"open\",\"29,4\":\"open\",\"29,5\":\"open\",\"29,6\":\"open\",\"30,-2\":\"outside\",\"30,-1\":\"outside\",\"30,0\":\"wall\",\"30,1\":\"open\",\"30,2\":\"wall\",\"30,3\":\"wall\",\"30,4\":\"open\",\"30,5\":\"open\",\"30,6\":\"wall\",\"31,-2\":\"outside\",\"31,-1\":\"outside\",\"31,0\":\"wall\",\"31,1\":\"open\",\"31,2\":\"open\",\"31,3\":\"open\",\"31,4\":\"open\",\"31,5\":\"open\",\"31,6\":\"open\",\"32,-2\":\"outside\",\"32,-1\":\"outside\",\"32,0\":\"wall\",\"32,1\":\"open\",\"32,2\":\"wall\",\"32,3\":\"wall\",\"32,4\":\"open\",\"32,5\":\"wall\",\"32,6\":\"wall\",\"33,-2\":\"outside\",\"33,-1\":\"outside\",\"33,0\":\"wall\",\"33,1\":\"open\",\"33,2\":\"agent\",\"33,3\":\"open\",\"33,4\":\"open\",\"33,5\":\"open\",\"33,6\":\"open\",\"34,-2\":\"outside\",\"34,-1\":\"outside\",\"34,0\":\"wall\",\"34,1\":\"wall\",\"34,2\":\"wall\",\"34,3\":\"wall\",\"34,4\":\"wall\",\"34,5\":\"wall\",\"34,6\":\"wall\",\"35,-2\":\"outside\",\"35,-1\":\"outside\",\"35,0\":\"wall\",\"35,1\":\"wall\",\"35,2\":\"wall\",\"35,3\":\"wall\",\"35,4\":\"wall\",\"35,5\":\"wall\",\"35,6\":\"wall\",\"36,-2\":\"outside\",\"36,-1\":\"outside\",\"36,0\":\"outside\",\"36,1\":\"outside\",\"36,2\":\"outside\",\"36,3\":\"outside\",\"36,4\":\"outside\",\"36,5\":\"outside\",\"36,6\":\"outside\",\"37,-2\":\"outside\",\"37,-1\":\"outside\",\"37,0\":\"outside\",\"37,1\":\"outside\",\"37,2\":\"outside\",\"37,3\":\"outside\",\"37,4\":\"outside\",\"37,5\":\"outside\",\"37,6\":\"outside\"}}", "question": "The agent wants to move north. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "9cb5f050cf598051b589ce6af9b8441abec00a0b60c4bd017599610d015cee6e"}631{"id": "plan_survives-000630", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[2,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,-3\":\"outside\",\"-2,-2\":\"outside\",\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-2,5\":\"outside\",\"-1,-3\":\"outside\",\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"0,-3\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"1,-3\":\"outside\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"2,-3\":\"outside\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"agent\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"3,-3\":\"outside\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"4,-3\":\"outside\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"wall\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"open\",\"5,-3\":\"outside\",\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"open\",\"5,5\":\"open\",\"6,-3\":\"outside\",\"6,-2\":\"outside\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"wall\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"open\"}}", "question": "The agent will attempt this plan in order: west, south, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "8b82972ac384ab1978a701b5195134b6dc34b8e6748b5f9bf85515e399c05a89"}632{"id": "cell_kind-000631", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[4,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"1,6\":\"wall\",\"1,7\":\"open\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"open\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"agent\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"open\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"open\",\"6,6\":\"wall\",\"6,7\":\"open\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"wall\",\"7,3\":\"open\",\"7,4\":\"wall\",\"7,5\":\"open\",\"7,6\":\"wall\",\"7,7\":\"open\",\"8,-1\":\"outside\",\"8,0\":\"wall\",\"8,1\":\"open\",\"8,2\":\"wall\",\"8,3\":\"open\",\"8,4\":\"wall\",\"8,5\":\"open\",\"8,6\":\"wall\",\"8,7\":\"open\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "7fe05c97ae9d8e4fdb994c4e9e137ea57eddbc8e3b982ab433f5f9b4a8cd200b"}633{"id": "open_count-000632", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[17,23],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"15,21\":\"open\",\"15,22\":\"open\",\"15,23\":\"open\",\"15,24\":\"wall\",\"15,25\":\"open\",\"16,21\":\"wall\",\"16,22\":\"wall\",\"16,23\":\"wall\",\"16,24\":\"wall\",\"16,25\":\"open\",\"17,21\":\"open\",\"17,22\":\"wall\",\"17,23\":\"agent\",\"17,24\":\"wall\",\"17,25\":\"open\",\"18,21\":\"open\",\"18,22\":\"wall\",\"18,23\":\"open\",\"18,24\":\"wall\",\"18,25\":\"open\",\"19,21\":\"open\",\"19,22\":\"wall\",\"19,23\":\"open\",\"19,24\":\"open\",\"19,25\":\"open\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 1, "answer": "1", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "aada0057363acfd8a16c0e4bdcbe80caacdfd74b164bf7897b5177a5f28b2d21"}634{"id": "plan_survives-000633", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (33,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX#.#.....\nX#.###.#.\nX#.#...#.\nX#.#.#.#.\nX#..A....\nX########\nX########\nXXXXXXXXX\nXXXXXXXXX", "question": "The agent will attempt this plan in order: west, south, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "9f921b837437876aa0cf45de6fb66663bfe8b4a15eca2e461e2c18a3b96c3462"}635{"id": "goal_bearing-000634", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,5), goal at (3,5).\n############\n#.........##\n#.#.#####.##\n#.#.#G..#.##\n#.#.###.#.##\n#.#.#A#.#.##\n###.#.#.#.##\n#...#.#.#.##\n#.###.#.#.##\n#.....#...##\n############\n############", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "efd950407fa4342a1a72dcdc24bed5f21a9eb75e61d24d2121dcb3e141f0c111"}636{"id": "open_count-000635", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (33,32); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n####.##\n.....##\n##..###\n...A.##\n#######\n#######\nXXXXXXX", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 3, "answer": "3", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "703f75ab08f0052648f0910d774ac90c2f561fa87569c3aa52aefc6688f65a46"}637{"id": "distance_band-000636", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,10), goal at (0,9).\n#.#....##G...###\n..###..##.#.####\n#.#....##.A..###\n...#.....##..###\n.#......##....##\n#...#.#.....#.##\n...###....######\n..###....#..####\n...#.#.#......##\n#.........######\n#.#.#..#.#######\n#####...########\n###....#########\n#......#########\n..#.#..#########\n........########", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "461cf37755dce72ab71672d62e99949cd68215d3b98ba36f73ff2d3fb9ddd992"}638{"id": "snake_safe-000637", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,6), travelling east, length 3.\n............\n............\n............\n............\n............\n.......*....\n....ooH.....\n............\n............\n............\n............\n............", "question": "If the snake moves west on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "53950d5b186bc581f33b2eb056238f989e378c174e6069fd0add53c4a4a43c0a"}639{"id": "snake_safe-000638", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,6), travelling east, length 3.\n..........\n..........\n..........\n..........\n..........\n.....o*...\n.....oH...\n..........\n..........\n..........", "question": "If the snake moves east on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "4875ff04eb18398b930a7e264a9cdbae5f671eb9f4aa5cfbd363755596afd83a"}640{"id": "snake_safe-000639", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (2,7), travelling north, length 3.\n........\n........\n.......H\n.......o\n.......o\n........\n...*....\n........", "question": "If the snake moves east on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "18320fd2e894327c36302885498a78db66cd4ffa38c03dbbfab1c97fcc0e89ec"}641{"id": "onward-000640", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 3, column 11. Rows increase to the south and columns to the east.\nRow -1: column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze\nRow 0: column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall\nRow 1: column 7 is open, column 8 is wall, column 9 is open, column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is open, column 15 is open\nRow 2: column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is open\nRow 3: column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is the agent, column 12 is open, column 13 is open, column 14 is wall, column 15 is open\nRow 4: column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open\nRow 5: column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is open\nRow 6: column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is open, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall\nRow 7: column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is open, column 12 is open, column 13 is open, column 14 is open, column 15 is open", "question": "Suppose the agent moves one cell east. From that new cell, would a further step east be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "69c73a61b6ba915d1d2fdfb7e5b813b08cd0bf007e8d6589c3417ba5047384ad"}642{"id": "blocked_pair-000641", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 20, column 17. Rows increase to the south and columns to the east.\nRow 18: column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open\nRow 19: column 15 is open, column 16 is open, column 17 is open, column 18 is wall, column 19 is open\nRow 20: column 15 is open, column 16 is wall, column 17 is the agent, column 18 is open, column 19 is open\nRow 21: column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze\nRow 22: column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze", "question": "Are BOTH the cell one step north and the cell one step west blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "2904c83d513955a5e74b292d213f06cb6513863b070b7f6d7f7e279cc6c921c2"}643{"id": "onward-000642", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,5],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"0,8\":\"wall\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"wall\",\"1,5\":\"open\",\"1,6\":\"open\",\"1,7\":\"open\",\"1,8\":\"open\",\"2,2\":\"wall\",\"2,3\":\"open\",\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"open\",\"2,8\":\"wall\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"agent\",\"3,6\":\"wall\",\"3,7\":\"open\",\"3,8\":\"open\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"wall\",\"4,8\":\"wall\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"open\",\"5,5\":\"open\",\"5,6\":\"open\",\"5,7\":\"open\",\"5,8\":\"wall\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"6,7\":\"wall\",\"6,8\":\"wall\"}}", "question": "Suppose the agent moves one cell north. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "72f748a515af277db7e6425630d63e5af90d46362886735a9e31b0ce243c438d"}644{"id": "distance_band-000643", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 40x40. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,34), goal at (39,4).\n.....#..#.#..#.##..........#....#....###\n.....#.#....#...#.#.#.#........###A.####\n.......................##...#...#...####\n#......#.....#####....#..##....#.#...###\n........#....####.#......##.......##....\n......##....#..#.....#....#.......#....#\n.................#.#..........#.##......\n.##...........##....#......#.##...##.#..\n..#..##......#.#...#...#.#.......##...#.\n...#.....#..#....#.........#..#......#..\n...#..#.#.........#.#..........###.#....\n.........##....#.......#.....#..##.#....\n...........##...#...#..#.#....#..#....#.\n..#.#..#.####.....##..#........#....#..#\n.#.......###..#..##.#....##.............\n.....##....#.#...#....##...........#.##.\n.###.#....#..........#.####..##...###...\n###.....#..#.......#........##.....##...\n..........#..#........#.#......#..##..##\n................#..#..#......#..#.#....#\n.....#.#........#.....##................\n....#...##..#........#..##.......#.##...\n...###....#.#...#......#......#.#..#...#\n....#.....#..#..#...................#.##\n...#....##.#................#...#....###\n....#...........#.......#.#...#..#.#....\n#.....#....####...###..###.........###..\n#...#..........#..........#.#........#..\n....#....###...#........#.##..........#.\n..#.#.#...#.....#.....#.##.....###...#.#\n.#.....#....##......#...............##.#\n#..#...#..#....#.#.#.......#..#.#..#....\n...##.#.......#...###.#.....##......#..#\n....#....#..#..##.#..##.#..#.......##...\n.#..#.....#....##.#...........##...#..##\n..######........##...#.#.##...##.....#..\n#.....#..##.......#.###.#....#...#..#..#\n##...##.#........###.#.......##..#.#...#\n##.#.#......#..##.#.......#.....#...#...\n##..G#.....#......#.##....#......#......", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 4, "answer": "40 moves or more", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "2bfb639f1efae6cb234e591e3e9d8fbf49f3eb9c016167924e1ad70d95030f21"}645{"id": "reachable_within-000644", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,7), goal at (1,1).\n############\n#G..#.....##\n###.#.###.##\n#.#.#.#A..##\n#.#.#.######\n#.#.#.....##\n#.#.#.###.##\n#.#.#...#.##\n#.#.#####.##\n#.........##\n############\n############", "question": "Can the agent reach the goal in at most 33 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "f041f0c1637d3fc290020a41a4ea6bbd7df055c42d7d8673b58026f682f1ab23"}646{"id": "plan_survives-000645", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[8,4],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"6,2\":\"open\",\"6,3\":\"wall\",\"6,4\":\"open\",\"6,5\":\"open\",\"6,6\":\"open\",\"7,2\":\"wall\",\"7,3\":\"wall\",\"7,4\":\"wall\",\"7,5\":\"wall\",\"7,6\":\"wall\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"8,4\":\"agent\",\"8,5\":\"open\",\"8,6\":\"open\",\"9,2\":\"outside\",\"9,3\":\"outside\",\"9,4\":\"outside\",\"9,5\":\"outside\",\"9,6\":\"outside\",\"10,2\":\"outside\",\"10,3\":\"outside\",\"10,4\":\"outside\",\"10,5\":\"outside\",\"10,6\":\"outside\"}}", "question": "The agent will attempt this plan in order: north, east, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "0249e62ca701a5ceb5ca293a3ed3ef76689197a5789714debf20c3242b6be146"}647{"id": "move_two_step-000646", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (10,10); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.....#XXX\n......XXX\n......XXX\n..#.#.XXX\n....A.XXX\n..##.#XXX\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX", "question": "The agent plans to move north, then east. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "ccd7d560cc57e638d119f195e007e5007213d7bdd7c7e1bf2b499243365e2666"}648{"id": "only_way-000647", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 2. Rows increase to the south and columns to the east.\nRow -2: column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow -1: column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 0: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 1: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is the agent, column 3 is open, column 4 is open, column 5 is open\nRow 2: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall\nRow 3: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 4: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 5, "answer": "more than one of them", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "7fd770dee5c066047af6a46728e61a3fbc6d2bd33f00b9d5168d046b7248df5d"}649{"id": "distance_band-000648", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,5), goal at (9,1).\n################\n#.....#.....#.##\n#.#.#.#.#.#.#.##\n#...#.....#.#.##\n###.###.#.#.#.##\n#.............##\n#.#.#.#.#.###.##\n#...#A#...#...##\n#####.###.#.#.##\n#G....#...#.#.##\n#.#.###.#.###.##\n#...#...#.#...##\n###.#.#.#.#.#.##\n#.....#.....#.##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 1, "answer": "5 to 9 moves", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "23fee1b8dfa3d4d54932baa8ca52c6df97e9174c547d76d7fef795190ce2dafd"}650{"id": "onward-000649", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 1. Rows increase to the south and columns to the east.\nRow 5: column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open, column 3 is open\nRow 6: column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is wall\nRow 7: column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is open, column 3 is wall\nRow 8: column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is wall, column 3 is wall\nRow 9: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall", "question": "Suppose the agent moves one cell east. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "e2e326297f432282b50de448de5700e17ee117eb69b9960a33f558e8f3a8d2bb"}651{"id": "move_two_step-000650", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (5,4); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.###.#.\n.#.#.#.\n.#.#.#.\n.#.A.#.\n.#####.\n.......\n#######", "question": "The agent plans to move east, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "4a686ea95b569ef8d84a8fd7bb00fc8ded498661bbef78f29bb57d9aa14c3d5d"}652{"id": "goal_bearing-000651", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,6), goal at (7,3).\n#########\n#...#...#\n#.#.#.#.#\n#.#...#.#\n#...#.#.#\n#.#.....#\n#.#####.#\n#..G#.A.#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "92d5891b8920469aa41a7d1623d70c7ad0efd4821cc1c93373bcbb8629c7af11"}653{"id": "open_count-000652", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (0,2); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXX\nXXXXX\n..A.#\n##...\n.....", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 3, "answer": "3", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "1bbb39d5a76f43babd3c7c12445445e6d4bfc4d76eea9f8efe8b378633141ac7"}654{"id": "move_two_step-000653", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (35,35); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.....XXXX\n...#.XXXX\n...#.XXXX\n..##.XXXX\n.##.AXXXX\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX", "question": "The agent plans to move east, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "718a03d7895f5aecb4eaa02174bd37b699f255dadba62d37d9b17c6bf81f5751"}655{"id": "cell_kind-000654", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (21,9); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.#.#.#\n#.#.#.#\n#.#.#.#\n#.#A#.#\n#.###.#\n#.#....\n#.#####", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "c0ec3e57a18717212a4ad8813f5e4d5d072f2f4f1fa5a48f666826cdbedb3c91"}656{"id": "first_move-000655", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (6,7), goal at (5,5).\n#########\n#.......#\n#.#####.#\n#.#.....#\n#.#####.#\n#....G#.#\n#######A#\n#.......#\n#########", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "5b5bd53b401b22f644b3522e7949443e583200c568ed37ed29c6bfcbf1027cb3"}657{"id": "onward-000656", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,16); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX\n#########\n....A.#.#\n#.#.#.#.#\n....#...#\n#######.#\n........#", "question": "Suppose the agent moves one cell west. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "54f2a56a9fc697e663eeade29574a13639b896c2620fabf9fd662a8427566cd2"}658{"id": "first_move-000657", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (20,8), goal at (0,13).\n#############G.#.....\n#############........\n###############.....#\n#############...#....\n##############.#.#..#\n################...##\n################.....\n################.#...\n################..##.\n###############....#.\n##############...#.##\n#############..#.#.#.\n#############........\n############......##.\n#############.......#\n#########..#..#.##...\n#######....#..#..#...\n######....#.#.....#..\n######......#....###.\n######.#.##..#.#####.\n####....A.##...###...", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "fabbbc5aa19a7a4771e645a49f6698fab19f91cfc758534a877064b3399286f6"}659{"id": "only_way-000658", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,-2\":\"outside\",\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"agent\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"wall\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"open\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "621b1059713994685227facf1cdb318f34987da0928b832a16666feec0a263af"}660{"id": "first_move-000659", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,8), goal at (3,5).\n################\n#.#.....A.....##\n#.#.###.##.##.##\n#....G#.....#.##\n##.####.###.#.##\n#.....#.#...#.##\n#.#######.###.##\n#.............##\n#...#.#.###.#.##\n#.#...#...#.#.##\n#.###.#.#...#.##\n#.#.....#.#...##\n#.#######.###.##\n#.........#...##\n################\n################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "76cc9ca64594fd0f0d25fec9ecead5ab0b1530aeec6ea93fc1f7a29a62e86ee8"}661{"id": "first_move-000660", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (21,7), goal at (3,21).\n####################################\n#...............#.........#.......##\n#.#############.#.#####.#.#.#####.##\n#.#...........#.#.#.#G#.#.#.#...#.##\n#.#.#########.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.........#.#.#.#.#.#.#.#.#.#.##\n#.#.###########.#.#.#.#.#.#.#.###.##\n#.#.............#.#.#.#.#.#.#...#.##\n#.#.#############.#.#.#.#.#.#.#.#.##\n#.#.............#.#.#.#.#.#.#.#.#.##\n#.###############.#.#.#.#.#.#.#.#.##\n#.................#.#.#.#.#.#.#.#.##\n#.#################.#.#.#.#.#.#.#.##\n#.#.................#.#.#.#.#.#.#.##\n#.#.###########.###.#.#.#.#.###.#.##\n#.#.#.........#.#.#.#.#.#.#...#.#.##\n#.#.#.#######.#.#.#.#.#.#####.#.#.##\n#.#.#.#.....#.#.#.#.#.#.....#.#.#.##\n#.#.#.#.#####.#.#.#.#.#.###.#.#.#.##\n#.#.#.#.......#.#...#.#.#.#.#.#.#.##\n#.#.#.#########.#####.#.#.#.#.#.#.##\n#.#.#..A......#.#.....#...#.#.#.#.##\n#.#.#.#######.#.#.#########.#.#.#.##\n#.#.#.#.....#.#.#.#.........#.#.#.##\n#.#.#.#.###.#.#.#.#.#######.#.#.#.##\n#.#.#.#...#.#.#.#.#.#.....#.#.#.#.##\n#.#.#.#####.#.#.#.#.#.###.#.#.#.#.##\n#.#.#.......#.#.#.#.#.#...#.#.#.#.##\n#.#.#########.#.#.#.#.#####.#.#.#.##\n#.#...........#.#.#.#.......#...#.##\n#.###########.#.#.#.###########.#.##\n#.#...........#.#.#.............#.##\n#.#############.#.###############.##\n#...............#.................##\n####################################\n####################################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "14f3f2f23136c308a26b6706e82d6248f27123c41f10d43ddfe4dfe0fe5e14b8"}662{"id": "goal_bearing-000661", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (18,7), goal at (1,9).\n#####################\n#........G#.........#\n#.###.#####.#.#.##.##\n#.#.......#.#.#.....#\n#.#.###.#.#.#.#.###.#\n#.#.....#.......#...#\n#.#.#..##########.#.#\n#...........#.....#.#\n#######.###.#.#####.#\n#.#.................#\n#.#.###########.#####\n#.#.#.......#.#.....#\n#.#.#.#.#.#.#.##.##.#\n#...#.............#.#\n#.###.#########.###.#\n#.....#.....#...#...#\n##.##.#.#.#.#.###.#.#\n#...#.#.#.#.#.....#.#\n#.#.#.#A#.#.#.##..#.#\n#.....#.#.....#.....#\n#####################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "e3db6da2fc484868b87b78d30d24a24e050813fc40dee3133f1589cf05f61681"}663{"id": "blocked_pair-000662", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,16); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX\n#########\n#...A...#\n#######.#\n#.....#.#\n#.#.###.#\n#.#...#.#", "question": "Are BOTH the cell one step south and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "074eb1299fcd8bd17a046c3520dfcdf2ddb739a89e558f389d8ce5ef5168d27e"}664{"id": "move_legal-000663", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[2,11],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,8\":\"outside\",\"-1,9\":\"outside\",\"-1,10\":\"outside\",\"-1,11\":\"outside\",\"-1,12\":\"outside\",\"-1,13\":\"outside\",\"-1,14\":\"outside\",\"0,8\":\"wall\",\"0,9\":\"wall\",\"0,10\":\"wall\",\"0,11\":\"wall\",\"0,12\":\"wall\",\"0,13\":\"wall\",\"0,14\":\"wall\",\"1,8\":\"open\",\"1,9\":\"open\",\"1,10\":\"open\",\"1,11\":\"open\",\"1,12\":\"open\",\"1,13\":\"open\",\"1,14\":\"wall\",\"2,8\":\"wall\",\"2,9\":\"wall\",\"2,10\":\"wall\",\"2,11\":\"agent\",\"2,12\":\"wall\",\"2,13\":\"open\",\"2,14\":\"wall\",\"3,8\":\"open\",\"3,9\":\"open\",\"3,10\":\"wall\",\"3,11\":\"open\",\"3,12\":\"open\",\"3,13\":\"open\",\"3,14\":\"wall\",\"4,8\":\"wall\",\"4,9\":\"open\",\"4,10\":\"wall\",\"4,11\":\"wall\",\"4,12\":\"wall\",\"4,13\":\"open\",\"4,14\":\"wall\",\"5,8\":\"open\",\"5,9\":\"open\",\"5,10\":\"wall\",\"5,11\":\"open\",\"5,12\":\"open\",\"5,13\":\"open\",\"5,14\":\"wall\"}}", "question": "If the agent attempts one cell north (row-1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "8b878a73222ab1e70509930ef038bd762f5979e1953c43d92d2458755976ea59"}665{"id": "goal_bearing-000664", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,3), goal at (3,3).\n############\n#.#.......##\n#.#.#####.##\n#.#G#...#.##\n#.#####.#.##\n#.......#.##\n#.#####.#.##\n#.#A....#.##\n#.#######.##\n#.........##\n############\n############", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "902f4f5e08d9ea3ec93cceebf4537c71642ec1fefd35563818bf5344a50813df"}666{"id": "plan_survives-000665", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (4,9); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n....##X\n###.##X\n..#.##X\n#.#A##X\n#.#.##X\n#.####X\n#.#.##X", "question": "The agent will attempt this plan in order: west, east, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "0f60db0f16c80faaeb7fc5e5393812a175ec3c33fa5ad3c9c6cf29fa4ffefbf9"}667{"id": "onward-000666", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (9,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#...\n.#.##\n.#A..\n.####\n.#...", "question": "Suppose the agent moves one cell east. From that new cell, would a further step east be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "53920e92a80a0461cf2968546a53fca2177c1f7ea2322f1a884f8d499e08dc21"}668{"id": "plan_survives-000667", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,0); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXX.#..\nXXX...#\nXXX.#..\nXXXA...\nXXX#..#\nXXX##..\nXXX....", "question": "The agent will attempt this plan in order: east, south, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "acdf11bf0b1add40b5b5318f99d6c402fb6dfef9e2bfb013adb76d63458f7866"}669{"id": "move_two_step-000668", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 5, column 3. Rows increase to the south and columns to the east.\nRow 2: column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall\nRow 3: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open, column 6 is open\nRow 4: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall\nRow 5: column 0 is wall, column 1 is open, column 2 is wall, column 3 is the agent, column 4 is wall, column 5 is open, column 6 is wall\nRow 6: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall\nRow 7: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall\nRow 8: column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall", "question": "The agent plans to move south, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "476ea78a50c1308f5ac4ae760b29e2b8c58ddffdf8c5fc934af168ca55fb496b"}670{"id": "plan_survives-000669", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 6. Rows increase to the south and columns to the east.\nRow -2: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze\nRow -1: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze\nRow 0: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is outside the maze\nRow 1: column 3 is open, column 4 is open, column 5 is open, column 6 is the agent, column 7 is open, column 8 is wall, column 9 is outside the maze\nRow 2: column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is outside the maze\nRow 3: column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is outside the maze\nRow 4: column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze", "question": "The agent will attempt this plan in order: east, east, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "109e72d660208af62a9e404d0ff67ce359d1096390a148822732bdf8e82bb50d"}671{"id": "snake_food-000670", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,6), travelling south, length 3.\n..........\n..........\n..........\n.........*\n.....oo...\n......H...\n..........\n..........\n..........\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "de52236be0cde9f21c0f77d2f3f88c836e0dccc0709c6b51cc4508fd13a3449f"}672{"id": "open_count-000671", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[21,19],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"18,16\":\"wall\",\"18,17\":\"wall\",\"18,18\":\"wall\",\"18,19\":\"wall\",\"18,20\":\"wall\",\"18,21\":\"wall\",\"18,22\":\"wall\",\"19,16\":\"open\",\"19,17\":\"open\",\"19,18\":\"open\",\"19,19\":\"open\",\"19,20\":\"open\",\"19,21\":\"open\",\"19,22\":\"wall\",\"20,16\":\"wall\",\"20,17\":\"wall\",\"20,18\":\"wall\",\"20,19\":\"wall\",\"20,20\":\"wall\",\"20,21\":\"open\",\"20,22\":\"wall\",\"21,16\":\"open\",\"21,17\":\"open\",\"21,18\":\"open\",\"21,19\":\"agent\",\"21,20\":\"wall\",\"21,21\":\"open\",\"21,22\":\"wall\",\"22,16\":\"wall\",\"22,17\":\"wall\",\"22,18\":\"wall\",\"22,19\":\"wall\",\"22,20\":\"wall\",\"22,21\":\"open\",\"22,22\":\"wall\",\"23,16\":\"open\",\"23,17\":\"open\",\"23,18\":\"open\",\"23,19\":\"open\",\"23,20\":\"open\",\"23,21\":\"open\",\"23,22\":\"wall\",\"24,16\":\"wall\",\"24,17\":\"wall\",\"24,18\":\"wall\",\"24,19\":\"wall\",\"24,20\":\"wall\",\"24,21\":\"wall\",\"24,22\":\"wall\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 1, "answer": "1", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "12e7be7b886f08c84106eab68cf71fc4e371a68930a98fb5ea3f62bacca82e9a"}673{"id": "plan_survives-000672", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (11,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#...#...#\n###..#.##\n##..#....\n...#..#..\n.##.A..##\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX", "question": "The agent will attempt this plan in order: west, south, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "f175702c2298cbf9767639753edda34f07f822a0a5c552c126063b77a86c6a13"}674{"id": "only_way-000673", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 2. Rows increase to the south and columns to the east.\nRow 4: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open\nRow 5: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 6: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 7: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is the agent, column 3 is open, column 4 is open, column 5 is open\nRow 8: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 9: column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 10: column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 5, "answer": "more than one of them", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "30414afcd22085db97f9746a55f180bb3e3531349bf67c685ed68dd5f0661fa5"}675{"id": "plan_survives-000674", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 0. Rows increase to the south and columns to the east.\nRow -1: column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze\nRow 0: column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open\nRow 1: column -2 is outside the maze, column -1 is outside the maze, column 0 is the agent, column 1 is wall, column 2 is open\nRow 2: column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open\nRow 3: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall", "question": "The agent will attempt this plan in order: north, south, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "fc27502862029a8b89050e2594bf47ada0e13aaa7eb43e3072cfb87500e0afb6"}676{"id": "move_legal-000675", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[27,27],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"25,25\":\"open\",\"25,26\":\"open\",\"25,27\":\"open\",\"25,28\":\"outside\",\"25,29\":\"outside\",\"26,25\":\"open\",\"26,26\":\"open\",\"26,27\":\"open\",\"26,28\":\"outside\",\"26,29\":\"outside\",\"27,25\":\"open\",\"27,26\":\"open\",\"27,27\":\"agent\",\"27,28\":\"outside\",\"27,29\":\"outside\",\"28,25\":\"outside\",\"28,26\":\"outside\",\"28,27\":\"outside\",\"28,28\":\"outside\",\"28,29\":\"outside\",\"29,25\":\"outside\",\"29,26\":\"outside\",\"29,27\":\"outside\",\"29,28\":\"outside\",\"29,29\":\"outside\"}}", "question": "If the agent attempts one cell south (row+1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "f1d2b2c120d6db06d917ec4e1682fbb0ce94b2f7938e5977b6b6804447d77334"}677{"id": "snake_food-000676", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,6), travelling south, length 3.\n..........\n..........\n..........\n*.........\n..........\n..........\n......o...\n......o...\n......H...\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "e96044db2ce9cb2dc0ee080ac607c87e2e58297f8a5ffe4f4344411e2882bbc5"}678{"id": "snake_safe-000677", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (3,6), travelling west, length 3.\n..........\n..........\n.....*....\n......Ho..\n.......o..\n..........\n..........\n..........\n..........\n..........", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "88273b21c99d41267f96f8a9e15a89ef42db9bae51cbd1b871b4cf80ed6071a1"}679{"id": "snake_safe-000678", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,5), travelling east, length 3.\n..........\n..........\n..........\n..........\n..........\n...ooH....\n..........\n..........\n..........\n...*......", "question": "If the snake moves east on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "27a47be8133c90a560e14af87fd3665d51504abec7dac44e799b71777fb0b375"}680{"id": "cell_kind-000679", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[17,17],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"13,13\":\"open\",\"13,14\":\"open\",\"13,15\":\"open\",\"13,16\":\"open\",\"13,17\":\"open\",\"13,18\":\"open\",\"13,19\":\"open\",\"13,20\":\"wall\",\"13,21\":\"outside\",\"14,13\":\"wall\",\"14,14\":\"wall\",\"14,15\":\"wall\",\"14,16\":\"wall\",\"14,17\":\"wall\",\"14,18\":\"wall\",\"14,19\":\"wall\",\"14,20\":\"wall\",\"14,21\":\"outside\",\"15,13\":\"open\",\"15,14\":\"open\",\"15,15\":\"open\",\"15,16\":\"open\",\"15,17\":\"open\",\"15,18\":\"open\",\"15,19\":\"open\",\"15,20\":\"wall\",\"15,21\":\"outside\",\"16,13\":\"wall\",\"16,14\":\"wall\",\"16,15\":\"wall\",\"16,16\":\"wall\",\"16,17\":\"wall\",\"16,18\":\"wall\",\"16,19\":\"open\",\"16,20\":\"wall\",\"16,21\":\"outside\",\"17,13\":\"open\",\"17,14\":\"open\",\"17,15\":\"open\",\"17,16\":\"open\",\"17,17\":\"agent\",\"17,18\":\"wall\",\"17,19\":\"open\",\"17,20\":\"wall\",\"17,21\":\"outside\",\"18,13\":\"wall\",\"18,14\":\"wall\",\"18,15\":\"wall\",\"18,16\":\"wall\",\"18,17\":\"wall\",\"18,18\":\"wall\",\"18,19\":\"open\",\"18,20\":\"wall\",\"18,21\":\"outside\",\"19,13\":\"open\",\"19,14\":\"open\",\"19,15\":\"open\",\"19,16\":\"open\",\"19,17\":\"open\",\"19,18\":\"open\",\"19,19\":\"open\",\"19,20\":\"wall\",\"19,21\":\"outside\",\"20,13\":\"wall\",\"20,14\":\"wall\",\"20,15\":\"wall\",\"20,16\":\"wall\",\"20,17\":\"wall\",\"20,18\":\"wall\",\"20,19\":\"wall\",\"20,20\":\"wall\",\"20,21\":\"outside\",\"21,13\":\"outside\",\"21,14\":\"outside\",\"21,15\":\"outside\",\"21,16\":\"outside\",\"21,17\":\"outside\",\"21,18\":\"outside\",\"21,19\":\"outside\",\"21,20\":\"outside\",\"21,21\":\"outside\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "4c1fee5e2ec8d99fbb1c81a73ddd41cd90a2675fe87891ec5df5638dc631c6f4"}681{"id": "open_count-000680", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,6],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-2,5\":\"outside\",\"-2,6\":\"outside\",\"-2,7\":\"outside\",\"-2,8\":\"outside\",\"-2,9\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"-1,6\":\"outside\",\"-1,7\":\"outside\",\"-1,8\":\"outside\",\"-1,9\":\"outside\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"open\",\"0,7\":\"wall\",\"0,8\":\"open\",\"0,9\":\"open\",\"1,3\":\"open\",\"1,4\":\"wall\",\"1,5\":\"open\",\"1,6\":\"agent\",\"1,7\":\"open\",\"1,8\":\"open\",\"1,9\":\"wall\",\"2,3\":\"open\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"2,6\":\"open\",\"2,7\":\"wall\",\"2,8\":\"open\",\"2,9\":\"open\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"wall\",\"3,6\":\"open\",\"3,7\":\"open\",\"3,8\":\"open\",\"3,9\":\"open\",\"4,3\":\"open\",\"4,4\":\"open\",\"4,5\":\"wall\",\"4,6\":\"open\",\"4,7\":\"open\",\"4,8\":\"open\",\"4,9\":\"wall\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 4, "answer": "4", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "2af55a4002ef544677f2edbdaa44e98c1c36b87d8a3f649064161cbe9137a6ba"}682{"id": "onward-000681", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,5); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#######\n....#..\n###.#.#\n..#A..#\n#.#####\n.......\n##.####", "question": "Suppose the agent moves one cell north. From that new cell, would a further step north be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "6af8e793a7acb3d3bcf7ffbecab8f72790e60b1e8717ee4650e381b314010606"}683{"id": "only_way-000682", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 25, column 21. Rows increase to the south and columns to the east.\nRow 21: column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is open, column 22 is wall, column 23 is open, column 24 is open, column 25 is open\nRow 22: column 17 is open, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is open, column 22 is wall, column 23 is wall, column 24 is wall, column 25 is open\nRow 23: column 17 is open, column 18 is wall, column 19 is open, column 20 is open, column 21 is open, column 22 is wall, column 23 is open, column 24 is wall, column 25 is open\nRow 24: column 17 is wall, column 18 is wall, column 19 is open, column 20 is wall, column 21 is wall, column 22 is wall, column 23 is open, column 24 is wall, column 25 is open\nRow 25: column 17 is open, column 18 is open, column 19 is open, column 20 is wall, column 21 is the agent, column 22 is open, column 23 is open, column 24 is open, column 25 is open\nRow 26: column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is wall, column 22 is wall, column 23 is wall, column 24 is wall, column 25 is wall\nRow 27: column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is wall, column 22 is wall, column 23 is wall, column 24 is wall, column 25 is wall\nRow 28: column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze, column 20 is outside the maze, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze, column 24 is outside the maze, column 25 is outside the maze\nRow 29: column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze, column 20 is outside the maze, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze, column 24 is outside the maze, column 25 is outside the maze", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "44491b38f64da7e19b7ded1767d10f8433b18d6ef129b293a8b4e481b6c1d916"}684{"id": "stochastic-000683", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[0,13],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,11\":\"outside\",\"-2,12\":\"outside\",\"-2,13\":\"outside\",\"-2,14\":\"outside\",\"-2,15\":\"outside\",\"-1,11\":\"outside\",\"-1,12\":\"outside\",\"-1,13\":\"outside\",\"-1,14\":\"outside\",\"-1,15\":\"outside\",\"0,11\":\"wall\",\"0,12\":\"open\",\"0,13\":\"agent\",\"0,14\":\"open\",\"0,15\":\"open\",\"1,11\":\"open\",\"1,12\":\"open\",\"1,13\":\"open\",\"1,14\":\"open\",\"1,15\":\"open\",\"2,11\":\"open\",\"2,12\":\"open\",\"2,13\":\"wall\",\"2,14\":\"open\",\"2,15\":\"open\"}}", "question": "The actuator executes the intended move with probability 0.80; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.06666666666666665, 0.9333333333333333], "deterministic": false, "fingerprint": "76c9d7c1f3d8a55880573b3b4531ebb3e5f78150ff08b11d8421080e5dc8f290"}685{"id": "plan_survives-000684", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[11,13],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"8,10\":\"wall\",\"8,11\":\"wall\",\"8,12\":\"wall\",\"8,13\":\"open\",\"8,14\":\"wall\",\"8,15\":\"wall\",\"8,16\":\"outside\",\"9,10\":\"wall\",\"9,11\":\"open\",\"9,12\":\"open\",\"9,13\":\"open\",\"9,14\":\"wall\",\"9,15\":\"wall\",\"9,16\":\"outside\",\"10,10\":\"wall\",\"10,11\":\"open\",\"10,12\":\"wall\",\"10,13\":\"wall\",\"10,14\":\"wall\",\"10,15\":\"wall\",\"10,16\":\"outside\",\"11,10\":\"open\",\"11,11\":\"open\",\"11,12\":\"wall\",\"11,13\":\"agent\",\"11,14\":\"wall\",\"11,15\":\"wall\",\"11,16\":\"outside\",\"12,10\":\"wall\",\"12,11\":\"wall\",\"12,12\":\"wall\",\"12,13\":\"open\",\"12,14\":\"wall\",\"12,15\":\"wall\",\"12,16\":\"outside\",\"13,10\":\"open\",\"13,11\":\"open\",\"13,12\":\"open\",\"13,13\":\"open\",\"13,14\":\"wall\",\"13,15\":\"wall\",\"13,16\":\"outside\",\"14,10\":\"wall\",\"14,11\":\"wall\",\"14,12\":\"wall\",\"14,13\":\"wall\",\"14,14\":\"wall\",\"14,15\":\"wall\",\"14,16\":\"outside\"}}", "question": "The agent will attempt this plan in order: south, south, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "aa9fdec81d330c136a529e42bcf71a1f410d923ab557167ed2901d1fa8c2987f"}686{"id": "stochastic-000685", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 3, column 4. Rows increase to the south and columns to the east.\nRow 0: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 1: column 1 is open, column 2 is open, column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is open\nRow 2: column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open\nRow 3: column 1 is open, column 2 is wall, column 3 is open, column 4 is the agent, column 5 is open, column 6 is wall, column 7 is open\nRow 4: column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open\nRow 5: column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open\nRow 6: column 1 is open, column 2 is open, column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open", "question": "The actuator executes the intended move with probability 0.80; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.1333333333333333, 0.8666666666666667], "deterministic": false, "fingerprint": "e27fb6a1d61bf539a9fe44fe52d044e6fabae4c135303a3c75157e9182e27a40"}687{"id": "only_way-000686", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 9, column 11. Rows increase to the south and columns to the east.\nRow 7: column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open\nRow 8: column 9 is open, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open\nRow 9: column 9 is open, column 10 is open, column 11 is the agent, column 12 is wall, column 13 is open\nRow 10: column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open\nRow 11: column 9 is open, column 10 is open, column 11 is open, column 12 is wall, column 13 is open", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "68a89fc66b61e2f3a9763a4fabf1d70a7b2c0966af16f025b7bf8cbb9690621b"}688{"id": "snake_safe-000687", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (1,8), travelling north, length 4.\n..........\n........H.\n..*.....o.\n.......oo.\n..........\n..........\n..........\n..........\n..........\n..........", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "2d39bc5ee36c0fc6cef93eb1920d7b78322989515e593eba4e1d5ee87d917b93"}689{"id": "onward-000688", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 15, column 17. Rows increase to the south and columns to the east.\nRow 11: column 13 is open, column 14 is open, column 15 is open, column 16 is wall, column 17 is open, column 18 is open, column 19 is open, column 20 is wall, column 21 is outside the maze\nRow 12: column 13 is wall, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze\nRow 13: column 13 is open, column 14 is open, column 15 is open, column 16 is wall, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze\nRow 14: column 13 is wall, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze\nRow 15: column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is the agent, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze\nRow 16: column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze\nRow 17: column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open, column 18 is open, column 19 is open, column 20 is wall, column 21 is outside the maze\nRow 18: column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is open, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is outside the maze\nRow 19: column 13 is open, column 14 is open, column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is wall, column 21 is outside the maze", "question": "Suppose the agent moves one cell north. From that new cell, would a further step north be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "da5a7d7fd205fc3c0224b3ad1450cccc9b2a456ccea383a2ce19d2c2f3ba3e9e"}690{"id": "only_way-000689", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,25],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,21\":\"outside\",\"-1,22\":\"outside\",\"-1,23\":\"outside\",\"-1,24\":\"outside\",\"-1,25\":\"outside\",\"-1,26\":\"outside\",\"-1,27\":\"outside\",\"-1,28\":\"outside\",\"-1,29\":\"outside\",\"0,21\":\"wall\",\"0,22\":\"wall\",\"0,23\":\"wall\",\"0,24\":\"wall\",\"0,25\":\"wall\",\"0,26\":\"wall\",\"0,27\":\"wall\",\"0,28\":\"outside\",\"0,29\":\"outside\",\"1,21\":\"open\",\"1,22\":\"open\",\"1,23\":\"open\",\"1,24\":\"open\",\"1,25\":\"open\",\"1,26\":\"wall\",\"1,27\":\"wall\",\"1,28\":\"outside\",\"1,29\":\"outside\",\"2,21\":\"open\",\"2,22\":\"wall\",\"2,23\":\"open\",\"2,24\":\"wall\",\"2,25\":\"open\",\"2,26\":\"wall\",\"2,27\":\"wall\",\"2,28\":\"outside\",\"2,29\":\"outside\",\"3,21\":\"open\",\"3,22\":\"wall\",\"3,23\":\"open\",\"3,24\":\"wall\",\"3,25\":\"agent\",\"3,26\":\"wall\",\"3,27\":\"wall\",\"3,28\":\"outside\",\"3,29\":\"outside\",\"4,21\":\"wall\",\"4,22\":\"wall\",\"4,23\":\"open\",\"4,24\":\"wall\",\"4,25\":\"wall\",\"4,26\":\"wall\",\"4,27\":\"wall\",\"4,28\":\"outside\",\"4,29\":\"outside\",\"5,21\":\"open\",\"5,22\":\"wall\",\"5,23\":\"open\",\"5,24\":\"open\",\"5,25\":\"open\",\"5,26\":\"wall\",\"5,27\":\"wall\",\"5,28\":\"outside\",\"5,29\":\"outside\",\"6,21\":\"open\",\"6,22\":\"wall\",\"6,23\":\"wall\",\"6,24\":\"wall\",\"6,25\":\"open\",\"6,26\":\"wall\",\"6,27\":\"wall\",\"6,28\":\"outside\",\"6,29\":\"outside\",\"7,21\":\"open\",\"7,22\":\"open\",\"7,23\":\"open\",\"7,24\":\"open\",\"7,25\":\"open\",\"7,26\":\"wall\",\"7,27\":\"wall\",\"7,28\":\"outside\",\"7,29\":\"outside\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "9c9bcdc213b4a24c022845aa95c825caf59cb15991d21ded7d523424f0d47786"}691{"id": "move_legal-000690", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 11, column 14. Rows increase to the south and columns to the east.\nRow 7: column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is open, column 15 is open, column 16 is wall, column 17 is open, column 18 is wall\nRow 8: column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open, column 18 is wall\nRow 9: column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is open, column 15 is open, column 16 is wall, column 17 is open, column 18 is open\nRow 10: column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is open, column 18 is wall\nRow 11: column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is the agent, column 15 is open, column 16 is wall, column 17 is open, column 18 is open\nRow 12: column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall\nRow 13: column 10 is open, column 11 is open, column 12 is wall, column 13 is open, column 14 is open, column 15 is open, column 16 is wall, column 17 is open, column 18 is open\nRow 14: column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is wall, column 18 is wall\nRow 15: column 10 is open, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is open, column 17 is open, column 18 is wall", "question": "Is the cell one step east of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "a3b8885659b65a70af5b0d281b5289cfab13c85378412437f35ee893fdaeadfd"}692{"id": "snake_food-000691", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (1,5), travelling north, length 3.\n........\n.....H..\n.....o..\n.....o..\n........\n........\n.....*..\n........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 1, "answer": "south", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "0cdec98b00ccf8ddc167a13d363252d81ffd72ea69a3d3ee47fec3aff019eaa3"}693{"id": "only_way-000692", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 9, column 7. Rows increase to the south and columns to the east.\nRow 6: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall\nRow 7: column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall\nRow 8: column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open, column 10 is wall\nRow 9: column 4 is wall, column 5 is open, column 6 is open, column 7 is the agent, column 8 is wall, column 9 is open, column 10 is wall\nRow 10: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall\nRow 11: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall\nRow 12: column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b6947eb59ac0eeec00245b56c7062ef41864c92a7e9b738fb95d2cb240469e82"}694{"id": "first_move-000693", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,9), goal at (7,5).\n############\n#.....#...##\n#.###.#.#.##\n#.#.#.#.#A##\n#.#.#.#.#.##\n#.#.#.#.#.##\n#.#.#.#.#.##\n#...#G#.#.##\n###.#####.##\n#.........##\n############\n############", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "31c437c54ab8413a7b9979aabba0e814f1e62f287ae456f085b9474c463e0ded"}695{"id": "first_move-000694", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,23), goal at (27,1).\n........#....#.##........###\n......##.....#.###...#....##\n.........#.........#....#.##\n.....#.#...#.............###\n##....#...#....#...#........\n..#..#......#....#.#.##A.#..\n......##......#.......#.###.\n..###..#..#..###...#..#...##\n......#..##.#....#..........\n##..#.#..###.........###.#..\n..........#..##..##.#.#.....\n......#................##.#.\n.#.#.......#......#.#.......\n.#...####..#.......#........\n.#...#.....#...#...#..#.....\n#.....##..##.....#...###...#\n#.......#..##...#..####....#\n.........####....#..........\n.....#.....###.###...#.....#\n.......##.......#......#.##.\n...#...#.................##.\n.....#..#.#.......#....##...\n..#........#.#...#.....#....\n....#....#.................#\n#.###.##.#....#...##...#.#..\n..#...#........#.#.#....#..#\n.........##...#.....#.#...##\n#G###....#........#...#.....", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "465c2aed727e2f9fce3ca216db37cfb353956a685bfd35e558f1a60dee7d4424"}696{"id": "reachable_within-000695", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,3), goal at (15,1).\n########################\n#...#.......#...#.....##\n#.#A###.###.#.###.#.####\n#.#.....#...#.....#...##\n#.#######.###.#######.##\n#.#.......#...#.......##\n###.###########.#####.##\n#.#.........#...#...#.##\n#.#########.#.###.#.#.##\n#.........#...#...#.#.##\n#.#####.#######.#####.##\n#...#...#.....#.......##\n#.###.###.#.###.########\n#.#...#...#.#...#.....##\n#.#.#.#.###.#.#######.##\n#G#.#.#.#.#...#.......##\n###.#.#.#.#######.###.##\n#...#.#.....#...#...#.##\n#.#.#######.#.#.###.#.##\n#.#.#.....#.#.#.#...#.##\n#.###.###.#.#.#.#.###.##\n#.......#.....#...#...##\n########################\n########################", "question": "Can the agent reach the goal in at most 135 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "d9d98d61b11881c4849106dd060c7192ce806473b66eff2019d4d90547bcd234"}697{"id": "snake_food-000696", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,6), travelling west, length 3.\n............\n............\n............\n............\n............\n............\n.......o....\n.....*Ho....\n............\n............\n............\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "4437b8f0d67039d6b963c38019c4591fd6acb9ed39b103ed6d8974aa18107072"}698{"id": "blocked_pair-000697", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (12,15); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#.#.#.#.\n.#...#.#.\n.#.###.#.\n.#.#...#.\n.#.#A#.#.\n.#.#.#.#.\n.#.#.#.#.\n.#.#.#.#.\n.#.###.#.", "question": "Are BOTH the cell one step north and the cell one step south blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "2d7cdbfacb9d5eaa4b6345d8138099311d134a7e4c224675232ab074288bd6e2"}699{"id": "cell_kind-000698", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[11,13],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"8,10\":\"wall\",\"8,11\":\"wall\",\"8,12\":\"wall\",\"8,13\":\"wall\",\"8,14\":\"wall\",\"8,15\":\"wall\",\"8,16\":\"outside\",\"9,10\":\"open\",\"9,11\":\"open\",\"9,12\":\"open\",\"9,13\":\"open\",\"9,14\":\"wall\",\"9,15\":\"wall\",\"9,16\":\"outside\",\"10,10\":\"wall\",\"10,11\":\"open\",\"10,12\":\"wall\",\"10,13\":\"open\",\"10,14\":\"wall\",\"10,15\":\"wall\",\"10,16\":\"outside\",\"11,10\":\"wall\",\"11,11\":\"open\",\"11,12\":\"open\",\"11,13\":\"agent\",\"11,14\":\"wall\",\"11,15\":\"wall\",\"11,16\":\"outside\",\"12,10\":\"wall\",\"12,11\":\"open\",\"12,12\":\"wall\",\"12,13\":\"open\",\"12,14\":\"wall\",\"12,15\":\"wall\",\"12,16\":\"outside\",\"13,10\":\"open\",\"13,11\":\"open\",\"13,12\":\"open\",\"13,13\":\"open\",\"13,14\":\"wall\",\"13,15\":\"wall\",\"13,16\":\"outside\",\"14,10\":\"wall\",\"14,11\":\"wall\",\"14,12\":\"wall\",\"14,13\":\"wall\",\"14,14\":\"wall\",\"14,15\":\"wall\",\"14,16\":\"outside\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "4eb480be747f247657e1160453a7395f4268d36e907d1991957816d270447aa8"}700{"id": "open_count-000699", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,10); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.#..\n#..#.\n#.A..\n#....\n.###.", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 4, "answer": "4", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "8f8a3a4f00f697f123bc1073995b5ea587c185b1860297183044372719614fa4"}701{"id": "blocked_pair-000700", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,6); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXX\nXXXXXXX\n######X\n...A.#X\n.###.#X\n...#.#X\n.#.#.#X", "question": "Are BOTH the cell one step north and the cell one step south blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "47fa0a7a0fc9f06bc973946fc219bf65d0d55641831c750c6bd22e9355947d16"}702{"id": "cell_kind-000701", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 12, column 2. Rows increase to the south and columns to the east.\nRow 10: column 0 is open, column 1 is open, column 2 is wall, column 3 is open, column 4 is open\nRow 11: column 0 is wall, column 1 is open, column 2 is open, column 3 is wall, column 4 is open\nRow 12: column 0 is wall, column 1 is open, column 2 is the agent, column 3 is wall, column 4 is open\nRow 13: column 0 is wall, column 1 is wall, column 2 is open, column 3 is wall, column 4 is wall\nRow 14: column 0 is wall, column 1 is wall, column 2 is open, column 3 is wall, column 4 is wall", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "c8a19c90b84f565b8a3dc443b2fd06a6f5253af1e6d525637ee9eff3cdaa1e47"}703{"id": "onward-000702", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 25, column 12. Rows increase to the south and columns to the east.\nRow 23: column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is wall\nRow 24: column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall\nRow 25: column 10 is wall, column 11 is open, column 12 is the agent, column 13 is open, column 14 is open\nRow 26: column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall\nRow 27: column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall", "question": "Suppose the agent moves one cell west. From that new cell, would a further step north be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "ff75dbb5052ddcd45326fd795d7c8e409399da543f79e230b5f83185c4ff26fb"}704{"id": "only_way-000703", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,15); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXX\nXXXXXXX\n#######\n..#A...\n#.#####\n#.#...#\n#.#.#.#", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "ebaea9555df35e38a57c1aaa0f2153e58d3fef73d3cbd562313de2c9755cc526"}705{"id": "reachable_within-000704", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,6), goal at (11,9).\n################\n#...#.....#...##\n#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.##\n#.#.#.#.#.#.####\n#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.##\n#.#.###.#.#.#.##\n#.#.....#G#...##\n#.###########.##\n#.....A.......##\n################\n################", "question": "Can the agent reach the goal in at most 56 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "05c6706bcd36c857d5d5ec40f299941e3d2bac14043e021f72a9b2b592d895ea"}706{"id": "move_two_step-000705", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[10,2],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"open\",\"7,3\":\"wall\",\"7,4\":\"open\",\"7,5\":\"open\",\"8,-1\":\"outside\",\"8,0\":\"open\",\"8,1\":\"open\",\"8,2\":\"open\",\"8,3\":\"open\",\"8,4\":\"open\",\"8,5\":\"wall\",\"9,-1\":\"outside\",\"9,0\":\"open\",\"9,1\":\"open\",\"9,2\":\"open\",\"9,3\":\"open\",\"9,4\":\"open\",\"9,5\":\"open\",\"10,-1\":\"outside\",\"10,0\":\"open\",\"10,1\":\"open\",\"10,2\":\"agent\",\"10,3\":\"open\",\"10,4\":\"wall\",\"10,5\":\"open\",\"11,-1\":\"outside\",\"11,0\":\"open\",\"11,1\":\"wall\",\"11,2\":\"open\",\"11,3\":\"open\",\"11,4\":\"open\",\"11,5\":\"open\",\"12,-1\":\"outside\",\"12,0\":\"outside\",\"12,1\":\"outside\",\"12,2\":\"outside\",\"12,3\":\"outside\",\"12,4\":\"outside\",\"12,5\":\"outside\",\"13,-1\":\"outside\",\"13,0\":\"outside\",\"13,1\":\"outside\",\"13,2\":\"outside\",\"13,3\":\"outside\",\"13,4\":\"outside\",\"13,5\":\"outside\"}}", "question": "The agent plans to move east, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "a3e155cbb1833e7ad37967490f79c49573d432c9f3e59cb0fae3d0e52ea9d5fa"}707{"id": "plan_survives-000706", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,20],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-3,16\":\"outside\",\"-3,17\":\"outside\",\"-3,18\":\"outside\",\"-3,19\":\"outside\",\"-3,20\":\"outside\",\"-3,21\":\"outside\",\"-3,22\":\"outside\",\"-3,23\":\"outside\",\"-3,24\":\"outside\",\"-2,16\":\"outside\",\"-2,17\":\"outside\",\"-2,18\":\"outside\",\"-2,19\":\"outside\",\"-2,20\":\"outside\",\"-2,21\":\"outside\",\"-2,22\":\"outside\",\"-2,23\":\"outside\",\"-2,24\":\"outside\",\"-1,16\":\"outside\",\"-1,17\":\"outside\",\"-1,18\":\"outside\",\"-1,19\":\"outside\",\"-1,20\":\"outside\",\"-1,21\":\"outside\",\"-1,22\":\"outside\",\"-1,23\":\"outside\",\"-1,24\":\"outside\",\"0,16\":\"wall\",\"0,17\":\"wall\",\"0,18\":\"wall\",\"0,19\":\"wall\",\"0,20\":\"wall\",\"0,21\":\"wall\",\"0,22\":\"wall\",\"0,23\":\"wall\",\"0,24\":\"wall\",\"1,16\":\"open\",\"1,17\":\"open\",\"1,18\":\"open\",\"1,19\":\"open\",\"1,20\":\"agent\",\"1,21\":\"open\",\"1,22\":\"open\",\"1,23\":\"open\",\"1,24\":\"open\",\"2,16\":\"wall\",\"2,17\":\"open\",\"2,18\":\"wall\",\"2,19\":\"open\",\"2,20\":\"wall\",\"2,21\":\"wall\",\"2,22\":\"wall\",\"2,23\":\"wall\",\"2,24\":\"wall\",\"3,16\":\"open\",\"3,17\":\"open\",\"3,18\":\"wall\",\"3,19\":\"open\",\"3,20\":\"open\",\"3,21\":\"open\",\"3,22\":\"wall\",\"3,23\":\"open\",\"3,24\":\"open\",\"4,16\":\"wall\",\"4,17\":\"wall\",\"4,18\":\"wall\",\"4,19\":\"wall\",\"4,20\":\"wall\",\"4,21\":\"open\",\"4,22\":\"open\",\"4,23\":\"open\",\"4,24\":\"wall\",\"5,16\":\"open\",\"5,17\":\"open\",\"5,18\":\"open\",\"5,19\":\"open\",\"5,20\":\"wall\",\"5,21\":\"open\",\"5,22\":\"open\",\"5,23\":\"open\",\"5,24\":\"wall\"}}", "question": "The agent will attempt this plan in order: west, north, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "34dc791aa3bec701b564835a4b4d1d47e390e97633ff7ddb6ab69c682bf862b3"}708{"id": "plan_survives-000707", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 4. Rows increase to the south and columns to the east.\nRow 4: column 1 is open, column 2 is wall, column 3 is wall, column 4 is open, column 5 is wall, column 6 is wall, column 7 is open\nRow 5: column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open\nRow 6: column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open\nRow 7: column 1 is open, column 2 is open, column 3 is open, column 4 is the agent, column 5 is open, column 6 is open, column 7 is open\nRow 8: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 9: column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze\nRow 10: column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze", "question": "The agent will attempt this plan in order: east, south, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "0a5b393ad8ffbbdc2d536fd407061d7ea6db289f1f93c11b0be922c1210211f1"}709{"id": "goal_bearing-000708", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (8,9), goal at (21,17).\n############################\n#...............#.........##\n#.#############.#.#######.##\n#.#...........#.#.#.....#.##\n#.#.#.#######.#.#.#.###.#.##\n#.#.#.#.....#.#.#.#.#.#.#.##\n#.#.#.#.###.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#A#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#...#.##\n#.#.#.#.#.#.#.#.#.#.#####.##\n#.#.#.#.#.#.#.#.#.#.......##\n#.#.#.#.#.#.#.#.#.#.########\n#.#.#.#.#.#.#.#.#.#.#.....##\n#.#.#.#.#.#.#.#.#.#.#.###.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.###.#.#.#.#.#.#.#.#.#.#.##\n#...#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#...#.#.#.#.#.#.#.##\n#.#.#.#.#####.#.#.#.#.#.#.##\n#.#.#.#.......#.#G#.#.#.#.##\n#.#.#.#########.###.#.#.#.##\n#.#.#.............#.#...#.##\n#.#.###############.#####.##\n#.#.......................##\n############################\n############################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 6, "answer": "south-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "b8942bd5e0b0be064617fbe1f17cf3e645ec9b60c966785570b867c60d62121d"}710{"id": "snake_food-000709", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (4,4), travelling west, length 3.\n..........\n..........\n..........\n..........\n...*Ho....\n.....o....\n..........\n..........\n..........\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "e86b0537cb7758e8da8030a1eb0f9c58014575876a6226f9f79ff997b3d3574b"}711{"id": "open_count-000710", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[5,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"2,0\":\"wall\",\"2,1\":\"wall\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"wall\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"wall\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"agent\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"wall\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"wall\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"open\",\"7,3\":\"open\",\"7,4\":\"open\",\"7,5\":\"open\",\"7,6\":\"open\",\"8,0\":\"wall\",\"8,1\":\"wall\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"wall\",\"8,6\":\"wall\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 1, "answer": "1", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b27ecc506bcb5ae0dd83de1856ff6e710e36c9406e50d17c8387b2cd9f12ffe9"}712{"id": "open_count-000711", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[5,4],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"wall\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"5,2\":\"wall\",\"5,3\":\"open\",\"5,4\":\"agent\",\"5,5\":\"open\",\"5,6\":\"wall\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"7,2\":\"wall\",\"7,3\":\"open\",\"7,4\":\"open\",\"7,5\":\"open\",\"7,6\":\"open\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 2, "answer": "2", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "7903f4e50caef06b703b2fab9b45731061d497aab56a882d13b3532374f9ac33"}713{"id": "open_count-000712", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 26, column 1. Rows increase to the south and columns to the east.\nRow 22: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 23: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 24: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 25: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 26: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is open, column 3 is open, column 4 is wall, column 5 is open\nRow 27: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is wall, column 4 is open, column 5 is open\nRow 28: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 29: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 30: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 3, "answer": "3", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "63466f1cc479bcfc1e308121ef82daec6c838677f70ef3b7aeb99d73a741e382"}714{"id": "snake_safe-000713", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,5), travelling east, length 3.\n........\n........\n........\n....*...\n....o...\n....oH..\n........\n........", "question": "If the snake moves north on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "915cdee204079e30116b34f4f9d411b7ace6343206727feb5e612004274b7614"}715{"id": "cell_kind-000714", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,24],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,21\":\"wall\",\"0,22\":\"wall\",\"0,23\":\"wall\",\"0,24\":\"wall\",\"0,25\":\"wall\",\"0,26\":\"wall\",\"0,27\":\"wall\",\"1,21\":\"open\",\"1,22\":\"open\",\"1,23\":\"open\",\"1,24\":\"open\",\"1,25\":\"open\",\"1,26\":\"open\",\"1,27\":\"open\",\"2,21\":\"open\",\"2,22\":\"wall\",\"2,23\":\"wall\",\"2,24\":\"wall\",\"2,25\":\"wall\",\"2,26\":\"wall\",\"2,27\":\"open\",\"3,21\":\"open\",\"3,22\":\"wall\",\"3,23\":\"open\",\"3,24\":\"agent\",\"3,25\":\"open\",\"3,26\":\"wall\",\"3,27\":\"open\",\"4,21\":\"open\",\"4,22\":\"wall\",\"4,23\":\"open\",\"4,24\":\"wall\",\"4,25\":\"wall\",\"4,26\":\"wall\",\"4,27\":\"wall\",\"5,21\":\"open\",\"5,22\":\"wall\",\"5,23\":\"open\",\"5,24\":\"wall\",\"5,25\":\"open\",\"5,26\":\"open\",\"5,27\":\"open\",\"6,21\":\"open\",\"6,22\":\"wall\",\"6,23\":\"open\",\"6,24\":\"wall\",\"6,25\":\"open\",\"6,26\":\"wall\",\"6,27\":\"wall\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "5f0358a3533413c9d7e81bd501717981e97b0064856c031426041308dc61b8fa"}716{"id": "move_legal-000715", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (7,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#.#.\n##.#.\n.#A#.\n.####\n.....", "question": "Is the cell one step south of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "f2bf8829756e30a37d86858a5f3a0b2a766a123dabe41d2bcabb072af6769186"}717{"id": "goal_bearing-000716", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,0), goal at (6,5).\n############\nA###########\n..##########\n#.##########\n#....#######\n....########\n#.##.G######\n..#..#######\n..#.########\n.#...#######\n...#..######\n.....#######", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 6, "answer": "south-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "97a66c17e24719e1140366e982ccd375d13f4db92537f8562fa56d0fb4274a35"}718{"id": "cell_kind-000717", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (32,33); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.####\n.#.##\n.#A##\n...##\n#####", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "fdc7ed1e711f783382fbb2efbc689942b7d6711240bab3b0ef3ffb24ae9477b6"}719{"id": "onward-000718", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 4, column 5. Rows increase to the south and columns to the east.\nRow 0: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is outside the maze\nRow 1: column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is outside the maze\nRow 2: column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze\nRow 3: column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze\nRow 4: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is the agent, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze\nRow 5: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze\nRow 6: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze\nRow 7: column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is outside the maze\nRow 8: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is outside the maze", "question": "Suppose the agent moves one cell north. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "8b7c662b45a3b002257298522451be2a218976c2a6ecdd350fd4ba658e8618a7"}720{"id": "stochastic-000719", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 7, column 1. Rows increase to the south and columns to the east.\nRow 4: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall\nRow 5: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is wall\nRow 6: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is open, column 4 is wall\nRow 7: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is open, column 3 is open, column 4 is wall\nRow 8: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall\nRow 9: column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze\nRow 10: column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.8666666666666667, 0.13333333333333333], "deterministic": false, "fingerprint": "379bf1f99d9351493b8c97db346e07331bde7fc83ea1a18264d868c4837da12e"}721{"id": "distance_band-000720", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,26), goal at (1,29).\n################################\n#.#.......#.....#...#...#.A..G##\n#.#.#.###.#.##.##.#.#.#.#.###.##\n#...#.......#...#.#...#...#...##\n#.###.###.###.#.#.###.#.###.#.##\n#.........#.......#.......#...##\n#.#.#.#.#...#######.###.#.#.#.##\n#.#...#.#.#.......#.#...#.....##\n#.#.#.#.###...#.#.#.#.#.#.#.####\n#.....#.........#.#.......#...##\n#.#####.###.#####.#####.#.###.##\n#.#.........#...#.....#.#.#...##\n#.#.#.###.#.#.#.#.#.#.###...#.##\n#...#...#.#...#.............#.##\n#.#####.###.#####.#.###.###.#.##\n#.......#.......#...#.......#.##\n#.###...#.#..##.#.#...#######.##\n#...#.#.......#...#.#.#.......##\n#.#.#.###.#.#.###.#.#.#.###.#.##\n#.#.#...#.....#...#...#.#.#...##\n#.#.###.#.###.#.###.###.#.#.#.##\n#.#.#...#...#...#.#...#...#.#.##\n#.#.#.#.##.####.#.#.#.#.#...####\n#...#...#.....#.........#.#...##\n#.#.#.###.#.#.##.##.###.#####.##\n#...#.......#.#.......#.......##\n#.#####.###.#.#..##.#.###.###.##\n#...#.#.#...#.....#.......#...##\n###.#...#.#######.#####.#.#.####\n#.....#.........#.............##\n################################\n################################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "52bd09ce7d2e5aee7929fdeb0735d37e208e66f2f3945e9656d842a43a72651a"}722{"id": "distance_band-000721", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 40x40. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (18,27), goal at (21,24).\n#######..#....#.#...##......####...##...\n####............#..##.##..#####.#..###.#\n###..........#....###.......##.....##...\n####.###.#.##.##.####....##..#.#.#......\n####..##.##......###.#..####.#....##...#\n####....#..##...####..#..###..#.#..#.##.\n####.......#............####.#####.##...\n##..#.....#.....#.......#.#####..#..#...\n##..#.....##............#..#.#..........\n##..#.#...#.##..##.........#...#.#.##...\n...#.###.............##..##.#...#.#..#.#\n.#...#######..#.....##......#.#.........\n......######.#.#..#..#.###..#.#...##.#..\n...#..###......#....#.......#...#....#..\n.#...######.#...#.....##.##...#.##.#....\n#.##..#####...#...#..#####.#..######....\n.....#####.#...#..#.#.####..#..####..#..\n.#.#.####.....#....##.#####.#.#####..#.#\n....##..##...#.#..#.#...##.A.#####...##.\n.........#..#....#..#...#..#..####...#..\n###.....###.##.#..#.#..##.#....####.....\n###.#...##.......#...#.#G.#....###...#..\n##.....###......#....#..##......#..###.#\n####.#.#.......##..#..#.#######..#....##\n#......#.##...###....#....##...#..#..#..\n##..#.........##.#..........##.........#\n..#....#.#....#....##.####.##...#....#..\n....##.##.##.....#.###...##..##.....#...\n..###..#..#####....###.#...#..##...#....\n#...#.#.........#..##...#.#..###........\n#.#..##........#............####.##.....\n#.....#.#..##.###.#....###..####...#.#..\n##.####.##.......#....#....#..#..#.#....\n.....#.....#.##..####......##....#.#....\n.##....#.#.#..#..#....#.#..#..##.#.....#\n......#......##....#.###....#......#.##.\n....#....#.#..#.#....###.##...#.##...#..\n#.##...#........#.##.#..#.##..#.##......\n####..#...###.##...#...#..#.......#...##\n##...#..##..........#.....#.....#...#.##", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 1, "answer": "5 to 9 moves", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "2c4a7064d6f2b4f087dce1f104906b6d66984847bb96189a53ad030763da8b05"}723{"id": "reachable_within-000722", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,7), goal at (1,2).\n################\n#.G...#.......##\n#.###.#.#.######\n#.#...#.#.....##\n#.##.##.#.###.##\n#.#...#.#...#.##\n#.#.#.#.#.#.#.##\n#......A..#.#.##\n#####.#.#.###.##\n#...#.#...#...##\n#.#.#...###.#.##\n#.#.....#.#.#.##\n#.#####.#...####\n#.........#...##\n################\n################", "question": "Can the agent reach the goal in at most 10 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "824fff1ba5e6227f2401e1effd95f063d591e94f0d1887b8a9c2fc7434ec7122"}724{"id": "snake_safe-000723", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (9,7), travelling south, length 3.\n..............\n........*.....\n..............\n..............\n..............\n..............\n..............\n.......o......\n.......o......\n.......H......\n..............\n..............\n..............\n..............", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "729091dfcdfafbe1f1837d1279d17a970af676595efeefb3bbdea8eacb077708"}725{"id": "goal_bearing-000724", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (8,1), goal at (5,17).\n####################################\n#...........#.........#...........##\n#.#.#######.#.#######.#.#########.##\n#.#.#.....#.#.#.....#.#.#.......#.##\n#.#.#.###.#.#.#.###.#.#.#.#####.#.##\n#.#.#.#.#.#.#.#.#G#.#.#.#.#...#.#.##\n###.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#A#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.#.###.#.##\n#.#.#.#.#.#.#.#.#...#.#.#.#.....#.##\n#.#.#.#.#.#.#.#.#####.#.#.#####.#.##\n#.#.#...#.#.#.#.......#.#.....#.#.##\n#.#.###.#.#.#.#########.#####.#.#.##\n#.#.#.#.#.#.#.................#.#.##\n#.#.#.#.#.#.#.###################.##\n#.#.#.#.#.#.#.#.....#.#...........##\n#.#.#.#.#.#.#.#.#.#.#.#.#########.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.......#.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.#####.#.##\n#.#.#.#.#.#.#.#.#.#...#.#.#...#.#.##\n#.#.#.#.#.#.#.###.###.#.#.#.#.#.#.##\n#.#.#...#.#.#.....#.#.#.#.#.#.#.#.##\n#.#.#####.#.#######.#.#.#.#.#.#.#.##\n#.#.....#.#.#.......#.#.#.#.#.#.#.##\n#.#.###.#.#.#.#####.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#...#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.###.#.#.#.###.#.#.##\n#.#.#.#.#.#.#.#.....#.#.#.....#.#.##\n#.#.#.#.#.#.#.#######.#.#######.#.##\n#.#...#.#.#.#.........#.#.......#.##\n#.#####.#.#.###########.#.#######.##\n#.......#...............#.........##\n####################################\n####################################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "19c6b2a176e2aecd825a6fa55e8c0919b0ee66841217462683d9e1a2bcd2a3a4"}726{"id": "snake_food-000725", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,5), travelling south, length 3.\n..........\n..........\n..........\n..........\n..........\n....oo....\n....*H....\n..........\n..........\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "872e80d7a4f05435845df660532cbf78401c11358586c7fe29aa09a015892cab"}727{"id": "snake_food-000726", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,6), travelling south, length 3.\n............\n............\n............\n............\n............\n............\n.....oo.....\n.....*H.....\n............\n............\n............\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "522a7388fb3c97754db88bfc196b0d88ddbdd9dee26b4454806ad90e7509782b"}728{"id": "plan_survives-000727", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[27,2],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"25,0\":\"wall\",\"25,1\":\"open\",\"25,2\":\"wall\",\"25,3\":\"open\",\"25,4\":\"open\",\"26,0\":\"wall\",\"26,1\":\"open\",\"26,2\":\"wall\",\"26,3\":\"wall\",\"26,4\":\"wall\",\"27,0\":\"wall\",\"27,1\":\"open\",\"27,2\":\"agent\",\"27,3\":\"open\",\"27,4\":\"wall\",\"28,0\":\"wall\",\"28,1\":\"wall\",\"28,2\":\"wall\",\"28,3\":\"wall\",\"28,4\":\"wall\",\"29,0\":\"wall\",\"29,1\":\"open\",\"29,2\":\"open\",\"29,3\":\"open\",\"29,4\":\"open\"}}", "question": "The agent will attempt this plan in order: north, south, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a0bc00aa763cafa410713fcc4c1538bd6701c10efe0604a97dff86166f807fff"}729{"id": "snake_food-000728", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (1,3), travelling north, length 3.\n........\n...H....\n...o....\n...o....\n........\n.......*\n........\n........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 6, "answer": "south-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "46491ef80ae5555ffb01212c90d263516230822bca937a85465071b8b0e6f40a"}730{"id": "move_two_step-000729", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,6); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\n#######XX\n....#.#XX\n#####.#XX\n#...A.#XX\n###.###XX\n......#XX\n#.###.#XX\n......#XX", "question": "The agent plans to move west, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "d99c5f1465a75a06a91a25bbe887a0d1930882d06938a5a5c855b52c6a1a28d9"}731{"id": "onward-000730", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 35, column 34. Rows increase to the south and columns to the east.\nRow 32: column 31 is wall, column 32 is open, column 33 is open, column 34 is wall, column 35 is open, column 36 is outside the maze, column 37 is outside the maze\nRow 33: column 31 is open, column 32 is open, column 33 is open, column 34 is open, column 35 is open, column 36 is outside the maze, column 37 is outside the maze\nRow 34: column 31 is open, column 32 is open, column 33 is open, column 34 is open, column 35 is wall, column 36 is outside the maze, column 37 is outside the maze\nRow 35: column 31 is open, column 32 is open, column 33 is open, column 34 is the agent, column 35 is open, column 36 is outside the maze, column 37 is outside the maze\nRow 36: column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze, column 36 is outside the maze, column 37 is outside the maze\nRow 37: column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze, column 36 is outside the maze, column 37 is outside the maze\nRow 38: column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze, column 36 is outside the maze, column 37 is outside the maze", "question": "Suppose the agent moves one cell west. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "7c533df2e5fe46888d227bcaacf867ec863a8c9c4a4f3434743e77c7f2a61173"}732{"id": "onward-000731", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[11,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"8,4\":\"wall\",\"8,5\":\"wall\",\"8,6\":\"wall\",\"8,7\":\"wall\",\"8,8\":\"wall\",\"8,9\":\"wall\",\"8,10\":\"wall\",\"9,4\":\"open\",\"9,5\":\"open\",\"9,6\":\"open\",\"9,7\":\"open\",\"9,8\":\"open\",\"9,9\":\"open\",\"9,10\":\"wall\",\"10,4\":\"wall\",\"10,5\":\"wall\",\"10,6\":\"wall\",\"10,7\":\"wall\",\"10,8\":\"wall\",\"10,9\":\"open\",\"10,10\":\"wall\",\"11,4\":\"wall\",\"11,5\":\"open\",\"11,6\":\"open\",\"11,7\":\"agent\",\"11,8\":\"wall\",\"11,9\":\"open\",\"11,10\":\"wall\",\"12,4\":\"wall\",\"12,5\":\"open\",\"12,6\":\"wall\",\"12,7\":\"wall\",\"12,8\":\"wall\",\"12,9\":\"open\",\"12,10\":\"wall\",\"13,4\":\"wall\",\"13,5\":\"open\",\"13,6\":\"open\",\"13,7\":\"open\",\"13,8\":\"open\",\"13,9\":\"open\",\"13,10\":\"wall\",\"14,4\":\"wall\",\"14,5\":\"wall\",\"14,6\":\"wall\",\"14,7\":\"wall\",\"14,8\":\"wall\",\"14,9\":\"wall\",\"14,10\":\"wall\"}}", "question": "Suppose the agent moves one cell west. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "1538f0b7874ac01715be57fb40f973c71a7b53e78bc4bd99355552df6c57de58"}733{"id": "move_legal-000732", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 5, column 13. Rows increase to the south and columns to the east.\nRow 1: column 9 is open, column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is open, column 16 is open, column 17 is open\nRow 2: column 9 is wall, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open\nRow 3: column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is open, column 15 is open, column 16 is wall, column 17 is open\nRow 4: column 9 is wall, column 10 is wall, column 11 is open, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall\nRow 5: column 9 is open, column 10 is open, column 11 is open, column 12 is wall, column 13 is the agent, column 14 is open, column 15 is open, column 16 is open, column 17 is open\nRow 6: column 9 is open, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall\nRow 7: column 9 is open, column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is open, column 16 is open, column 17 is open\nRow 8: column 9 is open, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is wall\nRow 9: column 9 is open, column 10 is open, column 11 is open, column 12 is wall, column 13 is open, column 14 is open, column 15 is open, column 16 is open, column 17 is open", "question": "If the agent attempts one cell west (col-1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "7e7857645262b382947da420a04a3d15607c9c1ce9233da992a98c59bc5012ae"}734{"id": "snake_safe-000733", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (0,3), travelling west, length 3.\n*..Hoo..\n........\n........\n........\n........\n........\n........\n........", "question": "If the snake moves north on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "634c23559b926b58af82727c9faa02927403412a8a5e4021a1da278341fb8e5b"}735{"id": "snake_safe-000734", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,4), travelling west, length 3.\n..*.........\n............\n............\n............\n............\n............\n.....o......\n....Ho......\n............\n............\n............\n............", "question": "If the snake moves east on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "3eb8de260d736b458facb61f7ddd36ab2343a9599d73d8de4093d8b55de7a77f"}736{"id": "reachable_within-000735", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,5), goal at (13,5).\n################\n#.............##\n#.######..###.##\n#.#.......#...##\n#.#.#.#####.####\n#.....#...#...##\n#.#.#.#.#.###.##\n#....A..#...#.##\n#...###.###.#.##\n#.#...#.....#.##\n#.#.#.###.#.#.##\n#.#.....#.#...##\n#.#.###.#.######\n#...#G..#.....##\n################\n################", "question": "Can the agent reach the goal in at most 11 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "2255f9fb176caaf715fd6b9c566dde3fe89ef322c6c3ddaa900e26f43464350a"}737{"id": "first_move-000736", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (21,3), goal at (14,25).\n################.....#......\n##############.#..........#.\n#############....#...##....#\n###############..#.......#..\n#############..............#\n######..#######.#....#...##.\n######.######.####....#.....\n#####..#.###..###...#..##..#\n#####....#............###.##\n########.......###.#...##...\n######....#.#...#####...#..#\n######....#....#####..###.##\n#####..#..#.#....####..#.###\n#####...#.##....##.####..###\n###....#..#...#.#..###..#G#.\n#......#..#.....##.###..#.#.\n...#....#.#.#....#.##...#...\n.#......####..##......##....\n#..##..####............##...\n...#...###...###..#.....#.#.\n.#..#####..#.#.#......#.##..\n#.#A..#######....##...#.....\n....#..###.#.#...#.#.....#..\n.####..#.......##..#..####..\n...###.#...##.....##...#....\n#.####..#.####..#...#..#.##.\n####....######....###.#.###.\n####.#.########..........#..", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3196d3a3ddfbc932426857e69228491da34c9202a24b8f2b36410759975241c5"}738{"id": "distance_band-000737", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 40x40. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,37), goal at (6,39).\n...#..#....#.......##..##.....##########\n..##.......#............#.#..###########\n.#...#..##....#.##.....##........#######\n..#..#...#.#..#.......#......#....##.###\n..###......#...##........#............##\n......##..#....##....####..#..##...#.#.#\n..........#.#..###........###..........G\n#...#....#......##.....#..#....##.#..A.#\n#.#....#....##.........#..#.....#.#.#...\n.....#..#.#.##....##.......#..#.......#.\n.....#..##....###...#...#.##.#.....#.###\n.##.#........#....#....#.......#.#......\n.##...#.##..####.###.#......#...###...#.\n##...#.......#.#.###........#.##.#.##..#\n##..........##.##.#........#.......#.#..\n..###...#.#.......#.....#.......#.#....#\n..#.....#.........##.......#.###....#..#\n#.....#....##....#.....#.....###....#...\n###.......#..#..#...#.##..#...#.......#.\n##..#...........#....#.....#..#.......#.\n###...###....#.........#..#........#.#..\n####..........###....#.#.#....#.#...###.\n######....#............###..###.#...#..#\n######..#.....#.#..#...#............#..#\n#######...##.##.....##.#..#......#....##\n#######....#......##...#..#..#.......###\n######....##.#..#......#.#....#......#..\n#####...#.#..#...##...###...#....#.#.#..\n#####......#.#.#......#.#.......##.##..#\n####.....###........#....#..###..#.##...\n....#..#..##..#.###..###.#...#......##.#\n.#.#...##.#.#.....#..#..#...#....#......\n...#.#.#..#........#.#....#.##.#.##.....\n......#..##..##..##.#....#....#..#......\n#.#.#...##.#..............#..###........\n...#.##.....#..#.............##....#..#.\n.............#.....#..#...........#....#\n......#.#.........#..#.##..##....#......\n.#........#.#.......#...##......#.......\n.....##.##.......##.........#...#....#..", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b8731712ea2b74bcb8278eb9c25e9c3b73dae73275765b9a4ce85286524e3acf"}739{"id": "move_legal-000738", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[1,6],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"-1,6\":\"outside\",\"-1,7\":\"outside\",\"-1,8\":\"outside\",\"0,4\":\"wall\",\"0,5\":\"open\",\"0,6\":\"open\",\"0,7\":\"wall\",\"0,8\":\"wall\",\"1,4\":\"open\",\"1,5\":\"open\",\"1,6\":\"agent\",\"1,7\":\"wall\",\"1,8\":\"open\",\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"open\",\"2,8\":\"wall\",\"3,4\":\"open\",\"3,5\":\"wall\",\"3,6\":\"wall\",\"3,7\":\"open\",\"3,8\":\"wall\"}}", "question": "If the agent attempts one cell north (row-1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "ad7b2d69ce457932802261d4df38b3e46ff5ad061eec906b60764e2b8d6d149a"}740{"id": "first_move-000739", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (14,2), goal at (5,0).\n..............####.##\n..#....#.#..#.###....\n#..#...#.#..........#\n..##....#..#........#\n..#.#..............#.\nG##....##.##..#......\n###.##..........#....\n.....#....#..#.#..#..\n#.........#.......#..\n#.....#.....##.#..##.\n..#.#....#.#.......#.\n.###.#....#..#.....##\n##...#.##.#..#.....##\n#...##......####.....\n##A#..#.#.#....#.....\n#..#.#......#......#.\n..#....#.........###.\n...##..............#.\n....#.#.........#....\n......##.###...#.#...\n....##........#......", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "9f4d03b0b1e5dc264aa05c92626341a88c0d80e6699d8b210801a8a036cd5fa7"}741{"id": "move_two_step-000740", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 30, column 20. Rows increase to the south and columns to the east.\nRow 27: column 17 is wall, column 18 is open, column 19 is open, column 20 is open, column 21 is open, column 22 is open, column 23 is wall\nRow 28: column 17 is open, column 18 is open, column 19 is open, column 20 is wall, column 21 is open, column 22 is wall, column 23 is open\nRow 29: column 17 is open, column 18 is open, column 19 is wall, column 20 is wall, column 21 is open, column 22 is open, column 23 is open\nRow 30: column 17 is wall, column 18 is wall, column 19 is open, column 20 is the agent, column 21 is wall, column 22 is wall, column 23 is wall\nRow 31: column 17 is wall, column 18 is wall, column 19 is open, column 20 is open, column 21 is wall, column 22 is wall, column 23 is wall\nRow 32: column 17 is open, column 18 is open, column 19 is open, column 20 is wall, column 21 is wall, column 22 is wall, column 23 is wall\nRow 33: column 17 is open, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is wall, column 22 is open, column 23 is wall", "question": "The agent plans to move south, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "67b47c27955c9fc42987297af7a1a2a45f95cf41bfef5b40159f7d4e8486cb7b"}742{"id": "reachable_within-000741", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (10,1), goal at (11,7).\n##....##...#\n###..#....##\n####....##..\n..#.###.#...\n...........#\n#.##.....###\n..#....#....\n..#.........\n..###..##...\n#..####.#..#\n.A####.....#\n#..####G#..#", "question": "Can the agent reach the goal in at most 22 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "82b96f8d0658f5c003ae83c66d59e6c12b2434518b4e90f73e0d82adab46a374"}743{"id": "goal_bearing-000742", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,5), goal at (5,7).\n#########\n#.......#\n#.###.#.#\n#.......#\n#.#####.#\n#....A#G#\n#.#####.#\n#.......#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "c04d626ded2704398f01961d60c40213465c3cdfa9996b9eaef7df7cc454740e"}744{"id": "blocked_pair-000743", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 4, column 13. Rows increase to the south and columns to the east.\nRow 2: column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall\nRow 3: column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is wall\nRow 4: column 11 is open, column 12 is wall, column 13 is the agent, column 14 is wall, column 15 is wall\nRow 5: column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall\nRow 6: column 11 is open, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall", "question": "Are BOTH the cell one step south and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "c7b9f92273bea46396c7126ee895005fa7a172e5b386a27ebc1bae8953f34e38"}745{"id": "snake_safe-000744", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (9,5), travelling west, length 3.\n..............\n..............\n*.............\n..............\n..............\n..............\n..............\n..............\n......o.......\n.....Ho.......\n..............\n..............\n..............\n..............", "question": "If the snake moves east on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "844fc03baa91592c14d7eadc92ae3021ca256d7bd73d03466d73c0686b7dc1a1"}746{"id": "only_way-000745", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,13],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-3,9\":\"outside\",\"-3,10\":\"outside\",\"-3,11\":\"outside\",\"-3,12\":\"outside\",\"-3,13\":\"outside\",\"-3,14\":\"outside\",\"-3,15\":\"outside\",\"-3,16\":\"outside\",\"-3,17\":\"outside\",\"-2,9\":\"outside\",\"-2,10\":\"outside\",\"-2,11\":\"outside\",\"-2,12\":\"outside\",\"-2,13\":\"outside\",\"-2,14\":\"outside\",\"-2,15\":\"outside\",\"-2,16\":\"outside\",\"-2,17\":\"outside\",\"-1,9\":\"outside\",\"-1,10\":\"outside\",\"-1,11\":\"outside\",\"-1,12\":\"outside\",\"-1,13\":\"outside\",\"-1,14\":\"outside\",\"-1,15\":\"outside\",\"-1,16\":\"outside\",\"-1,17\":\"outside\",\"0,9\":\"wall\",\"0,10\":\"wall\",\"0,11\":\"wall\",\"0,12\":\"wall\",\"0,13\":\"wall\",\"0,14\":\"wall\",\"0,15\":\"wall\",\"0,16\":\"wall\",\"0,17\":\"wall\",\"1,9\":\"open\",\"1,10\":\"open\",\"1,11\":\"open\",\"1,12\":\"wall\",\"1,13\":\"agent\",\"1,14\":\"open\",\"1,15\":\"open\",\"1,16\":\"wall\",\"1,17\":\"open\",\"2,9\":\"open\",\"2,10\":\"wall\",\"2,11\":\"open\",\"2,12\":\"wall\",\"2,13\":\"wall\",\"2,14\":\"wall\",\"2,15\":\"open\",\"2,16\":\"wall\",\"2,17\":\"open\",\"3,9\":\"open\",\"3,10\":\"wall\",\"3,11\":\"open\",\"3,12\":\"wall\",\"3,13\":\"open\",\"3,14\":\"open\",\"3,15\":\"open\",\"3,16\":\"wall\",\"3,17\":\"open\",\"4,9\":\"open\",\"4,10\":\"wall\",\"4,11\":\"open\",\"4,12\":\"wall\",\"4,13\":\"open\",\"4,14\":\"wall\",\"4,15\":\"wall\",\"4,16\":\"wall\",\"4,17\":\"open\",\"5,9\":\"open\",\"5,10\":\"wall\",\"5,11\":\"open\",\"5,12\":\"wall\",\"5,13\":\"open\",\"5,14\":\"open\",\"5,15\":\"open\",\"5,16\":\"open\",\"5,17\":\"open\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "9061a9d046f353451e0976e48b3b406bbc06d20e5d56a2c4d24a2fcd014f1466"}747{"id": "snake_food-000746", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,4), travelling west, length 3.\n............\n............\n............\n............\n............\n............\n............\n....Hoo.*...\n............\n............\n............\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "c8dadb217410413fd4b3a242d41dbc4d84cd2fb6c3c232e0c30a76816ef12f15"}748{"id": "move_two_step-000747", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 26, column 1. Rows increase to the south and columns to the east.\nRow 23: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall\nRow 24: column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is open\nRow 25: column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is wall, column 3 is open, column 4 is open\nRow 26: column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is the agent, column 2 is open, column 3 is open, column 4 is wall\nRow 27: column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is wall, column 3 is open, column 4 is open\nRow 28: column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze\nRow 29: column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze", "question": "The agent plans to move west, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "0f5d28eb44688275441618835d051800c41cf19c7b2363d0eac9a9ec973b3354"}749{"id": "stochastic-000748", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[4,0],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,-3\":\"outside\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"wall\",\"2,-3\":\"outside\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"open\",\"2,1\":\"open\",\"2,2\":\"open\",\"2,3\":\"open\",\"3,-3\":\"outside\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"open\",\"3,1\":\"wall\",\"3,2\":\"open\",\"3,3\":\"open\",\"4,-3\":\"outside\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"agent\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"5,-3\":\"outside\",\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"open\",\"5,1\":\"wall\",\"5,2\":\"open\",\"5,3\":\"open\",\"6,-3\":\"outside\",\"6,-2\":\"outside\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"wall\",\"6,2\":\"open\",\"6,3\":\"open\",\"7,-3\":\"outside\",\"7,-2\":\"outside\",\"7,-1\":\"outside\",\"7,0\":\"open\",\"7,1\":\"open\",\"7,2\":\"open\",\"7,3\":\"open\"}}", "question": "The actuator executes the intended move with probability 0.80; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.8, 0.19999999999999996], "deterministic": false, "fingerprint": "24754d1ccd56f31fab994cdbe889310cde6021d85e6f6f07d7cde3ced5f4ea5b"}750{"id": "snake_food-000749", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,8), travelling east, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n.......oH.....\n.......o*.....\n..............\n..............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 1, "answer": "south", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a8f6b45eafd1d2f4ec9a8d721b12f42eea17053e016527ade45a4a0c3b484a08"}751{"id": "cell_kind-000750", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 10, column 33. Rows increase to the south and columns to the east.\nRow 6: column 29 is open, column 30 is wall, column 31 is open, column 32 is open, column 33 is open, column 34 is open, column 35 is wall, column 36 is outside the maze, column 37 is outside the maze\nRow 7: column 29 is open, column 30 is open, column 31 is wall, column 32 is open, column 33 is wall, column 34 is wall, column 35 is wall, column 36 is outside the maze, column 37 is outside the maze\nRow 8: column 29 is open, column 30 is open, column 31 is wall, column 32 is wall, column 33 is open, column 34 is open, column 35 is open, column 36 is outside the maze, column 37 is outside the maze\nRow 9: column 29 is open, column 30 is wall, column 31 is wall, column 32 is open, column 33 is open, column 34 is open, column 35 is open, column 36 is outside the maze, column 37 is outside the maze\nRow 10: column 29 is open, column 30 is wall, column 31 is open, column 32 is open, column 33 is the agent, column 34 is open, column 35 is open, column 36 is outside the maze, column 37 is outside the maze\nRow 11: column 29 is open, column 30 is wall, column 31 is open, column 32 is wall, column 33 is wall, column 34 is open, column 35 is open, column 36 is outside the maze, column 37 is outside the maze\nRow 12: column 29 is wall, column 30 is wall, column 31 is wall, column 32 is wall, column 33 is wall, column 34 is wall, column 35 is open, column 36 is outside the maze, column 37 is outside the maze\nRow 13: column 29 is open, column 30 is open, column 31 is open, column 32 is wall, column 33 is wall, column 34 is open, column 35 is open, column 36 is outside the maze, column 37 is outside the maze\nRow 14: column 29 is open, column 30 is wall, column 31 is wall, column 32 is wall, column 33 is open, column 34 is open, column 35 is open, column 36 is outside the maze, column 37 is outside the maze", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "4953bb14ff4806573fc609ae3d58ce45102a36fe01798e68a895e9f166fbc0e2"}752{"id": "snake_safe-000751", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,5), travelling west, length 3.\n..........\n..........\n..........\n.....*....\n..........\n..........\n......o...\n.....Ho...\n..........\n..........", "question": "If the snake moves north on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "ab2e4469a55d4bcaddfa4fb43a6fb885b6a4b24ce3040830b38ff23cb8c0e708"}753{"id": "move_two_step-000752", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (5,2); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXX#.#....\nXX#.###.#\nXX#.....#\nXX#######\nXX#.A.#..\nXX#.#.#.#\nXX#.#....\nXX#######\nXXXXXXXXX", "question": "The agent plans to move west, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "e5a72a3f9fa027c516ccb7ebf21f94ccdc14cb81e59d8907aff87696e1145ee9"}754{"id": "only_way-000753", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 7. Rows increase to the south and columns to the east.\nRow 4: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall\nRow 5: column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is open, column 10 is wall\nRow 6: column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open, column 10 is wall\nRow 7: column 4 is wall, column 5 is open, column 6 is wall, column 7 is the agent, column 8 is wall, column 9 is open, column 10 is wall\nRow 8: column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall\nRow 9: column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open, column 10 is open\nRow 10: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3d73c2605b3ddf5455d9b482eb57777277d5b89d795bf3b48fd70aa629ff893d"}755{"id": "snake_food-000754", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,5), travelling south, length 3.\n..............\n............*.\n..............\n..............\n..............\n..............\n.....oo.......\n.....H........\n..............\n..............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a4bd990d782587725ea994c6f8eb6c81232ccf3b1ee7e6e5dd8ad0cacbd7e7d0"}756{"id": "stochastic-000755", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[19,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"15,5\":\"open\",\"15,6\":\"open\",\"15,7\":\"open\",\"15,8\":\"open\",\"15,9\":\"open\",\"15,10\":\"wall\",\"15,11\":\"open\",\"15,12\":\"open\",\"15,13\":\"open\",\"16,5\":\"wall\",\"16,6\":\"open\",\"16,7\":\"wall\",\"16,8\":\"wall\",\"16,9\":\"wall\",\"16,10\":\"wall\",\"16,11\":\"wall\",\"16,12\":\"wall\",\"16,13\":\"open\",\"17,5\":\"open\",\"17,6\":\"open\",\"17,7\":\"open\",\"17,8\":\"open\",\"17,9\":\"open\",\"17,10\":\"wall\",\"17,11\":\"open\",\"17,12\":\"wall\",\"17,13\":\"open\",\"18,5\":\"open\",\"18,6\":\"wall\",\"18,7\":\"open\",\"18,8\":\"wall\",\"18,9\":\"open\",\"18,10\":\"wall\",\"18,11\":\"open\",\"18,12\":\"wall\",\"18,13\":\"wall\",\"19,5\":\"open\",\"19,6\":\"open\",\"19,7\":\"open\",\"19,8\":\"wall\",\"19,9\":\"agent\",\"19,10\":\"open\",\"19,11\":\"open\",\"19,12\":\"open\",\"19,13\":\"open\",\"20,5\":\"wall\",\"20,6\":\"wall\",\"20,7\":\"wall\",\"20,8\":\"wall\",\"20,9\":\"wall\",\"20,10\":\"wall\",\"20,11\":\"wall\",\"20,12\":\"wall\",\"20,13\":\"wall\",\"21,5\":\"outside\",\"21,6\":\"outside\",\"21,7\":\"outside\",\"21,8\":\"outside\",\"21,9\":\"outside\",\"21,10\":\"outside\",\"21,11\":\"outside\",\"21,12\":\"outside\",\"21,13\":\"outside\",\"22,5\":\"outside\",\"22,6\":\"outside\",\"22,7\":\"outside\",\"22,8\":\"outside\",\"22,9\":\"outside\",\"22,10\":\"outside\",\"22,11\":\"outside\",\"22,12\":\"outside\",\"22,13\":\"outside\",\"23,5\":\"outside\",\"23,6\":\"outside\",\"23,7\":\"outside\",\"23,8\":\"outside\",\"23,9\":\"outside\",\"23,10\":\"outside\",\"23,11\":\"outside\",\"23,12\":\"outside\",\"23,13\":\"outside\"}}", "question": "The actuator executes the intended move with probability 0.80; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.8666666666666667, 0.1333333333333333], "deterministic": false, "fingerprint": "f8bdbbbb46459271ea17399316e6f8ca25484a8a65232d9d891e1067de66e78f"}757{"id": "snake_food-000756", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,9), travelling east, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n..*....ooH....\n..............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "fee499110565d43d463aed00704762d469e80f0fe950082af37a8b67cf8aedaa"}758{"id": "snake_food-000757", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,5), travelling south, length 3.\n........\n...*....\n........\n........\n....oo..\n.....H..\n........\n........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "7471961d5890f61da96a531f78575e4cc5bffa2e194f1ef3f52ac91b8c2ce515"}759{"id": "open_count-000758", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (2,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.....\n...#.\n..A..\n.....\n...#.", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 4, "answer": "4", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "b90b0c71792a7bd4492748a74f9e42a0190eab564ee7c361b5b082b3c88355ec"}760{"id": "stochastic-000759", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[2,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"-1,6\":\"outside\",\"-1,7\":\"outside\",\"-1,8\":\"outside\",\"-1,9\":\"outside\",\"-1,10\":\"outside\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"0,8\":\"wall\",\"0,9\":\"outside\",\"0,10\":\"outside\",\"1,4\":\"open\",\"1,5\":\"open\",\"1,6\":\"open\",\"1,7\":\"open\",\"1,8\":\"wall\",\"1,9\":\"outside\",\"1,10\":\"outside\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"2,6\":\"wall\",\"2,7\":\"agent\",\"2,8\":\"wall\",\"2,9\":\"outside\",\"2,10\":\"outside\",\"3,4\":\"open\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"3,8\":\"wall\",\"3,9\":\"outside\",\"3,10\":\"outside\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"open\",\"4,8\":\"wall\",\"4,9\":\"outside\",\"4,10\":\"outside\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"wall\",\"5,9\":\"outside\",\"5,10\":\"outside\"}}", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.2666666666666667, 0.7333333333333333], "deterministic": false, "fingerprint": "4b2927ace26ca8c6d29274f6777197846a3f427152b900e90a39a3597f3704e8"}761{"id": "stochastic-000760", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[2,5],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-2,5\":\"outside\",\"-2,6\":\"outside\",\"-2,7\":\"outside\",\"-2,8\":\"outside\",\"-2,9\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"-1,6\":\"outside\",\"-1,7\":\"outside\",\"-1,8\":\"outside\",\"-1,9\":\"outside\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"0,8\":\"wall\",\"0,9\":\"outside\",\"1,1\":\"open\",\"1,2\":\"wall\",\"1,3\":\"open\",\"1,4\":\"wall\",\"1,5\":\"open\",\"1,6\":\"open\",\"1,7\":\"open\",\"1,8\":\"wall\",\"1,9\":\"outside\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"open\",\"2,4\":\"wall\",\"2,5\":\"agent\",\"2,6\":\"wall\",\"2,7\":\"open\",\"2,8\":\"wall\",\"2,9\":\"outside\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"3,8\":\"wall\",\"3,9\":\"outside\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"open\",\"4,8\":\"wall\",\"4,9\":\"outside\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"wall\",\"5,9\":\"outside\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"outside\"}}", "question": "The actuator executes the intended move with probability 0.85; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.09999999999999998, 0.9], "deterministic": false, "fingerprint": "3f71428ed909d885cb91c198d1226a400b201927a5aceb0c288ec6c870e4bf25"}762{"id": "onward-000761", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 4, column 5. Rows increase to the south and columns to the east.\nRow 1: column 2 is wall, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall\nRow 2: column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall\nRow 3: column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 4: column 2 is wall, column 3 is open, column 4 is wall, column 5 is the agent, column 6 is wall, column 7 is open, column 8 is wall\nRow 5: column 2 is wall, column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 6: column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall\nRow 7: column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall", "question": "Suppose the agent moves one cell south. From that new cell, would a further step east be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "8fbd2ba21eaec85e8e23b0900c92cfe072bd7ca1f2d5eedb6b775cd103309d0d"}763{"id": "snake_food-000762", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,5), travelling east, length 3.\n*.........\n..........\n..........\n..........\n..........\n....o.....\n....oH....\n..........\n..........\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "0376ffb2f0b7a68f91bb9aa2292b05dead609731e08a2cef6822881c2afb641e"}764{"id": "reachable_within-000763", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,5), goal at (7,1).\n################\n#...#.#.......##\n#.#.#.###.###.##\n#.#.#...#...#.##\n#.#.###.###.#.##\n#.#...#.....#.##\n#.#.#########.##\n#G#.#.........##\n###.#.##########\n#...#A..#.....##\n#.#####.#.###.##\n#.....#.#...#.##\n#.###.#.#####.##\n#...#.........##\n################\n################", "question": "Can the agent reach the goal in at most 33 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "0323022c33960e05b3b2ed21ea16c39a75f1275dc7de16d5ee8c9db015332792"}765{"id": "cell_kind-000764", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 9, column 9. Rows increase to the south and columns to the east.\nRow 6: column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 7: column 6 is wall, column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 8: column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 9: column 6 is open, column 7 is open, column 8 is open, column 9 is the agent, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 10: column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 11: column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 12: column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "cdf7228b551f8157ecc1aa0730de12e2e983e0181f99f126bff6ffb9be193109"}766{"id": "reachable_within-000765", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,7), goal at (11,1).\n............\n##..........\n##....#.....\n...#...##...\n..........#.\n.#...#.A##.#\n..#..##.....\n.......#.##.\n#....##.....\n..#..##....#\n....#...#..#\n#G.#.....#..", "question": "Can the agent reach the goal in at most 11 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "2bbbb01f73ec955d9197b360082b51cbfecf204e9f8a59e33d750cd07bde043e"}767{"id": "reachable_within-000766", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (4,7), goal at (5,3).\n############\n#...#.....##\n#.#.#.###.##\n#.#.#.#.#.##\n#.#.#.#A#.##\n#.#G#.#.#.##\n#.###.#.#.##\n#.#...#.#.##\n#.###.#.#.##\n#.....#...##\n############\n############", "question": "Can the agent reach the goal in at most 42 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "492dc09dab926e7c5630417e73a0d45dba54c07d09077df2af40e66e51a37cbf"}768{"id": "goal_bearing-000767", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,6), goal at (7,3).\n#########\n#.......#\n#.#####.#\n#.....#.#\n#.###.#.#\n#.#...#.#\n#.#.###.#\n#.#G#.A.#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "2bb88eaa3bf60988be87f3a894f645d9aeace3b1b4aac585632550ff4f0a823d"}769{"id": "goal_bearing-000768", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,1), goal at (1,1).\n#########\n#G#.....#\n#.#.#.###\n#...#...#\n#######.#\n#A..#...#\n#.###.#.#\n#.....#.#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "62810f38b301525180230081af0968ddb0f696e38602bfb985c148b440151154"}770{"id": "move_two_step-000769", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[17,5],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"13,1\":\"open\",\"13,2\":\"wall\",\"13,3\":\"open\",\"13,4\":\"wall\",\"13,5\":\"open\",\"13,6\":\"open\",\"13,7\":\"open\",\"13,8\":\"open\",\"13,9\":\"open\",\"14,1\":\"open\",\"14,2\":\"wall\",\"14,3\":\"open\",\"14,4\":\"wall\",\"14,5\":\"wall\",\"14,6\":\"wall\",\"14,7\":\"wall\",\"14,8\":\"wall\",\"14,9\":\"wall\",\"15,1\":\"open\",\"15,2\":\"wall\",\"15,3\":\"open\",\"15,4\":\"open\",\"15,5\":\"open\",\"15,6\":\"open\",\"15,7\":\"open\",\"15,8\":\"open\",\"15,9\":\"open\",\"16,1\":\"open\",\"16,2\":\"wall\",\"16,3\":\"wall\",\"16,4\":\"wall\",\"16,5\":\"wall\",\"16,6\":\"wall\",\"16,7\":\"wall\",\"16,8\":\"wall\",\"16,9\":\"wall\",\"17,1\":\"open\",\"17,2\":\"wall\",\"17,3\":\"open\",\"17,4\":\"open\",\"17,5\":\"agent\",\"17,6\":\"open\",\"17,7\":\"open\",\"17,8\":\"open\",\"17,9\":\"open\",\"18,1\":\"open\",\"18,2\":\"wall\",\"18,3\":\"wall\",\"18,4\":\"wall\",\"18,5\":\"wall\",\"18,6\":\"wall\",\"18,7\":\"wall\",\"18,8\":\"wall\",\"18,9\":\"wall\",\"19,1\":\"open\",\"19,2\":\"open\",\"19,3\":\"open\",\"19,4\":\"open\",\"19,5\":\"open\",\"19,6\":\"open\",\"19,7\":\"open\",\"19,8\":\"open\",\"19,9\":\"open\",\"20,1\":\"wall\",\"20,2\":\"wall\",\"20,3\":\"wall\",\"20,4\":\"wall\",\"20,5\":\"wall\",\"20,6\":\"wall\",\"20,7\":\"wall\",\"20,8\":\"wall\",\"20,9\":\"wall\",\"21,1\":\"open\",\"21,2\":\"open\",\"21,3\":\"open\",\"21,4\":\"open\",\"21,5\":\"open\",\"21,6\":\"open\",\"21,7\":\"open\",\"21,8\":\"open\",\"21,9\":\"open\"}}", "question": "The agent plans to move west, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "eb61633230b1401ff70b53f126d296ecfc6b846e593f1716fc83bd4756ea957b"}771{"id": "snake_food-000770", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (3,2), travelling west, length 3.\n..*.....\n........\n........\n..Hoo...\n........\n........\n........\n........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "34bd00d1beeb2d8ef6039f6d3afb1dd2364bde99ce42527fd8b102b1f030be0b"}772{"id": "first_move-000771", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,1), goal at (1,1).\n############\n#G#...#...##\n#.#.#.###.##\n#...#...#.##\n#######.#.##\n#.....#...##\n#.#.#.###.##\n#.#.#...#.##\n#.#.#####.##\n#A#.......##\n############\n############", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "ee66778f2047ab8a927daae75f257cf7a6d064bfb589aac91f25d21828e92256"}773{"id": "move_legal-000772", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (3,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\n######XXX\n.....#XXX\n.###.#XXX\n...#A#XXX\n##.###XXX\n.#...#XXX\n####.#XXX\n.....#XXX", "question": "If the agent attempts one cell east (col+1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "88e535131a88b87a5b44be8f1ad5711653ee54e7322d0c77ce9097e8391a15a8"}774{"id": "cell_kind-000773", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[7,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"5,5\":\"wall\",\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"wall\",\"5,9\":\"outside\",\"6,5\":\"wall\",\"6,6\":\"open\",\"6,7\":\"open\",\"6,8\":\"open\",\"6,9\":\"outside\",\"7,5\":\"open\",\"7,6\":\"open\",\"7,7\":\"agent\",\"7,8\":\"wall\",\"7,9\":\"outside\",\"8,5\":\"open\",\"8,6\":\"open\",\"8,7\":\"open\",\"8,8\":\"open\",\"8,9\":\"outside\",\"9,5\":\"outside\",\"9,6\":\"outside\",\"9,7\":\"outside\",\"9,8\":\"outside\",\"9,9\":\"outside\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "bf930d48568e772e7266d324923e74f2eecfda68aa918e9f6cbdbd0d32e6d947"}775{"id": "reachable_within-000774", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,9), goal at (5,5).\n############\n#.........##\n#.#######.##\n#.......#A##\n#######.#.##\n#....G#.#.##\n#.#####.#.##\n#.#.....#.##\n#.#######.##\n#.........##\n############\n############", "question": "Can the agent reach the goal in at most 23 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "14e801625e99dbd0b60119282dc5088a6febcffced1d7ce8303a7479467cf249"}776{"id": "snake_safe-000775", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,7), travelling east, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n.....ooH......\n..............\n..............\n..............\n..............\n.....*........\n..............", "question": "If the snake moves west on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "52f99f3d855e673660eee5742abd3c618f765388a5961fa714b6aff059d588e4"}777{"id": "cell_kind-000776", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#######\n.......\n#######\n#..A..#\n#.#.###\n#.#...#\n#####.#", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "360bdc6d5df28b33bb92a9a2142d25fc5b3d7ae744a962a09158e799454f626d"}778{"id": "only_way-000777", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (25,24); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.........\n#########\n#........\n#.#######\n#.#.A...#\n#.#####.#\n#.......#\n#########\n.........", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 5, "answer": "more than one of them", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "27b2682c42b9db78add6d28ddf3cc909e16c92ef20643245fc52777f293d19ee"}779{"id": "move_two_step-000778", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 2, column 13. Rows increase to the south and columns to the east.\nRow -2: column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze\nRow -1: column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze\nRow 0: column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 1: column 9 is open, column 10 is open, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 2: column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is the agent, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 3: column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 4: column 9 is wall, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 5: column 9 is open, column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze\nRow 6: column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall, column 16 is outside the maze, column 17 is outside the maze", "question": "The agent plans to move north, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "9886df68c86aab115537272b4506bb1f244651461d2666bcd231c838edf7f7ff"}780{"id": "blocked_pair-000779", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 10, column 5. Rows increase to the south and columns to the east.\nRow 8: column 3 is wall, column 4 is open, column 5 is open, column 6 is open, column 7 is open\nRow 9: column 3 is open, column 4 is open, column 5 is wall, column 6 is open, column 7 is open\nRow 10: column 3 is open, column 4 is wall, column 5 is the agent, column 6 is open, column 7 is open\nRow 11: column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is wall\nRow 12: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze", "question": "Are BOTH the cell one step west and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "ee2afe45cb8a1e389dfa58d35fe4ee26a301002d393c06dc455ff2c939b12162"}781{"id": "goal_bearing-000780", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,7), goal at (11,5).\n################\n#...#...#.....##\n#.#.#.#A#.###.##\n#.#.#.#.#.#.#.##\n#.#.#.###.#.#.##\n#.#.#.......#.##\n#.#.#########.##\n#.#...........##\n#.##############\n#.............##\n#.###########.##\n#.#.#G........##\n#.#.############\n#.............##\n################\n################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 7, "answer": "south-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "96e72c55f39cebfa4b5839df7dd842ae05f37a8475c787e917e9d6b94ddad2c7"}782{"id": "reachable_within-000781", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,6), goal at (1,1).\n################\n#G........#...##\n#######.#.#.#.##\n#.......#...#.##\n#.###.#######.##\n#...#.A.......##\n#.#.##.##.###.##\n#...#.....#...##\n#.###.###.######\n#.#.....#.....##\n#.###.#######.##\n#.#...........##\n#.#.######.##.##\n#.............##\n################\n################", "question": "Can the agent reach the goal in at most 17 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "7e3398dd4498f1e47237b0f112e908277f2fa913f94f25c8e47e82268b9faa11"}783{"id": "plan_survives-000782", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 23, column 7. Rows increase to the south and columns to the east.\nRow 21: column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open\nRow 22: column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open\nRow 23: column 5 is open, column 6 is wall, column 7 is the agent, column 8 is wall, column 9 is open\nRow 24: column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open\nRow 25: column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open", "question": "The agent will attempt this plan in order: south, south, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "ef239dbfca2117b3f0ed5654b5e514a0cd37205e10ad7a0c4a0d1b4a82226d66"}784{"id": "move_two_step-000783", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[19,21],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"15,17\":\"open\",\"15,18\":\"wall\",\"15,19\":\"open\",\"15,20\":\"wall\",\"15,21\":\"open\",\"15,22\":\"wall\",\"15,23\":\"open\",\"15,24\":\"wall\",\"15,25\":\"open\",\"16,17\":\"open\",\"16,18\":\"wall\",\"16,19\":\"open\",\"16,20\":\"wall\",\"16,21\":\"open\",\"16,22\":\"wall\",\"16,23\":\"open\",\"16,24\":\"wall\",\"16,25\":\"open\",\"17,17\":\"open\",\"17,18\":\"wall\",\"17,19\":\"open\",\"17,20\":\"open\",\"17,21\":\"open\",\"17,22\":\"wall\",\"17,23\":\"open\",\"17,24\":\"wall\",\"17,25\":\"open\",\"18,17\":\"open\",\"18,18\":\"wall\",\"18,19\":\"wall\",\"18,20\":\"wall\",\"18,21\":\"wall\",\"18,22\":\"wall\",\"18,23\":\"open\",\"18,24\":\"wall\",\"18,25\":\"open\",\"19,17\":\"open\",\"19,18\":\"wall\",\"19,19\":\"open\",\"19,20\":\"open\",\"19,21\":\"agent\",\"19,22\":\"open\",\"19,23\":\"open\",\"19,24\":\"wall\",\"19,25\":\"open\",\"20,17\":\"wall\",\"20,18\":\"wall\",\"20,19\":\"wall\",\"20,20\":\"wall\",\"20,21\":\"wall\",\"20,22\":\"wall\",\"20,23\":\"wall\",\"20,24\":\"wall\",\"20,25\":\"open\",\"21,17\":\"open\",\"21,18\":\"open\",\"21,19\":\"open\",\"21,20\":\"open\",\"21,21\":\"open\",\"21,22\":\"open\",\"21,23\":\"open\",\"21,24\":\"open\",\"21,25\":\"open\",\"22,17\":\"wall\",\"22,18\":\"wall\",\"22,19\":\"wall\",\"22,20\":\"wall\",\"22,21\":\"wall\",\"22,22\":\"wall\",\"22,23\":\"wall\",\"22,24\":\"wall\",\"22,25\":\"open\",\"23,17\":\"open\",\"23,18\":\"open\",\"23,19\":\"open\",\"23,20\":\"open\",\"23,21\":\"open\",\"23,22\":\"open\",\"23,23\":\"open\",\"23,24\":\"wall\",\"23,25\":\"open\"}}", "question": "The agent plans to move east, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "6ddae40dfa9818f0d3f98dd45d13b668937372eaab553aa0bcb36ef786630109"}785{"id": "only_way-000784", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (21,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#######\n#...#..\n#.#.#.#\n#.#A#..\n#.#####\n#.#...#\n#.#.#.#", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "87467079d90346d1cce2c63a1ab00c9004fbd0ebc8f0aac68aa856814dcdc142"}786{"id": "only_way-000785", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (11,5); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#...\n#####\n.#A#.\n.#.#.\n...#.", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "fac49ca472041cebd2de20ce3c7d0056445e4073554713b6d3348166836d7ca7"}787{"id": "blocked_pair-000786", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 18. Rows increase to the south and columns to the east.\nRow 4: column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is open, column 21 is outside the maze\nRow 5: column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is wall, column 20 is open, column 21 is outside the maze\nRow 6: column 15 is wall, column 16 is open, column 17 is open, column 18 is wall, column 19 is open, column 20 is open, column 21 is outside the maze\nRow 7: column 15 is open, column 16 is wall, column 17 is open, column 18 is the agent, column 19 is open, column 20 is open, column 21 is outside the maze\nRow 8: column 15 is wall, column 16 is open, column 17 is wall, column 18 is open, column 19 is wall, column 20 is open, column 21 is outside the maze\nRow 9: column 15 is open, column 16 is open, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze\nRow 10: column 15 is wall, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is open, column 21 is outside the maze", "question": "Are BOTH the cell one step south and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "6a3c702e2467d704f99622304d72b502eaf038cb181b027e9191d4d14cfda735"}788{"id": "move_legal-000787", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[5,5],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"open\",\"5,3\":\"open\",\"5,4\":\"open\",\"5,5\":\"agent\",\"5,6\":\"wall\",\"5,7\":\"open\",\"6,3\":\"wall\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"6,7\":\"open\",\"7,3\":\"open\",\"7,4\":\"open\",\"7,5\":\"open\",\"7,6\":\"open\",\"7,7\":\"open\"}}", "question": "The agent wants to move north. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "c386303954dd8c75a306752df5bb9ba7fc7fdcc91c9ee9825f375295bd8c3afc"}789{"id": "snake_safe-000788", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,9), travelling north, length 3.\n........*.....\n..............\n..............\n..............\n..............\n.........H....\n........oo....\n..............\n..............\n..............\n..............\n..............\n..............\n..............", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "a8d7a7ff5370ff11792b23d8f9ae4901c42ac2e5065ce1b5480490a8799a9afe"}790{"id": "move_two_step-000789", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 17, column 3. Rows increase to the south and columns to the east.\nRow 13: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open\nRow 14: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 15: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open\nRow 16: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 17: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is the agent, column 4 is open, column 5 is open, column 6 is open, column 7 is open\nRow 18: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open\nRow 19: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open\nRow 20: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 21: column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze", "question": "The agent plans to move east, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "12a6fa01037ac75fe990e98ea92236567b9d8cb86aa910cc3a4e510590fdbf48"}791{"id": "goal_bearing-000790", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,7), goal at (5,3).\n#########\n#......A#\n#.#####.#\n#.#...#.#\n#.#.#.#.#\n#.#G#.#.#\n#.###.###\n#.......#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 7, "answer": "south-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "d6357919674a1e323453c8af34d4c8c9d8d0d22efdc69c1612b429789b6b05de"}792{"id": "first_move-000791", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,12), goal at (7,13).\n#####################\n#...........A.......#\n#.#################.#\n#.#...#...........#.#\n#.#.#.#.#########.#.#\n#.#.#.#.#.......#.#.#\n#.#.#.#.#.#####.#.#.#\n#.#.#.#.#....G#.#.#.#\n#.#.#.#.#######.#.#.#\n#.#.#.#.........#.#.#\n#.#.#.#.#########.#.#\n#.#.#.#.#.........#.#\n#.#.#.#.#.#######.#.#\n#.#.#.#.#.#.......#.#\n#.#.#.###.#########.#\n#.#.#.....#.........#\n#.#.#################\n#.#.................#\n#.#################.#\n#...................#\n#####################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "68af435043d25ef08f331b15c6118ebc368cc4885b068ae30ea0bebe7feeaeac"}793{"id": "first_move-000792", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (11,8), goal at (0,8).\n########G...\n#########...\n#########..#\n##########.#\n########....\n########....\n##########.#\n##########.#\n#########...\n########..#.\n###########.\n########A...", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "bfeda8a710952a12f89fa59664cff8e0a27d5ccee95d1f9e745d42abca924bf2"}794{"id": "move_legal-000793", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (9,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.###.#\n#.#.#.#\n#.#.#.#\n#.#A#.#\n#.#.#.#\n#.#.#.#\n#.#.#.#", "question": "Is the cell one step west of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "197076f9250fcb3863349666f6587421434d7147f9820ae8be554ae785c5c1ea"}795{"id": "first_move-000794", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,1), goal at (1,4).\n#########\n#...G.#.#\n#.###.#.#\n#...#...#\n###.###.#\n#A#.....#\n#.##.##.#\n#.......#\n#########", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "e6c74e3d7991b8daf52319890c905dfb5414dc22ae44d8922e8f2225db0a24e4"}796{"id": "stochastic-000795", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (4,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.###.\n.#.#.\n.#A##\n.#...\n.####", "question": "The actuator executes the intended move with probability 0.70; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.7999999999999999, 0.20000000000000004], "deterministic": false, "fingerprint": "f9d064fba77a53b14c2b351b4256a6150a9b4d2f8b53cf3ae8f674abbb2483de"}797{"id": "onward-000796", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (5,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX#.......\nX#.#####.\nX#...#...\nX#.###.#.\nX#.#A#.#.\nX#.#.#.#.\nX#.#...#.\nX########\nXXXXXXXXX", "question": "Suppose the agent moves one cell south. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "c0a47cbb4da03335ee399b4807e583ddc8077ef7a3049ea0a584ebf910b1b0c7"}798{"id": "plan_survives-000797", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (13,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#...#.#.\n.#.###.#.\n...#...#.\n##.#.####\n.#.#A#...\n##.#.#.#.\n...#.#.#.\n########.\n...#.....", "question": "The agent will attempt this plan in order: south, south, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "a124bf05c5ccf8a5c5d07bde1715f8c7dc76bbe9f693f665c015b9303573c009"}799{"id": "open_count-000798", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 6, column 3. Rows increase to the south and columns to the east.\nRow 3: column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is open, column 5 is open, column 6 is open\nRow 4: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open, column 6 is open\nRow 5: column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open\nRow 6: column 0 is open, column 1 is open, column 2 is open, column 3 is the agent, column 4 is wall, column 5 is wall, column 6 is wall\nRow 7: column 0 is open, column 1 is wall, column 2 is open, column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall\nRow 8: column 0 is open, column 1 is open, column 2 is open, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall\nRow 9: column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 3, "answer": "3", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "6df2bad498a648c6998c3dccb6ed4a4c7f8ba7c7f15c263daca8e376fe53d9a8"}800{"id": "blocked_pair-000799", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 10. Rows increase to the south and columns to the east.\nRow -2: column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze\nRow -1: column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze\nRow 0: column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is wall\nRow 1: column 7 is open, column 8 is wall, column 9 is open, column 10 is the agent, column 11 is open, column 12 is wall, column 13 is open\nRow 2: column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open\nRow 3: column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is open, column 12 is open, column 13 is open\nRow 4: column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is open, column 12 is wall, column 13 is wall", "question": "Are BOTH the cell one step south and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "8dd3f623f65a903e4be907ffcad8d100304177dcc37fdc3e84c54aa13ac035fa"}801{"id": "cell_kind-000800", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[13,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"9,-3\":\"outside\",\"9,-2\":\"outside\",\"9,-1\":\"outside\",\"9,0\":\"wall\",\"9,1\":\"open\",\"9,2\":\"open\",\"9,3\":\"open\",\"9,4\":\"wall\",\"9,5\":\"open\",\"10,-3\":\"outside\",\"10,-2\":\"outside\",\"10,-1\":\"outside\",\"10,0\":\"wall\",\"10,1\":\"open\",\"10,2\":\"wall\",\"10,3\":\"open\",\"10,4\":\"wall\",\"10,5\":\"open\",\"11,-3\":\"outside\",\"11,-2\":\"outside\",\"11,-1\":\"outside\",\"11,0\":\"wall\",\"11,1\":\"open\",\"11,2\":\"open\",\"11,3\":\"open\",\"11,4\":\"wall\",\"11,5\":\"open\",\"12,-3\":\"outside\",\"12,-2\":\"outside\",\"12,-1\":\"outside\",\"12,0\":\"wall\",\"12,1\":\"open\",\"12,2\":\"wall\",\"12,3\":\"wall\",\"12,4\":\"wall\",\"12,5\":\"wall\",\"13,-3\":\"outside\",\"13,-2\":\"outside\",\"13,-1\":\"outside\",\"13,0\":\"wall\",\"13,1\":\"agent\",\"13,2\":\"open\",\"13,3\":\"open\",\"13,4\":\"open\",\"13,5\":\"open\",\"14,-3\":\"outside\",\"14,-2\":\"outside\",\"14,-1\":\"outside\",\"14,0\":\"wall\",\"14,1\":\"wall\",\"14,2\":\"wall\",\"14,3\":\"wall\",\"14,4\":\"wall\",\"14,5\":\"wall\",\"15,-3\":\"outside\",\"15,-2\":\"outside\",\"15,-1\":\"outside\",\"15,0\":\"wall\",\"15,1\":\"wall\",\"15,2\":\"wall\",\"15,3\":\"wall\",\"15,4\":\"wall\",\"15,5\":\"wall\",\"16,-3\":\"outside\",\"16,-2\":\"outside\",\"16,-1\":\"outside\",\"16,0\":\"outside\",\"16,1\":\"outside\",\"16,2\":\"outside\",\"16,3\":\"outside\",\"16,4\":\"outside\",\"16,5\":\"outside\",\"17,-3\":\"outside\",\"17,-2\":\"outside\",\"17,-1\":\"outside\",\"17,0\":\"outside\",\"17,1\":\"outside\",\"17,2\":\"outside\",\"17,3\":\"outside\",\"17,4\":\"outside\",\"17,5\":\"outside\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "ac9a8b20f7eca73f61dda5679665b77da92a993ba2d085110d4b41b293f32696"}802{"id": "plan_survives-000801", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[33,33],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"29,29\":\"wall\",\"29,30\":\"open\",\"29,31\":\"open\",\"29,32\":\"open\",\"29,33\":\"open\",\"29,34\":\"open\",\"29,35\":\"open\",\"29,36\":\"outside\",\"29,37\":\"outside\",\"30,29\":\"open\",\"30,30\":\"open\",\"30,31\":\"open\",\"30,32\":\"open\",\"30,33\":\"wall\",\"30,34\":\"wall\",\"30,35\":\"open\",\"30,36\":\"outside\",\"30,37\":\"outside\",\"31,29\":\"open\",\"31,30\":\"wall\",\"31,31\":\"open\",\"31,32\":\"open\",\"31,33\":\"wall\",\"31,34\":\"wall\",\"31,35\":\"open\",\"31,36\":\"outside\",\"31,37\":\"outside\",\"32,29\":\"open\",\"32,30\":\"open\",\"32,31\":\"open\",\"32,32\":\"open\",\"32,33\":\"open\",\"32,34\":\"open\",\"32,35\":\"open\",\"32,36\":\"outside\",\"32,37\":\"outside\",\"33,29\":\"wall\",\"33,30\":\"open\",\"33,31\":\"wall\",\"33,32\":\"open\",\"33,33\":\"agent\",\"33,34\":\"open\",\"33,35\":\"open\",\"33,36\":\"outside\",\"33,37\":\"outside\",\"34,29\":\"wall\",\"34,30\":\"open\",\"34,31\":\"wall\",\"34,32\":\"open\",\"34,33\":\"open\",\"34,34\":\"open\",\"34,35\":\"open\",\"34,36\":\"outside\",\"34,37\":\"outside\",\"35,29\":\"open\",\"35,30\":\"open\",\"35,31\":\"open\",\"35,32\":\"open\",\"35,33\":\"open\",\"35,34\":\"open\",\"35,35\":\"wall\",\"35,36\":\"outside\",\"35,37\":\"outside\",\"36,29\":\"outside\",\"36,30\":\"outside\",\"36,31\":\"outside\",\"36,32\":\"outside\",\"36,33\":\"outside\",\"36,34\":\"outside\",\"36,35\":\"outside\",\"36,36\":\"outside\",\"36,37\":\"outside\",\"37,29\":\"outside\",\"37,30\":\"outside\",\"37,31\":\"outside\",\"37,32\":\"outside\",\"37,33\":\"outside\",\"37,34\":\"outside\",\"37,35\":\"outside\",\"37,36\":\"outside\",\"37,37\":\"outside\"}}", "question": "The agent will attempt this plan in order: west, west, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "7b65a37879a8a23c47b92a04ff36c59f2c8c8db3b5c0afb6aa44e2d5fd3f677c"}803{"id": "snake_food-000802", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,5), travelling south, length 3.\n..........\n..........\n..........\n..........\n..........\n..........\n.....o....\n.....o....\n.*...H....\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "0ab0824581894693089ff88ea71fadcdef145c7f6ed33524764a33075ee2ca8a"}804{"id": "move_legal-000803", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 30, column 15. Rows increase to the south and columns to the east.\nRow 28: column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall\nRow 29: column 13 is open, column 14 is wall, column 15 is open, column 16 is open, column 17 is open\nRow 30: column 13 is open, column 14 is wall, column 15 is the agent, column 16 is wall, column 17 is wall\nRow 31: column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open\nRow 32: column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is open", "question": "If the agent attempts one cell north (row-1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "5f2da0148cd8323679dd9ed4147b9f4a0895b2ca7a027e50e0c824057f4b2329"}805{"id": "move_two_step-000804", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[2,25],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,22\":\"outside\",\"-1,23\":\"outside\",\"-1,24\":\"outside\",\"-1,25\":\"outside\",\"-1,26\":\"outside\",\"-1,27\":\"outside\",\"-1,28\":\"outside\",\"0,22\":\"wall\",\"0,23\":\"wall\",\"0,24\":\"wall\",\"0,25\":\"wall\",\"0,26\":\"wall\",\"0,27\":\"wall\",\"0,28\":\"outside\",\"1,22\":\"open\",\"1,23\":\"open\",\"1,24\":\"open\",\"1,25\":\"open\",\"1,26\":\"wall\",\"1,27\":\"wall\",\"1,28\":\"outside\",\"2,22\":\"wall\",\"2,23\":\"wall\",\"2,24\":\"wall\",\"2,25\":\"agent\",\"2,26\":\"wall\",\"2,27\":\"wall\",\"2,28\":\"outside\",\"3,22\":\"open\",\"3,23\":\"open\",\"3,24\":\"wall\",\"3,25\":\"open\",\"3,26\":\"wall\",\"3,27\":\"wall\",\"3,28\":\"outside\",\"4,22\":\"wall\",\"4,23\":\"open\",\"4,24\":\"wall\",\"4,25\":\"wall\",\"4,26\":\"wall\",\"4,27\":\"wall\",\"4,28\":\"outside\",\"5,22\":\"wall\",\"5,23\":\"open\",\"5,24\":\"open\",\"5,25\":\"open\",\"5,26\":\"wall\",\"5,27\":\"wall\",\"5,28\":\"outside\"}}", "question": "The agent plans to move north, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "5d1a8b2fee50c1f9d39f49a5443c9d4d8d5de89e5f6c4b4fd92a3b6cce56f915"}806{"id": "snake_food-000805", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,5), travelling south, length 3.\n........\n......*.\n........\n........\n.....oo.\n.....H..\n........\n........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "1c5d4a16b768daa00a984f146200ea760664770b68c475510d38bf1a66e0d3bb"}807{"id": "distance_band-000806", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,9), goal at (13,1).\n################\n#.#...........##\n#.#.#.#.#.#.#.##\n#...#.#.#...#.##\n#..##.#.#.#.#.##\n#.....#.....#.##\n#.#####.#.###.##\n#...#...#.....##\n#.#.#.###.###.##\n#.#...#...#.#.##\n#.###.###.#.#.##\n#.........#...##\n###...#.###.####\n#G.......A....##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 1, "answer": "5 to 9 moves", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "e3ecef5624cc625870aa08fa5da193ebc72ce43e0e58228b79bccbc87ad26f33"}808{"id": "onward-000807", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 10, column 11. Rows increase to the south and columns to the east.\nRow 7: column 8 is wall, column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall\nRow 8: column 8 is wall, column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall\nRow 9: column 8 is wall, column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall\nRow 10: column 8 is wall, column 9 is open, column 10 is wall, column 11 is the agent, column 12 is wall, column 13 is open, column 14 is wall\nRow 11: column 8 is wall, column 9 is open, column 10 is wall, column 11 is open, column 12 is wall, column 13 is open, column 14 is wall\nRow 12: column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open, column 14 is wall\nRow 13: column 8 is open, column 9 is open, column 10 is open, column 11 is open, column 12 is open, column 13 is open, column 14 is wall", "question": "Suppose the agent moves one cell north. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "3521673ec53090ac81893da681ad4346f4c5310beab85102b107c13e307b914e"}809{"id": "stochastic-000808", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 7, column 1. Rows increase to the south and columns to the east.\nRow 3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open\nRow 4: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall\nRow 5: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 6: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 7: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 8: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall\nRow 9: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open\nRow 10: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 11: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.7333333333333334, 0.26666666666666666], "deterministic": false, "fingerprint": "8f95b7cadde44da95c4d5c57dd76c8064530fc50c0f3ab94238e948f76f3915e"}810{"id": "move_legal-000809", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[1,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,-2\":\"outside\",\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"agent\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"open\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"wall\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\"}}", "question": "If the agent attempts one cell north (row-1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "aae4f5f400c9dddd51c56f248009c601145d676dbf305942e5f648cbaa84b141"}811{"id": "move_two_step-000810", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (6,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n####.#XXX\n.....#XXX\n.#####XXX\n.#...#XXX\n##.#A#XXX\n...#.#XXX\n######XXX\nXXXXXXXXX\nXXXXXXXXX", "question": "The agent plans to move south, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "7677f8efa6ca7becfbfded692517ac025a9ef813b631630673bed9c10135e064"}812{"id": "only_way-000811", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (27,25); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.#.###\n......#\n#####.#\n...A#..\n#######\n.......\n####.##", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "5270176660aa3fe34b4378c84b180b08d3c489f5bb7e76db4d1cccde4a7327f1"}813{"id": "snake_safe-000812", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,4), travelling south, length 3.\n............\n............\n............\n............\n............\n....o.......\n....o.......\n....H.......\n............\n..........*.\n............\n............", "question": "If the snake moves north on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "0dc0c860800ed54a43041914ab8bbe8bd30a18f200cc52a54a8098c7a055f919"}814{"id": "blocked_pair-000813", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (12,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#...##X\n#.#.##X\n..#.##X\n#.#A##X\n..#.##X\n######X\n######X", "question": "Are BOTH the cell one step east and the cell one step west blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "9ba02afc9f397122ae2341b8136548e0df7d1b5218b45dbc6810dd5dc517c139"}815{"id": "open_count-000814", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 8, column 5. Rows increase to the south and columns to the east.\nRow 6: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is wall\nRow 7: column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall\nRow 8: column 3 is open, column 4 is open, column 5 is the agent, column 6 is open, column 7 is wall\nRow 9: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze\nRow 10: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 3, "answer": "3", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "ba372af631a65dd8cd186952161704c0321fcec47a34cb1ccb2f2d7f4d584b19"}816{"id": "plan_survives-000815", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (6,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX##..\nX...#\nX.A..\nX#...\nX###.", "question": "The agent will attempt this plan in order: north, south, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "eb257f75b21e2aa2830ce5e9f5fbc2313d0129054e0373c3f4122f6db287f8cf"}817{"id": "move_two_step-000816", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (10,11); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#...#.#\n#.###.#\n#.#.#.#\n#.#A#.#\n..#.#.#\n###.#.#\n#.....#", "question": "The agent plans to move south, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "a9b0076dac618e43abfb8ca78afebd3b137d597a8e0735a5f42736d4c7bf66e4"}818{"id": "goal_bearing-000817", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (10,15), goal at (19,27).\n####################################\n#.....#.....................#.....##\n#.###.#.###############.###.#.###.##\n#.#.#.#.#.............#.#.#.#.#.#.##\n#.#.#.#.#.###########.#.#.#.#.#.#.##\n#.#.#.#.#.#.........#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#######.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.....#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.###.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#A#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#...#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#####.#.#.#.#.#.#.#.##\n#.#...#.#.#.#.......#.#.#.#.#.#.#.##\n#.#####.#.#.#########.#.#.#.#.#.#.##\n#.........#.........#.#.#.#.#...#.##\n###################.#.#.#.#.#####.##\n#.......#...........#.#.#.........##\n#.#####.#.###########.#.#########.##\n#.#...#.#.............#.#..G#...#.##\n#.#.#.#.#.#############.#.#####.#.##\n#.#.#.#.#.#...........#.#.#.....#.##\n#.#.#.#.#.#.#######.#.#.#.#.###.#.##\n#.#.#.#.#.#.#.....#.#.#.#.#.#...#.##\n#.#.#.#.#.#.#.###.#.#.#.#.#.#####.##\n#.#.#.#.#.#.#...#.#.#.#.#.#.......##\n#.#.#.#.#.#.#####.#.#.#.#.#.########\n#.#.#.#.#.#.......#.#.#.#.#.......##\n#.#.#.#.#.#########.#.#.#.#######.##\n#.#.#.#.#...........#.#.#...#...#.##\n#.#.###.###############.###.#.#.#.##\n#.#.......................#.#.#.#.##\n#.#########################.#.###.##\n#...........................#.....##\n####################################\n####################################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 6, "answer": "south-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "1c534cc0151f897756373ba4c0aae82396489554aad0e4b84571231ceca59b53"}819{"id": "snake_food-000818", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,9), travelling east, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n.......*......\n..............\n.......ooH....\n..............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b3da3a5c3e011f59034d1ad7364a0a0b49a178107d8910b21a27ae7855519025"}820{"id": "goal_bearing-000819", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,3), goal at (1,3).\n############\n#..G#...#.##\n#.###.#.#.##\n#.#A..#.#.##\n#.#####.#.##\n#.....#.#.##\n#.###.#.#.##\n#.#...#.#.##\n#.#.###.#.##\n#.#.......##\n############\n############", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "9c06ca66e1f8d4aad63a4ab7331afc4d9b26397b41ed07164cce464d0389961f"}821{"id": "goal_bearing-000820", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,9), goal at (7,1).\n############\n#.........##\n#.#.#.#.#.##\n#.#...#..A##\n#.###.#.####\n#...#.#...##\n###.#####.##\n#G#.#.....##\n#.#.#.###.##\n#.....#...##\n############\n############", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 7, "answer": "south-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "10a8b3a320ffb4936e9c0086d78d8bf19d9b782fffe57e9d871547ca85404013"}822{"id": "plan_survives-000821", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (16,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX#.#.#.#.\nX#.#.#.#.\nX#.#.#.#.\nX#.#.#.#.\nX#.#A#.#.\nX#.#.#.#.\nX#.###.#.\nX#.....#.\nX#######.", "question": "The agent will attempt this plan in order: north, west, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "82f21209a450ed709f19b1b7a4e13ca3df30974e4b5aa9c4ce0c078ec46ffc4b"}823{"id": "stochastic-000822", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 2, column 3. Rows increase to the south and columns to the east.\nRow -2: column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze\nRow -1: column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze\nRow 0: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 1: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is open\nRow 2: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is the agent, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open\nRow 3: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open\nRow 4: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open\nRow 5: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is open\nRow 6: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall", "question": "The actuator executes the intended move with probability 0.85; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9, 0.10000000000000002], "deterministic": false, "fingerprint": "e0b8596322faca235b6f905784be30f04020b93c99fad7a5ee295f735c537909"}824{"id": "onward-000823", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[24,25],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"20,21\":\"open\",\"20,22\":\"wall\",\"20,23\":\"wall\",\"20,24\":\"wall\",\"20,25\":\"wall\",\"20,26\":\"wall\",\"20,27\":\"wall\",\"20,28\":\"outside\",\"20,29\":\"outside\",\"21,21\":\"open\",\"21,22\":\"open\",\"21,23\":\"open\",\"21,24\":\"open\",\"21,25\":\"open\",\"21,26\":\"wall\",\"21,27\":\"wall\",\"21,28\":\"outside\",\"21,29\":\"outside\",\"22,21\":\"wall\",\"22,22\":\"open\",\"22,23\":\"wall\",\"22,24\":\"wall\",\"22,25\":\"open\",\"22,26\":\"wall\",\"22,27\":\"wall\",\"22,28\":\"outside\",\"22,29\":\"outside\",\"23,21\":\"open\",\"23,22\":\"open\",\"23,23\":\"open\",\"23,24\":\"wall\",\"23,25\":\"open\",\"23,26\":\"wall\",\"23,27\":\"wall\",\"23,28\":\"outside\",\"23,29\":\"outside\",\"24,21\":\"wall\",\"24,22\":\"wall\",\"24,23\":\"open\",\"24,24\":\"wall\",\"24,25\":\"agent\",\"24,26\":\"wall\",\"24,27\":\"wall\",\"24,28\":\"outside\",\"24,29\":\"outside\",\"25,21\":\"open\",\"25,22\":\"open\",\"25,23\":\"open\",\"25,24\":\"open\",\"25,25\":\"open\",\"25,26\":\"wall\",\"25,27\":\"wall\",\"25,28\":\"outside\",\"25,29\":\"outside\",\"26,21\":\"wall\",\"26,22\":\"wall\",\"26,23\":\"wall\",\"26,24\":\"wall\",\"26,25\":\"wall\",\"26,26\":\"wall\",\"26,27\":\"wall\",\"26,28\":\"outside\",\"26,29\":\"outside\",\"27,21\":\"wall\",\"27,22\":\"wall\",\"27,23\":\"wall\",\"27,24\":\"wall\",\"27,25\":\"wall\",\"27,26\":\"wall\",\"27,27\":\"wall\",\"27,28\":\"outside\",\"27,29\":\"outside\",\"28,21\":\"outside\",\"28,22\":\"outside\",\"28,23\":\"outside\",\"28,24\":\"outside\",\"28,25\":\"outside\",\"28,26\":\"outside\",\"28,27\":\"outside\",\"28,28\":\"outside\",\"28,29\":\"outside\"}}", "question": "Suppose the agent moves one cell north. From that new cell, would a further step north be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "77934e5a8877c3b3da83397714a9397310f9d29aa5bf1a1fabb9b66f7b3ea799"}825{"id": "move_two_step-000824", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[19,19],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"16,16\":\"wall\",\"16,17\":\"open\",\"16,18\":\"wall\",\"16,19\":\"open\",\"16,20\":\"wall\",\"16,21\":\"outside\",\"16,22\":\"outside\",\"17,16\":\"open\",\"17,17\":\"open\",\"17,18\":\"open\",\"17,19\":\"open\",\"17,20\":\"wall\",\"17,21\":\"outside\",\"17,22\":\"outside\",\"18,16\":\"wall\",\"18,17\":\"wall\",\"18,18\":\"wall\",\"18,19\":\"wall\",\"18,20\":\"wall\",\"18,21\":\"outside\",\"18,22\":\"outside\",\"19,16\":\"open\",\"19,17\":\"open\",\"19,18\":\"open\",\"19,19\":\"agent\",\"19,20\":\"wall\",\"19,21\":\"outside\",\"19,22\":\"outside\",\"20,16\":\"wall\",\"20,17\":\"wall\",\"20,18\":\"wall\",\"20,19\":\"wall\",\"20,20\":\"wall\",\"20,21\":\"outside\",\"20,22\":\"outside\",\"21,16\":\"outside\",\"21,17\":\"outside\",\"21,18\":\"outside\",\"21,19\":\"outside\",\"21,20\":\"outside\",\"21,21\":\"outside\",\"21,22\":\"outside\",\"22,16\":\"outside\",\"22,17\":\"outside\",\"22,18\":\"outside\",\"22,19\":\"outside\",\"22,20\":\"outside\",\"22,21\":\"outside\",\"22,22\":\"outside\"}}", "question": "The agent plans to move west, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "60b649dbe670d4263865b6a9fdf38e01e0ade816c9175b579d5e0e0f2ce78b97"}826{"id": "only_way-000825", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (11,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXX#.#.#\nXX#.#.#\nXX###.#\nXX#A#..\nXX#.###\nXX#....\nXX#####", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "aac38d1f7c4f451767d9e46d13fd8a5fa2fe0366085c157ed8075e95cb5782d6"}827{"id": "cell_kind-000826", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,33],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,30\":\"wall\",\"0,31\":\"wall\",\"0,32\":\"wall\",\"0,33\":\"wall\",\"0,34\":\"wall\",\"0,35\":\"wall\",\"0,36\":\"outside\",\"1,30\":\"open\",\"1,31\":\"open\",\"1,32\":\"wall\",\"1,33\":\"open\",\"1,34\":\"wall\",\"1,35\":\"wall\",\"1,36\":\"outside\",\"2,30\":\"wall\",\"2,31\":\"open\",\"2,32\":\"wall\",\"2,33\":\"open\",\"2,34\":\"wall\",\"2,35\":\"wall\",\"2,36\":\"outside\",\"3,30\":\"wall\",\"3,31\":\"open\",\"3,32\":\"open\",\"3,33\":\"agent\",\"3,34\":\"wall\",\"3,35\":\"wall\",\"3,36\":\"outside\",\"4,30\":\"wall\",\"4,31\":\"open\",\"4,32\":\"wall\",\"4,33\":\"open\",\"4,34\":\"wall\",\"4,35\":\"wall\",\"4,36\":\"outside\",\"5,30\":\"wall\",\"5,31\":\"open\",\"5,32\":\"open\",\"5,33\":\"open\",\"5,34\":\"wall\",\"5,35\":\"wall\",\"5,36\":\"outside\",\"6,30\":\"wall\",\"6,31\":\"open\",\"6,32\":\"wall\",\"6,33\":\"open\",\"6,34\":\"wall\",\"6,35\":\"wall\",\"6,36\":\"outside\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "00928806119907b9dd293b0ac1d2b863963004b054f738344524526c207207a3"}828{"id": "cell_kind-000827", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n...#X\n.###X\n.#A#X\n.#.#X\n.#.#X", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "5dc1d9853be23b341d73b957521c615b11623a8505f10ef0c69e8864424e9dd4"}829{"id": "open_count-000828", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[5,4],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"1,6\":\"open\",\"1,7\":\"open\",\"1,8\":\"wall\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"2,6\":\"wall\",\"2,7\":\"open\",\"2,8\":\"wall\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"3,8\":\"wall\",\"4,0\":\"wall\",\"4,1\":\"wall\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"open\",\"4,8\":\"wall\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"agent\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"open\",\"6,0\":\"wall\",\"6,1\":\"wall\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"open\",\"6,6\":\"wall\",\"6,7\":\"wall\",\"6,8\":\"wall\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"open\",\"7,3\":\"open\",\"7,4\":\"wall\",\"7,5\":\"open\",\"7,6\":\"wall\",\"7,7\":\"open\",\"7,8\":\"open\",\"8,0\":\"wall\",\"8,1\":\"open\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"wall\",\"8,6\":\"wall\",\"8,7\":\"open\",\"8,8\":\"wall\",\"9,0\":\"wall\",\"9,1\":\"open\",\"9,2\":\"open\",\"9,3\":\"open\",\"9,4\":\"open\",\"9,5\":\"open\",\"9,6\":\"open\",\"9,7\":\"open\",\"9,8\":\"open\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 2, "answer": "2", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "6851e543b4d3962e555fcb0ee48131e1ec4fcb498ccbe041f2b90cbb3abf61f7"}830{"id": "onward-000829", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[4,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,4\":\"open\",\"1,5\":\"open\",\"1,6\":\"open\",\"1,7\":\"open\",\"1,8\":\"wall\",\"1,9\":\"outside\",\"1,10\":\"outside\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"2,6\":\"wall\",\"2,7\":\"wall\",\"2,8\":\"wall\",\"2,9\":\"outside\",\"2,10\":\"outside\",\"3,4\":\"open\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"3,8\":\"wall\",\"3,9\":\"outside\",\"3,10\":\"outside\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"agent\",\"4,8\":\"wall\",\"4,9\":\"outside\",\"4,10\":\"outside\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"wall\",\"5,9\":\"outside\",\"5,10\":\"outside\",\"6,4\":\"wall\",\"6,5\":\"open\",\"6,6\":\"wall\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"outside\",\"6,10\":\"outside\",\"7,4\":\"wall\",\"7,5\":\"open\",\"7,6\":\"open\",\"7,7\":\"open\",\"7,8\":\"wall\",\"7,9\":\"outside\",\"7,10\":\"outside\"}}", "question": "Suppose the agent moves one cell north. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "95998b3fa76b4d3b60bbd45bd57825ba0c22b4c39e2f85639db17ccc477faf89"}831{"id": "first_move-000830", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,9), goal at (29,11).\n####################################\n#........A..#.....................##\n#.###############################.##\n#.#...............................##\n#.#.###.#########################.##\n#.#.#.#.#.......................#.##\n#.#.#.#.#.###########.#########.#.##\n#.#...#.#.#.........#.#.......#.#.##\n#.#####.#.#.###.###.#.#.#####.#.#.##\n#.......#.#.#.#.#.#.#.#.#...#.#.#.##\n#########.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.......#.#.#.#.#...#.#.#.#.#.#.#.##\n#.#####.#.#.#.#.#####.#.###.#.#.#.##\n#.#...#.#.#.#.#.#.....#.....#.#.#.##\n#.#.###.#.#.#.#.#.###########.#.#.##\n#.#...#.#.#.#.#.#.#...........#.#.##\n#.###.#.#.#.#.#.#.#.#########.#.#.##\n#.#...#.#.#.#.#.#.#.#.......#.#.#.##\n#.#.###.#.#.#.#.#.#.#.#####.#.#.#.##\n#.#.....#.#.#.#.#.#.#.#...#.#.#.#.##\n#.#######.#.#.#.#.#.#.#.###.#.#.#.##\n#.........#.#.#.#.#.#.#.....#.#.#.##\n#.#########.#.#.#.#.#.#######.#.#.##\n#.#...........#.#.#.#.........#.#.##\n#.#############.#.#.###########.#.##\n#...............#.#.............#.##\n###############.#.#.#############.##\n#.............#.#.#.#...........#.##\n#.###########.#.#.#.#.#####.###.#.##\n#.#........G#.#.#.#.#.....#.#.#.#.##\n#.#.#########.#.#.#.#######.#.#.#.##\n#.#...........#.#.#.........#...#.##\n#.#############.#.###############.##\n#...............#.................##\n####################################\n####################################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "b6909fb2686983af883d7a9f16a131f0bc680ca18ce0e58806a9f4d7731115d5"}832{"id": "cell_kind-000831", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"1,6\":\"open\",\"2,0\":\"wall\",\"2,1\":\"wall\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"2,6\":\"wall\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"agent\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"open\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"open\",\"6,0\":\"wall\",\"6,1\":\"wall\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "74bddb929f43092412dd757e92a32fa1f68121a27b25ca88704613ea8f8d2db3"}833{"id": "snake_food-000832", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,7), travelling east, length 3.\n..............\n..............\n..*...........\n..............\n..............\n..............\n..............\n.....ooH......\n..............\n..............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "192aabf4b1295a9d5a8a0b8e3f12b868da3af14c9f7aedf764102e05d49aab18"}834{"id": "move_two_step-000833", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 20, column 17. Rows increase to the south and columns to the east.\nRow 16: column 13 is open, column 14 is open, column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is open, column 21 is outside the maze\nRow 17: column 13 is open, column 14 is open, column 15 is open, column 16 is open, column 17 is open, column 18 is wall, column 19 is open, column 20 is open, column 21 is outside the maze\nRow 18: column 13 is wall, column 14 is open, column 15 is wall, column 16 is open, column 17 is open, column 18 is open, column 19 is wall, column 20 is wall, column 21 is outside the maze\nRow 19: column 13 is open, column 14 is wall, column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is wall, column 21 is outside the maze\nRow 20: column 13 is open, column 14 is open, column 15 is open, column 16 is open, column 17 is the agent, column 18 is open, column 19 is open, column 20 is open, column 21 is outside the maze\nRow 21: column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze, column 20 is outside the maze, column 21 is outside the maze\nRow 22: column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze, column 20 is outside the maze, column 21 is outside the maze\nRow 23: column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze, column 20 is outside the maze, column 21 is outside the maze\nRow 24: column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze, column 20 is outside the maze, column 21 is outside the maze", "question": "The agent plans to move west, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "e776c452ce203a6c885e1418e44f8ecba0ba827936048915d7d7f77fb5b3b463"}835{"id": "first_move-000834", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,15), goal at (19,19).\n#####################\n#...#..........A#...#\n#.###.#########.###.#\n#.#...#.......#.....#\n#.#.###.###.#########\n#.#...#.#.#.....#...#\n#.###.#.#.###.#.#.###\n#.....#.....#.#.#...#\n#.###########.#.###.#\n#.#...........#...#.#\n#.#.#############.#.#\n#...#.....#.....#...#\n#####.#.###.#.#.###.#\n#.#...#.....#.#.#...#\n#.#.#########.#.#.###\n#.#...#.....#.#.#...#\n#.###.#.###.#.#####.#\n#...#.#.#...#.......#\n#.#.#.###.#.#########\n#.#.......#........G#\n#####################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "49a9deacf8e37db13501409cf665d51be4cb45dcfc011b01c3d446fac8bb4143"}836{"id": "open_count-000835", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,24); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX\n########X\n..#.A.##X\n#.###.##X\n#...#.##X\n###.#.##X\n#...#.##X", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 2, "answer": "2", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "113269e3a6b2395d1ea94cd2a3b0e2de283acdbdadfbb0e0ee6157d9ab1c1dd8"}837{"id": "snake_safe-000836", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (4,7), travelling north, length 3.\n........\n........\n........\n....*...\n.......H\n......oo\n........\n........", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "54532fa205a9c2568d3a8b48306d9b52eaa7f913a8762b4d8630adc97ec37e1f"}838{"id": "reachable_within-000837", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,11), goal at (21,13).\n################################\n#...#.....#...............#.#.##\n#.#.#.###.#A#####.#######.#.#.##\n#.#.#...#...#.....#...#...#.#.##\n#.#.###.#####.#######.#.###.#.##\n#.#...#.#.....#.....#.....#.#.##\n#.###.###.#####.###.#######.#.##\n#.#...#...#.....#.#.......#.#.##\n#.#.###.#.#.#####.#######.#.#.##\n#.#.....#.#.#...#.......#...#.##\n#.#########.###.#.#.#.#######.##\n#.#.......#.#...#.#.#.#.......##\n#.#.#####.#.#.#.#.#.###.###.####\n#...#.....#.#.#.#.#.....#.#...##\n#####.###.#.#.###.#######.###.##\n#.....#.#.#.#.............#...##\n#.#####.#.#.#.###########.#.#.##\n#.#.......#.#.#...#.....#.#.#.##\n#.#.#######.#.#.###.###.#.#.#.##\n#.#.#...#...#...#...#...#.#.#.##\n#.###.#.#.#####.#.###.###.#.####\n#.#...#.#.#..G#.#...#.#...#...##\n#.#.###.#.#.###.###.#.#.#####.##\n#...#.#.#.#.....#...#.#...#...##\n#.###.#.#.#######.###.#####.#.##\n#.#...#.#.......#...#...#...#.##\n#.#.###.#######.#.#.###.#.###.##\n#.#...#...#...#.#.#.#.#...#...##\n#.###.###.#.#.#.###.#.#####.####\n#.......#...#.......#.........##\n################################\n################################", "question": "Can the agent reach the goal in at most 228 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "cd7f44cf5e8fa658e13b42a078af402c9a97603510d596eb854e34186fd82e77"}839{"id": "blocked_pair-000838", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[8,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"wall\",\"6,10\":\"wall\",\"6,11\":\"wall\",\"7,7\":\"open\",\"7,8\":\"wall\",\"7,9\":\"open\",\"7,10\":\"wall\",\"7,11\":\"wall\",\"8,7\":\"wall\",\"8,8\":\"wall\",\"8,9\":\"agent\",\"8,10\":\"wall\",\"8,11\":\"wall\",\"9,7\":\"open\",\"9,8\":\"open\",\"9,9\":\"open\",\"9,10\":\"wall\",\"9,11\":\"wall\",\"10,7\":\"wall\",\"10,8\":\"wall\",\"10,9\":\"wall\",\"10,10\":\"wall\",\"10,11\":\"wall\"}}", "question": "Are BOTH the cell one step south and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "6c94adae3c66f8b693dc5c6998b26ea870206afb026e4264bf5659be682ff609"}840{"id": "stochastic-000839", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[24,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"20,-1\":\"outside\",\"20,0\":\"wall\",\"20,1\":\"open\",\"20,2\":\"open\",\"20,3\":\"open\",\"20,4\":\"wall\",\"20,5\":\"wall\",\"20,6\":\"wall\",\"20,7\":\"wall\",\"21,-1\":\"outside\",\"21,0\":\"wall\",\"21,1\":\"open\",\"21,2\":\"wall\",\"21,3\":\"open\",\"21,4\":\"wall\",\"21,5\":\"open\",\"21,6\":\"open\",\"21,7\":\"open\",\"22,-1\":\"outside\",\"22,0\":\"wall\",\"22,1\":\"open\",\"22,2\":\"open\",\"22,3\":\"open\",\"22,4\":\"wall\",\"22,5\":\"open\",\"22,6\":\"wall\",\"22,7\":\"open\",\"23,-1\":\"outside\",\"23,0\":\"wall\",\"23,1\":\"open\",\"23,2\":\"wall\",\"23,3\":\"open\",\"23,4\":\"wall\",\"23,5\":\"open\",\"23,6\":\"wall\",\"23,7\":\"open\",\"24,-1\":\"outside\",\"24,0\":\"wall\",\"24,1\":\"open\",\"24,2\":\"wall\",\"24,3\":\"agent\",\"24,4\":\"wall\",\"24,5\":\"open\",\"24,6\":\"wall\",\"24,7\":\"wall\",\"25,-1\":\"outside\",\"25,0\":\"wall\",\"25,1\":\"open\",\"25,2\":\"open\",\"25,3\":\"open\",\"25,4\":\"wall\",\"25,5\":\"open\",\"25,6\":\"open\",\"25,7\":\"open\",\"26,-1\":\"outside\",\"26,0\":\"wall\",\"26,1\":\"wall\",\"26,2\":\"wall\",\"26,3\":\"wall\",\"26,4\":\"wall\",\"26,5\":\"wall\",\"26,6\":\"wall\",\"26,7\":\"wall\",\"27,-1\":\"outside\",\"27,0\":\"wall\",\"27,1\":\"wall\",\"27,2\":\"wall\",\"27,3\":\"wall\",\"27,4\":\"wall\",\"27,5\":\"wall\",\"27,6\":\"wall\",\"27,7\":\"wall\",\"28,-1\":\"outside\",\"28,0\":\"outside\",\"28,1\":\"outside\",\"28,2\":\"outside\",\"28,3\":\"outside\",\"28,4\":\"outside\",\"28,5\":\"outside\",\"28,6\":\"outside\",\"28,7\":\"outside\"}}", "question": "The actuator executes the intended move with probability 0.85; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.09999999999999998, 0.9], "deterministic": false, "fingerprint": "72f02bfd2b57af646f11c9266708f4f0f1584173600254b56819ecd28bf88b36"}841{"id": "move_legal-000840", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[7,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"4,4\":\"wall\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"open\",\"4,8\":\"wall\",\"4,9\":\"open\",\"4,10\":\"wall\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"open\",\"5,9\":\"open\",\"5,10\":\"wall\",\"6,4\":\"wall\",\"6,5\":\"open\",\"6,6\":\"wall\",\"6,7\":\"wall\",\"6,8\":\"wall\",\"6,9\":\"wall\",\"6,10\":\"wall\",\"7,4\":\"open\",\"7,5\":\"open\",\"7,6\":\"wall\",\"7,7\":\"agent\",\"7,8\":\"open\",\"7,9\":\"open\",\"7,10\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"wall\",\"8,6\":\"wall\",\"8,7\":\"wall\",\"8,8\":\"wall\",\"8,9\":\"open\",\"8,10\":\"wall\",\"9,4\":\"open\",\"9,5\":\"open\",\"9,6\":\"open\",\"9,7\":\"open\",\"9,8\":\"open\",\"9,9\":\"open\",\"9,10\":\"wall\",\"10,4\":\"wall\",\"10,5\":\"wall\",\"10,6\":\"wall\",\"10,7\":\"wall\",\"10,8\":\"wall\",\"10,9\":\"wall\",\"10,10\":\"wall\"}}", "question": "The agent wants to move south. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "813d0796cae82e77f40b4d6cd01ba02c26873fd1be9a1ad3ff15df007d036b07"}842{"id": "cell_kind-000841", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[0,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-4,-3\":\"outside\",\"-4,-2\":\"outside\",\"-4,-1\":\"outside\",\"-4,0\":\"outside\",\"-4,1\":\"outside\",\"-4,2\":\"outside\",\"-4,3\":\"outside\",\"-4,4\":\"outside\",\"-4,5\":\"outside\",\"-3,-3\":\"outside\",\"-3,-2\":\"outside\",\"-3,-1\":\"outside\",\"-3,0\":\"outside\",\"-3,1\":\"outside\",\"-3,2\":\"outside\",\"-3,3\":\"outside\",\"-3,4\":\"outside\",\"-3,5\":\"outside\",\"-2,-3\":\"outside\",\"-2,-2\":\"outside\",\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-2,5\":\"outside\",\"-1,-3\":\"outside\",\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"0,-3\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"open\",\"0,1\":\"agent\",\"0,2\":\"open\",\"0,3\":\"open\",\"0,4\":\"wall\",\"0,5\":\"open\",\"1,-3\":\"outside\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"wall\",\"1,4\":\"open\",\"1,5\":\"open\",\"2,-3\":\"outside\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"open\",\"2,3\":\"open\",\"2,4\":\"open\",\"2,5\":\"open\",\"3,-3\":\"outside\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"open\",\"3,1\":\"wall\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"open\",\"4,-3\":\"outside\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"open\",\"4,1\":\"open\",\"4,2\":\"open\",\"4,3\":\"wall\",\"4,4\":\"open\",\"4,5\":\"wall\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "ea4351196aed052ed22d82ff4912c179c84a3c269339212aad3619fa5492efbf"}843{"id": "distance_band-000842", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,7), goal at (13,13).\n################\n#.#...#A......##\n#.###.#.###.####\n#.#...#...#...##\n#...#.#.#.###.##\n#.#.#...#.....##\n#.#.#.#...######\n#.....#.#.....##\n#.##.##.#.#.#.##\n#.....#.....#.##\n#.#.###.##.##.##\n#.#.........#.##\n#.###.###.#.#.##\n#.........#..G##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "464f5e4372e6bdd6506fe0b0f5e8ee5bcee227b29e839cc1b9c3226eca91c5f9"}844{"id": "distance_band-000843", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (15,13), goal at (11,3).\n########################\n#...#.............#...##\n#.###.#####.#####.#.#.##\n#.....#.....#.....#.#.##\n#.#####.#.#.#.#.#.###.##\n#.........#...#.#.....##\n###.#.###...###.#.#.####\n#.#.#.#.......#...#...##\n#.#.#.#.#####.###.#.#.##\n#...#.......#...#...#.##\n#.###...#.#####.##.##.##\n#..G#.#.#.............##\n#####.###.#.#.#######.##\n#...#.....#.#.......#.##\n#.#.#.##..#...#####.#.##\n#.#.#........A..#...#.##\n#...#####.#####.#.###.##\n#.#...#...#.....#...#.##\n#.###.#.###.#######.#.##\n#.#...#.#.......#...#.##\n#.#.#####.###.#.#.#.#.##\n#.............#.......##\n########################\n########################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 3, "answer": "20 to 39 moves", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "506876c5c8258576f2ca4f129585310668880768604afc78dccab04e2ab6c7c0"}845{"id": "first_move-000844", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (4,1), goal at (1,9).\n#####################\n#.#.....#G#.....#...#\n#.###.#.#.#.###.#.#.#\n#.#...#.#.#...#...#.#\n#A#.###.#.###.#####.#\n#...#...#.....#...#.#\n#####.#########.#.#.#\n#.....#...#...#.#...#\n#.#####.#.#.#.#.#####\n#.#.#...#...#.#.....#\n#.#.#.#.#####.#####.#\n#.#...#...#.#.#.....#\n#.#######.#.#.#.###.#\n#.......#.#.#.#.#.#.#\n#######.#.#.#.#.#.#.#\n#.....#...#.#...#...#\n#.#.#######.#####.###\n#.#...#...#...#...#.#\n#.###.#.#.#.#.#.###.#\n#...#...#...#.......#\n#####################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "498fbef7970c71d385754c148d9545b7ad3cc37dfc610bee0370ca83a35b0f21"}846{"id": "plan_survives-000845", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"1,6\":\"open\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"2,6\":\"wall\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"agent\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"open\",\"4,0\":\"wall\",\"4,1\":\"wall\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"open\",\"5,5\":\"open\",\"5,6\":\"wall\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"wall\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\"}}", "question": "The agent will attempt this plan in order: west, east, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "fbb3bb5a16b507b93ea115d89dc77885145a60e1b529961329a283f84db22968"}847{"id": "stochastic-000846", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[7,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"wall\",\"6,-2\":\"outside\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"7,-2\":\"outside\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"agent\",\"7,2\":\"open\",\"7,3\":\"open\",\"7,4\":\"open\",\"8,-2\":\"outside\",\"8,-1\":\"outside\",\"8,0\":\"wall\",\"8,1\":\"wall\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"8,4\":\"wall\",\"9,-2\":\"outside\",\"9,-1\":\"outside\",\"9,0\":\"outside\",\"9,1\":\"outside\",\"9,2\":\"outside\",\"9,3\":\"outside\",\"9,4\":\"outside\",\"10,-2\":\"outside\",\"10,-1\":\"outside\",\"10,0\":\"outside\",\"10,1\":\"outside\",\"10,2\":\"outside\",\"10,3\":\"outside\",\"10,4\":\"outside\"}}", "question": "The actuator executes the intended move with probability 0.90; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.06666666666666665, 0.9333333333333333], "deterministic": false, "fingerprint": "5f1f03ef1e4eb3f230df2bd2247717872e814def7cb078b63c8e916ba8f8eb09"}848{"id": "reachable_within-000847", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (19,18), goal at (13,9).\n########################\n#.......#.......#.....##\n#.#######.#####.#####.##\n#.#.....#.#...#.......##\n#.#.###.#.#.#########.##\n#...#...#.#.........#.##\n#.###.#.#.#.#####.###.##\n#.#.#.#.#.#.....#...#.##\n#.#.#.###.#####.###.#.##\n#...#...#.#...#...#...##\n#.#####.#.#.#####.######\n#.#.....#.#.....#.....##\n#.#.###.#.#.#.#.#####.##\n#.#.#...#G#.#.#.#.....##\n#.#.#.#####.#.###.######\n#.#.#.#...#.#...#.....##\n#.#.#.#.#.#.###.#####.##\n#.#.#.#.#...#...#.....##\n#.#.###.#####.###.######\n#.#...#...#.#...#.A...##\n#.###.###.#.###.#####.##\n#...#.........#.......##\n########################\n########################", "question": "Can the agent reach the goal in at most 80 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "f7c46941bb9e7fb800b9d9a76f7864853a59823df07823c1a81f67398762937d"}849{"id": "goal_bearing-000848", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (33,31), goal at (1,2).\n####################################\n#.G...#.......#.......#...........##\n#.###.#.#####.###.#####.#####.###.##\n#.......#.........#...#.#.....#...##\n#.#.###.##..#######.#.#...#######.##\n#.#.#...#...........#.#.#.......#.##\n#.#.##.##.#.#########.#.###.###.#.##\n#.#.....#.......#.....#.#...#...#.##\n#.#####.#######.#.###.#.#.#...######\n#...#.#.....#...#...#.#.#.#.#.....##\n###.#.#.#.###.#####.###...#.#####.##\n#.#...#.#.#.........#...#.#.....#.##\n#.###.#.#.#.###.###.#.###.###.###.##\n#...#...#...#.....#...#.#.....#...##\n#.#####.#######.#.#####.#.#.#.######\n#.....#.#...#...#.....#...#.#.....##\n#####.#.#.#.#.#.#.###.#.#.#.#####.##\n#.....#...#...#.........#.#.....#.##\n#.##.##.#######.#.#.#.#.#.#####.####\n#.............#...#.#.........#...##\n#.#########.#.##.##.####.#.##.###.##\n#.....#.....#.....#.........#...#.##\n#.###.#.###.#.#.###########.#.###.##\n#.....#...#.....#.......#...#.#...##\n#.#######.#####.#.###.#.#.#####.#.##\n#.#.............#...#.#.#.......#.##\n#.#.###.#.#.#.#.###.#.###.#.###.#.##\n#...#.#.#.#.........#...#.......#.##\n###.#.#.#.#.####.##.###.#####.#.#.##\n#.................#...#...........##\n#.#.#####.#.##.##.###.###.#.########\n#.........#.#...#.....#.#.#.....#.##\n#.#.#.#.###.#.#.#######.#.#.#.#.#.##\n#.#...#.......#.........#.....#A..##\n####################################\n####################################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "e5e5977ebb46d2ef9350e91971df437cf38f1981a47c1a1900d1289bb61aae16"}850{"id": "first_move-000849", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (4,6), goal at (20,0).\n####....##...###.#..#\n#####..#.#.#####.....\n#####..#...######..#.\n####......##.####....\n####..A#..#...###....\n####.#......##....#..\n###...#.##....#.....#\n##.##.......#.#.#.###\n##.#.#..##...##..#...\n.....#...#......#.#..\n..#...#.#..#.##......\n.....#.###.....#..#.#\n........###.....###..\n..........#.......#..\n..#.#.....####...#..#\n...#..#..###........#\n...#..##..###..#.....\n#..##..#.........#...\n#....##.............#\n##...##.#..##.....#.#\nG..#######....###....", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "17e4b0cb44b49cb26a4150bccbc4cf61c2774ba61fae9d637e49d0496a4c093f"}851{"id": "move_two_step-000850", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (35,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.......\n.....#.\n#......\n...A.#.\nXXXXXXX\nXXXXXXX\nXXXXXXX", "question": "The agent plans to move north, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "40cdd0da4386edc40cefbfb8b5e3ef5900a74fa51f3dbee99eacab486d92ddbb"}852{"id": "only_way-000851", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 3. Rows increase to the south and columns to the east.\nRow 4: column 0 is open, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open, column 6 is open\nRow 5: column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is open, column 5 is wall, column 6 is open\nRow 6: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is open\nRow 7: column 0 is open, column 1 is open, column 2 is open, column 3 is the agent, column 4 is wall, column 5 is open, column 6 is open\nRow 8: column 0 is open, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall\nRow 9: column 0 is open, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is wall, column 6 is wall\nRow 10: column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 5, "answer": "more than one of them", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "e8f339f8e11f11aba30cc2c62df7e4db2d60c722fcadd98d254be2ad16f8ff9c"}853{"id": "move_legal-000852", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 30, column 15. Rows increase to the south and columns to the east.\nRow 28: column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall\nRow 29: column 13 is open, column 14 is wall, column 15 is open, column 16 is open, column 17 is open\nRow 30: column 13 is open, column 14 is wall, column 15 is the agent, column 16 is wall, column 17 is wall\nRow 31: column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open\nRow 32: column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is open", "question": "If the agent attempts one cell south (row+1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "87b84c9f1724f952dc8c23f01e6867e63d6e1eb65d2598425b18da8a046a3d59"}854{"id": "snake_safe-000853", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,6), travelling south, length 3.\n............\n............\n............\n............\n............\n.....*......\n.....oo.....\n......H.....\n............\n............\n............\n............", "question": "If the snake moves west on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "01e081a98bff033667131b5404803c3110f54f10e0a6509a05a7d7f238e73b02"}855{"id": "reachable_within-000854", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,13), goal at (5,5).\n################\n#.........#.#.##\n#.#######.#.#.##\n#.#.....#.#.#.##\n#.#.###.#.#.#.##\n#.#.#G..#.#.#.##\n#.#.#####.#.#.##\n#.#.......#.#A##\n#.#########.#.##\n#...........#.##\n#.###########.##\n#.#...........##\n#.#########.#.##\n#...........#.##\n################\n################", "question": "Can the agent reach the goal in at most 61 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "5a2e54ae6c282049f06d37b5c0f305611135ea4c455f77d1c58ee089b806091d"}856{"id": "plan_survives-000855", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (22,23); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n##.#.\n.#...\n.#A..\n...#.\n.#.#.", "question": "The agent will attempt this plan in order: south, west, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "bfcdcaa73e84578aba04c88bc0ecaab7b647f191c75e66edf74bd2b1556829d2"}857{"id": "first_move-000856", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (10,3), goal at (19,17).\n#####################\n#.......#.......#...#\n#####.#.#.#####.#.#.#\n#.....#...#...#...#.#\n#.###.#####.#.#.###.#\n#.......#...#.......#\n#.###.#.#######.###.#\n#.#.....#.........#.#\n#.###...#.###..##.###\n#.....#.#.......#...#\n#.#A###.#.###.#.###.#\n#.#.#...#.#.......#.#\n#.#.#.###.#.#####.#.#\n#...#.........#.#.#.#\n###.#.#######.#.#.#.#\n#...#.........#.....#\n#.#.#.#############.#\n#...#.....#.....#...#\n#.#.#####.#####.#.#.#\n#.......#.......#G..#\n#####################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "4fcbb509dcde4442b706d16bacbf144a23c3770075a3a21ba27a03c26ed357d3"}858{"id": "open_count-000857", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[6,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"2,6\":\"wall\",\"2,7\":\"open\",\"2,8\":\"wall\",\"2,9\":\"wall\",\"2,10\":\"wall\",\"2,11\":\"wall\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"3,6\":\"open\",\"3,7\":\"open\",\"3,8\":\"open\",\"3,9\":\"open\",\"3,10\":\"open\",\"3,11\":\"open\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"wall\",\"4,8\":\"wall\",\"4,9\":\"wall\",\"4,10\":\"wall\",\"4,11\":\"wall\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"wall\",\"5,9\":\"open\",\"5,10\":\"open\",\"5,11\":\"open\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"open\",\"6,6\":\"wall\",\"6,7\":\"agent\",\"6,8\":\"wall\",\"6,9\":\"wall\",\"6,10\":\"wall\",\"6,11\":\"wall\",\"7,3\":\"open\",\"7,4\":\"wall\",\"7,5\":\"open\",\"7,6\":\"wall\",\"7,7\":\"open\",\"7,8\":\"open\",\"7,9\":\"open\",\"7,10\":\"open\",\"7,11\":\"open\",\"8,3\":\"open\",\"8,4\":\"wall\",\"8,5\":\"open\",\"8,6\":\"wall\",\"8,7\":\"wall\",\"8,8\":\"wall\",\"8,9\":\"wall\",\"8,10\":\"wall\",\"8,11\":\"wall\",\"9,3\":\"open\",\"9,4\":\"open\",\"9,5\":\"open\",\"9,6\":\"open\",\"9,7\":\"open\",\"9,8\":\"open\",\"9,9\":\"open\",\"9,10\":\"open\",\"9,11\":\"open\",\"10,3\":\"wall\",\"10,4\":\"wall\",\"10,5\":\"wall\",\"10,6\":\"wall\",\"10,7\":\"wall\",\"10,8\":\"wall\",\"10,9\":\"wall\",\"10,10\":\"wall\",\"10,11\":\"wall\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 2, "answer": "2", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "0e391ef099b585a09dd1bd35ec28b13714ff4643700ede68f40154eeb5a1f2d3"}859{"id": "only_way-000858", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 3, column 3. Rows increase to the south and columns to the east.\nRow 1: column 1 is open, column 2 is open, column 3 is open, column 4 is wall, column 5 is open\nRow 2: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 3: column 1 is open, column 2 is wall, column 3 is the agent, column 4 is wall, column 5 is open\nRow 4: column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open\nRow 5: column 1 is open, column 2 is open, column 3 is open, column 4 is wall, column 5 is open", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "d49dabe5887334b44a2d908557ff2492cbd239e3141aa9677627da586a6dc00c"}860{"id": "blocked_pair-000859", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[26,25],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"22,21\":\"open\",\"22,22\":\"open\",\"22,23\":\"wall\",\"22,24\":\"open\",\"22,25\":\"open\",\"22,26\":\"open\",\"22,27\":\"open\",\"22,28\":\"outside\",\"22,29\":\"outside\",\"23,21\":\"wall\",\"23,22\":\"wall\",\"23,23\":\"open\",\"23,24\":\"wall\",\"23,25\":\"open\",\"23,26\":\"open\",\"23,27\":\"open\",\"23,28\":\"outside\",\"23,29\":\"outside\",\"24,21\":\"wall\",\"24,22\":\"wall\",\"24,23\":\"open\",\"24,24\":\"wall\",\"24,25\":\"wall\",\"24,26\":\"wall\",\"24,27\":\"open\",\"24,28\":\"outside\",\"24,29\":\"outside\",\"25,21\":\"open\",\"25,22\":\"open\",\"25,23\":\"open\",\"25,24\":\"open\",\"25,25\":\"open\",\"25,26\":\"open\",\"25,27\":\"open\",\"25,28\":\"outside\",\"25,29\":\"outside\",\"26,21\":\"open\",\"26,22\":\"open\",\"26,23\":\"wall\",\"26,24\":\"open\",\"26,25\":\"agent\",\"26,26\":\"open\",\"26,27\":\"open\",\"26,28\":\"outside\",\"26,29\":\"outside\",\"27,21\":\"open\",\"27,22\":\"wall\",\"27,23\":\"wall\",\"27,24\":\"open\",\"27,25\":\"wall\",\"27,26\":\"open\",\"27,27\":\"wall\",\"27,28\":\"outside\",\"27,29\":\"outside\",\"28,21\":\"outside\",\"28,22\":\"outside\",\"28,23\":\"outside\",\"28,24\":\"outside\",\"28,25\":\"outside\",\"28,26\":\"outside\",\"28,27\":\"outside\",\"28,28\":\"outside\",\"28,29\":\"outside\",\"29,21\":\"outside\",\"29,22\":\"outside\",\"29,23\":\"outside\",\"29,24\":\"outside\",\"29,25\":\"outside\",\"29,26\":\"outside\",\"29,27\":\"outside\",\"29,28\":\"outside\",\"29,29\":\"outside\",\"30,21\":\"outside\",\"30,22\":\"outside\",\"30,23\":\"outside\",\"30,24\":\"outside\",\"30,25\":\"outside\",\"30,26\":\"outside\",\"30,27\":\"outside\",\"30,28\":\"outside\",\"30,29\":\"outside\"}}", "question": "Are BOTH the cell one step north and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "000566457e82d55e65ccbc8d9273785716fbcce6adcff793ab7f24320a064ebb"}861{"id": "cell_kind-000860", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (6,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n....#XX\n#.#.#XX\n..#.#XX\n#.#A#XX\n#...#XX\n#####XX\nXXXXXXX", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "2b6faf32f617013686eabc4f8fba8d2034f10cfb48adefd74b7f84e19f299e14"}862{"id": "move_legal-000861", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[27,27],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"25,25\":\"open\",\"25,26\":\"open\",\"25,27\":\"open\",\"25,28\":\"outside\",\"25,29\":\"outside\",\"26,25\":\"open\",\"26,26\":\"open\",\"26,27\":\"open\",\"26,28\":\"outside\",\"26,29\":\"outside\",\"27,25\":\"open\",\"27,26\":\"open\",\"27,27\":\"agent\",\"27,28\":\"outside\",\"27,29\":\"outside\",\"28,25\":\"outside\",\"28,26\":\"outside\",\"28,27\":\"outside\",\"28,28\":\"outside\",\"28,29\":\"outside\",\"29,25\":\"outside\",\"29,26\":\"outside\",\"29,27\":\"outside\",\"29,28\":\"outside\",\"29,29\":\"outside\"}}", "question": "If the agent attempts one cell east (col+1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "0241914018828ad0c7f5aa35ff32148f3917d2a6e9c4193f2c9d85b4135a7e07"}863{"id": "reachable_within-000862", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (0,3), goal at (15,15).\n...A.###########\n.##...##########\n.##..###########\n...#....########\n.#.##.##########\n.....###.#######\n.#..#.##.#######\n#.#.......######\n....#..#..######\n..#.#.#.#..####.\n....#......##...\n.##........##...\n.....#.#......#.\n...#.....##.#.#.\n.#..###.#.#.....\n#..#....#....#.G", "question": "Can the agent reach the goal in at most 34 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "bdc5c54b87adbac46b63029cc90bf1c50f3ffe99ee5f609e5175df2146f8f2fa"}864{"id": "blocked_pair-000863", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"-1,6\":\"outside\",\"-1,7\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"wall\",\"1,5\":\"open\",\"1,6\":\"open\",\"1,7\":\"open\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"open\",\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"wall\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"agent\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"open\",\"3,7\":\"open\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"open\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"open\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"6,7\":\"open\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"wall\",\"7,3\":\"open\",\"7,4\":\"open\",\"7,5\":\"open\",\"7,6\":\"open\",\"7,7\":\"open\"}}", "question": "Are BOTH the cell one step east and the cell one step west blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "9f45230d472170724d8a7a629022fa684a9d49c95649965cd7ab939e4ca06eeb"}865{"id": "only_way-000864", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,0); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXX##.\nXX#..\nXXA.#\nXX###\nXX###", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "2a6ef747757dae1e70f262a17a865e3fbb2a2ab7601f554feb92b09f03434627"}866{"id": "first_move-000865", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,9), goal at (13,1).\n################\n#...#...#.....##\n#.#.###.#.###.##\n#.#...#.#.#...##\n#.###.#...#.#.##\n#...#...#A..#.##\n###.#######.#.##\n#...........#.##\n#.###.###.######\n#...#...#.....##\n###.###.#####.##\n#...#.....#...##\n#.#####.###.#.##\n#G............##\n################\n################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "1201e3fec28c20872cd743f2bbef7735dfd80c1d73c7840e99c7161601a6f157"}867{"id": "blocked_pair-000866", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (4,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.....#\n#.###.#\n#.#.#.#\n#.#A#.#\n#.#.#.#\n#.#.#.#\n#...#..", "question": "Are BOTH the cell one step east and the cell one step west blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "af1f4b3d9ff4a9135c3d82d9e5232bf922c4875a94c86282870a58a27e53ca01"}868{"id": "goal_bearing-000867", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,7), goal at (7,7).\n############\n#......A..##\n#.##########\n#.........##\n#.##########\n#.#.......##\n#.#.#####.##\n#.#....G#.##\n#.#######.##\n#.........##\n############\n############", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 1, "answer": "south", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "6535efebbe691a2c2afb9bd82c57d856fbca8fdf8b9ec87e4f2311c9800f18e7"}869{"id": "onward-000868", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"open\",\"2,4\":\"wall\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"agent\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"wall\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"wall\",\"6,-2\":\"outside\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\"}}", "question": "Suppose the agent moves one cell north. From that new cell, would a further step north be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "e6b242edda8d8411af6476583669226151147928a40b7fd4243710683992c83c"}870{"id": "reachable_within-000869", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (15,9), goal at (5,9).\n########################\n#.............#.......##\n#.###########.#.#####.##\n#.#.........#.#.#...#.##\n#.#.#######.#.#.#.#.#.##\n#.#.#....G#.#.#.#.#.#.##\n#.#.#.#####.#.#.#.#.#.##\n#.#.#.......#.#.#.#.#.##\n#.#.#########.#.#.#.#.##\n#.#...........#.#.#.#.##\n#.#############.###.#.##\n#.#.................#.##\n#.#.#################.##\n#.#.#.......#.........##\n#.#.#.#####.############\n#.#.#.#..A#...........##\n#.#.#.#.#############.##\n#.#.#.#.............#.##\n#.#.#.#############.#.##\n#.#.#...............#.##\n#.#.#################.##\n#.....................##\n########################\n########################", "question": "Can the agent reach the goal in at most 161 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "3498ae3de07a119ea24e7c5f67d78759ee64b5b19a06819c4afe5a1f1d2459d7"}871{"id": "stochastic-000870", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[7,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"3,6\":\"open\",\"3,7\":\"open\",\"3,8\":\"open\",\"3,9\":\"open\",\"3,10\":\"open\",\"3,11\":\"open\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"wall\",\"4,8\":\"wall\",\"4,9\":\"wall\",\"4,10\":\"wall\",\"4,11\":\"open\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"wall\",\"5,9\":\"open\",\"5,10\":\"wall\",\"5,11\":\"open\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"open\",\"6,6\":\"wall\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"open\",\"6,10\":\"wall\",\"6,11\":\"open\",\"7,3\":\"open\",\"7,4\":\"wall\",\"7,5\":\"open\",\"7,6\":\"wall\",\"7,7\":\"agent\",\"7,8\":\"wall\",\"7,9\":\"open\",\"7,10\":\"wall\",\"7,11\":\"open\",\"8,3\":\"open\",\"8,4\":\"wall\",\"8,5\":\"open\",\"8,6\":\"wall\",\"8,7\":\"open\",\"8,8\":\"wall\",\"8,9\":\"open\",\"8,10\":\"wall\",\"8,11\":\"open\",\"9,3\":\"open\",\"9,4\":\"wall\",\"9,5\":\"open\",\"9,6\":\"wall\",\"9,7\":\"open\",\"9,8\":\"wall\",\"9,9\":\"open\",\"9,10\":\"wall\",\"9,11\":\"open\",\"10,3\":\"open\",\"10,4\":\"wall\",\"10,5\":\"open\",\"10,6\":\"wall\",\"10,7\":\"open\",\"10,8\":\"wall\",\"10,9\":\"open\",\"10,10\":\"wall\",\"10,11\":\"open\",\"11,3\":\"open\",\"11,4\":\"wall\",\"11,5\":\"open\",\"11,6\":\"wall\",\"11,7\":\"open\",\"11,8\":\"wall\",\"11,9\":\"open\",\"11,10\":\"open\",\"11,11\":\"open\"}}", "question": "The actuator executes the intended move with probability 0.70; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.20000000000000007, 0.7999999999999999], "deterministic": false, "fingerprint": "079875e575e0afc53de4fbd8ccdf5e944328bfb762fbc31ec3c26859d5806f3f"}872{"id": "blocked_pair-000871", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,4\":\"outside\",\"-2,5\":\"outside\",\"-2,6\":\"outside\",\"-2,7\":\"outside\",\"-2,8\":\"outside\",\"-2,9\":\"outside\",\"-2,10\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"-1,6\":\"outside\",\"-1,7\":\"outside\",\"-1,8\":\"outside\",\"-1,9\":\"outside\",\"-1,10\":\"outside\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"0,8\":\"wall\",\"0,9\":\"wall\",\"0,10\":\"wall\",\"1,4\":\"wall\",\"1,5\":\"open\",\"1,6\":\"open\",\"1,7\":\"agent\",\"1,8\":\"wall\",\"1,9\":\"open\",\"1,10\":\"open\",\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"wall\",\"2,8\":\"wall\",\"2,9\":\"open\",\"2,10\":\"wall\",\"3,4\":\"open\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"3,8\":\"open\",\"3,9\":\"open\",\"3,10\":\"open\",\"4,4\":\"open\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"wall\",\"4,8\":\"wall\",\"4,9\":\"open\",\"4,10\":\"wall\"}}", "question": "Are BOTH the cell one step south and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "a77af7e89d3d0e634b86addd1ebdd04af8eabd8c1f64fa81b71a126312b31608"}873{"id": "plan_survives-000872", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 3, column 9. Rows increase to the south and columns to the east.\nRow 1: column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is wall\nRow 2: column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall\nRow 3: column 7 is open, column 8 is open, column 9 is the agent, column 10 is wall, column 11 is wall\nRow 4: column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall\nRow 5: column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is wall", "question": "The agent will attempt this plan in order: north, west, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "2e74df12655594137e870dfdb4aa95fb477fcc66ad6c90c486bec1212bede08e"}874{"id": "first_move-000873", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,11), goal at (3,5).\n#####################\n#...#...............#\n#.#.#.#############.#\n#.#.#G#...........#.#\n#.###############.#.#\n#.................#.#\n#.###.###########.#.#\n#.#.#.#.........#.#.#\n#.#.#.#.#.#######.#.#\n#.#.#.#.#.........#.#\n#.#.#.#############.#\n#.#.#...............#\n#.#.#################\n#.#........A........#\n#.#.###############.#\n#.#.#...............#\n#.#.#################\n#.#.................#\n#.#################.#\n#...................#\n#####################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "33a71efaa2a5c30186f62b545399ee2c512020cf6827a45a68060bac11ba118a"}875{"id": "first_move-000874", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,1), goal at (5,7).\n############\n#.....#...##\n#.#.#.#.#.##\n#A..#...#.##\n#.#######.##\n#...#..G#.##\n#..##.###.##\n#...#...#.##\n#.#.#.#.#.##\n#.........##\n############\n############", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "145bd5fe33561485633fa8319b8e35d38c98f153a04702215a8fa9a0d787ecaf"}876{"id": "reachable_within-000875", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (28,19), goal at (1,3).\n################################\n#..G#.....#.............#.....##\n#.###.###.#.##.######.#.#.#.#.##\n#...#...#.#.............#.#.#.##\n###.###.#.#.#####.###.###.#.####\n#...#...#...#...#...#.#...#...##\n#.#.#.#####.#.#.###.###.#.#.#.##\n#.#.#...#...#.#.....#...#.#...##\n#.###.#.#.#.#.##.####.###.#.#.##\n#.#.......#...............#.#.##\n#.#.#.#####.###.#####.##.##.####\n#...#...#...#...#...#.....#...##\n#.###.#.#.###.###.#.#.###.#.#.##\n#.#...#.#.........#.#...#.....##\n###.###.###.##.#.##.###.#####.##\n#...#.................#...#.#.##\n#.#.#####.#.#####.#.#.#.#.#.#.##\n#.....#...#...........#...#...##\n#.#.#.#.#######.#.#####.########\n#...#.#.#...............#.....##\n###.#.#.#.#.#######.#...#.###.##\n#.....#.....#...#.....#.....#.##\n#.###.####.##.#.#.#####.###.#.##\n#.............#...#.....#.....##\n#.#####.#.#.#.#########.#.######\n#.#...#...#.#.#.......#.......##\n#.#.#.###.#.#.#.#.###.#######.##\n#.#.#.#...#.#.#.#.#.#.#.......##\n#.###.###.#.#.###.#A#.#.#.###.##\n#.......#...#.....#.....#.....##\n################################\n################################", "question": "Can the agent reach the goal in at most 74 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "c7a32e2664dc9d1a6f4ab2c67017e038224d09f2661ba66742dcd0bec88710c9"}877{"id": "blocked_pair-000876", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 5, column 13. Rows increase to the south and columns to the east.\nRow 3: column 11 is open, column 12 is wall, column 13 is open, column 14 is open, column 15 is open\nRow 4: column 11 is open, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is open\nRow 5: column 11 is open, column 12 is wall, column 13 is the agent, column 14 is open, column 15 is open\nRow 6: column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall\nRow 7: column 11 is open, column 12 is open, column 13 is open, column 14 is open, column 15 is open", "question": "Are BOTH the cell one step west and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "b56c403a02538afdce1f9e988eced8c94517f5b2ce6df5616b5610112213c662"}878{"id": "onward-000877", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[7,4],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"4,1\":\"wall\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"open\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"open\",\"5,5\":\"open\",\"5,6\":\"open\",\"5,7\":\"open\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"wall\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"6,7\":\"wall\",\"7,1\":\"open\",\"7,2\":\"open\",\"7,3\":\"open\",\"7,4\":\"agent\",\"7,5\":\"open\",\"7,6\":\"wall\",\"7,7\":\"open\",\"8,1\":\"wall\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"wall\",\"8,6\":\"wall\",\"8,7\":\"open\",\"9,1\":\"open\",\"9,2\":\"open\",\"9,3\":\"open\",\"9,4\":\"wall\",\"9,5\":\"open\",\"9,6\":\"open\",\"9,7\":\"open\",\"10,1\":\"open\",\"10,2\":\"wall\",\"10,3\":\"open\",\"10,4\":\"wall\",\"10,5\":\"open\",\"10,6\":\"wall\",\"10,7\":\"open\"}}", "question": "Suppose the agent moves one cell west. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "237c00fe3eaebf358f7a84cb9b274ea33a183fe5d8e4046fe02c779fcdcfa886"}879{"id": "first_move-000878", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,5), goal at (5,1).\n#########\n#....A..#\n#.#######\n#...#...#\n###.#.#.#\n#G#...#.#\n#.#####.#\n#.......#\n#########", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "a5431af7aae04c9b14ba98a7b97ba8d772c1414d993382600cdad58e1a7f5ebd"}880{"id": "distance_band-000879", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,1), goal at (5,11).\n################\n#.......#.....##\n#.###.#.#####.##\n#.#...#.#.....##\n###.###.#.###.##\n#A..#...#.#G#.##\n#.###.###.#.#.##\n#.#...#...#...##\n###.#####.######\n#.#.#...#.....##\n#.#.#.#.###.#.##\n#.#...#...#.#.##\n#.#######.###.##\n#.............##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 4, "answer": "40 moves or more", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "981939f0439e0d345d6f9b147363cc3ff202fe87c50a941c1823c7bc404631ee"}881{"id": "move_legal-000880", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 5, column 3. Rows increase to the south and columns to the east.\nRow 1: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open\nRow 2: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 3: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open\nRow 4: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open\nRow 5: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is the agent, column 4 is open, column 5 is open, column 6 is open, column 7 is open\nRow 6: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 7: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is open\nRow 8: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open\nRow 9: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open", "question": "Is the cell one step west of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "b0e5b833ffd1cce8d1a8674b16302251aadd9b99d99fd5a150c727ee09441abf"}882{"id": "move_legal-000881", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (3,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\n######XXX\n.....#XXX\n.###.#XXX\n...#A#XXX\n##.###XXX\n.#...#XXX\n####.#XXX\n.....#XXX", "question": "If the agent attempts one cell south (row+1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "5f5af54bceec77a5c0ca6aa505cd03a62fbcefee8e1fad60a99f6670868c2fd5"}883{"id": "stochastic-000882", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 3, column 9. Rows increase to the south and columns to the east.\nRow 1: column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall\nRow 2: column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall\nRow 3: column 7 is open, column 8 is wall, column 9 is the agent, column 10 is wall, column 11 is wall\nRow 4: column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall\nRow 5: column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall", "question": "The actuator executes the intended move with probability 0.70; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.7999999999999999, 0.20000000000000004], "deterministic": false, "fingerprint": "11328ec96e35c8e340473a7fd29338abdcc314e4e3fffd3e6c6b7627cf00493f"}884{"id": "stochastic-000883", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[15,11],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"11,7\":\"open\",\"11,8\":\"open\",\"11,9\":\"open\",\"11,10\":\"open\",\"11,11\":\"open\",\"11,12\":\"open\",\"11,13\":\"open\",\"11,14\":\"open\",\"11,15\":\"open\",\"12,7\":\"wall\",\"12,8\":\"wall\",\"12,9\":\"wall\",\"12,10\":\"wall\",\"12,11\":\"wall\",\"12,12\":\"wall\",\"12,13\":\"wall\",\"12,14\":\"wall\",\"12,15\":\"wall\",\"13,7\":\"open\",\"13,8\":\"open\",\"13,9\":\"open\",\"13,10\":\"open\",\"13,11\":\"open\",\"13,12\":\"open\",\"13,13\":\"open\",\"13,14\":\"open\",\"13,15\":\"open\",\"14,7\":\"wall\",\"14,8\":\"wall\",\"14,9\":\"wall\",\"14,10\":\"wall\",\"14,11\":\"wall\",\"14,12\":\"wall\",\"14,13\":\"wall\",\"14,14\":\"wall\",\"14,15\":\"wall\",\"15,7\":\"open\",\"15,8\":\"open\",\"15,9\":\"open\",\"15,10\":\"open\",\"15,11\":\"agent\",\"15,12\":\"open\",\"15,13\":\"open\",\"15,14\":\"open\",\"15,15\":\"open\",\"16,7\":\"wall\",\"16,8\":\"wall\",\"16,9\":\"wall\",\"16,10\":\"wall\",\"16,11\":\"wall\",\"16,12\":\"wall\",\"16,13\":\"wall\",\"16,14\":\"wall\",\"16,15\":\"wall\",\"17,7\":\"open\",\"17,8\":\"open\",\"17,9\":\"open\",\"17,10\":\"open\",\"17,11\":\"open\",\"17,12\":\"open\",\"17,13\":\"open\",\"17,14\":\"open\",\"17,15\":\"open\",\"18,7\":\"wall\",\"18,8\":\"wall\",\"18,9\":\"wall\",\"18,10\":\"wall\",\"18,11\":\"wall\",\"18,12\":\"wall\",\"18,13\":\"wall\",\"18,14\":\"wall\",\"18,15\":\"wall\",\"19,7\":\"open\",\"19,8\":\"open\",\"19,9\":\"open\",\"19,10\":\"open\",\"19,11\":\"open\",\"19,12\":\"open\",\"19,13\":\"open\",\"19,14\":\"open\",\"19,15\":\"open\"}}", "question": "The actuator executes the intended move with probability 0.75; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.8333333333333334, 0.16666666666666666], "deterministic": false, "fingerprint": "4bbb259c898c9b0f98754a4ebc2651a5726f454879684d5f98a3d6b1fb775cc5"}885{"id": "first_move-000884", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (21,33), goal at (9,25).\n####################################\n#.....#...........................##\n#.###.#.#####.###################.##\n#.#...#.#...#.#...#.............#.##\n#.#####.#.#.#.#.#.#.###########.#.##\n#.......#.#.#.#.#.#.#.........#.#.##\n#.#######.#.#.#.#.#.#.#######.#.#.##\n#.#.......#.#.#.#.#.#.#.....#.#.#.##\n#.#.#######.#.#.#.#.#.#.###.#.#.#.##\n#.#.#.....#.#.#.#.#.#.#.#G#.#.#.#.##\n#.#.#.###.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#...#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#####.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.......#.#.#.#.#.#.#...#.#.#.#.##\n#.#########.#.#.#.#.#.#####.#.#.#.##\n#...........#.#.#.#.#.......#.#.#.##\n#############.#.###.#########.#.#.##\n#.............#...............#.#.##\n###############################.#.##\n#...............................#.##\n#.#############################.#.##\n#.#...........................#.#A##\n#.#.#########################.#.#.##\n#.#.#.........................#.#.##\n#.#.###########################.#.##\n#.#...........................#.#.##\n#.#.#########################.#.#.##\n#.#.........................#...#.##\n#.#########################.#####.##\n#.#.........................#.....##\n#.###########################.######\n#.............................#...##\n#.###############################.##\n#.................................##\n####################################\n####################################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "0c090ad721b6792ee05b6158d69027fa86bc092ee39a1b1658f30d82731bb0db"}886{"id": "goal_bearing-000885", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,15), goal at (1,11).\n#####################\n#..........G#..A#...#\n#.#.#########.#.###.#\n#.#...#.....#.#...#.#\n#####.#.###.#####.#.#\n#.....#.#.....#.#...#\n#.###.#.#.###.#.###.#\n#...#.#.#.#.......#.#\n###.###.#.#########.#\n#...#...#.#.........#\n#.#.#.###.#.#########\n#.#.#...#.#.......#.#\n#.###.###.#######.#.#\n#...#.#...#.....#...#\n###.#.#.###.#.#####.#\n#...#.#.....#.#.....#\n#.###.#######.#.#####\n#.....#.......#.#...#\n#.#####.#######.###.#\n#.....#.............#\n#####################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "2656c2e45d557a5b63b79bc0c33a9df4e6d6a62bed20f55b954b7fdd8e89440b"}887{"id": "move_two_step-000886", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 19. Rows increase to the south and columns to the east.\nRow -3: column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze, column 20 is outside the maze, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow -2: column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze, column 20 is outside the maze, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow -1: column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze, column 20 is outside the maze, column 21 is outside the maze, column 22 is outside the maze, column 23 is outside the maze\nRow 0: column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is wall, column 22 is wall, column 23 is wall\nRow 1: column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is the agent, column 20 is wall, column 21 is open, column 22 is open, column 23 is open\nRow 2: column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is open, column 22 is wall, column 23 is wall\nRow 3: column 15 is open, column 16 is wall, column 17 is open, column 18 is open, column 19 is open, column 20 is open, column 21 is open, column 22 is open, column 23 is open\nRow 4: column 15 is open, column 16 is wall, column 17 is open, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is open, column 22 is wall, column 23 is wall\nRow 5: column 15 is open, column 16 is open, column 17 is open, column 18 is wall, column 19 is open, column 20 is open, column 21 is open, column 22 is wall, column 23 is open", "question": "The agent plans to move north, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "89a2d0989f3dd0a27a94c33741fb6b7d0be9dd9b823bfb57ac98e56c7e508303"}888{"id": "snake_food-000887", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,8), travelling north, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n........H.....\n........o.....\n........o.....\n*.............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 7, "answer": "south-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "655943bf166539dc9e170c0d4ce88c38f607ae65cbe108bbc393955051e94797"}889{"id": "blocked_pair-000888", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 21, column 19. Rows increase to the south and columns to the east.\nRow 19: column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is open\nRow 20: column 17 is open, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is open\nRow 21: column 17 is open, column 18 is wall, column 19 is the agent, column 20 is wall, column 21 is open\nRow 22: column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is open\nRow 23: column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is open", "question": "Are BOTH the cell one step north and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "3d7e5cd2ce05feef3b798a769176dc164e84f2dbd4b2f06b8b682823aa302807"}890{"id": "plan_survives-000889", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 14, column 21. Rows increase to the south and columns to the east.\nRow 11: column 18 is open, column 19 is open, column 20 is open, column 21 is open, column 22 is open, column 23 is open, column 24 is open\nRow 12: column 18 is wall, column 19 is open, column 20 is wall, column 21 is wall, column 22 is wall, column 23 is open, column 24 is wall\nRow 13: column 18 is wall, column 19 is open, column 20 is wall, column 21 is open, column 22 is wall, column 23 is open, column 24 is wall\nRow 14: column 18 is wall, column 19 is wall, column 20 is wall, column 21 is the agent, column 22 is wall, column 23 is open, column 24 is wall\nRow 15: column 18 is open, column 19 is open, column 20 is wall, column 21 is open, column 22 is wall, column 23 is open, column 24 is wall\nRow 16: column 18 is wall, column 19 is open, column 20 is wall, column 21 is open, column 22 is wall, column 23 is open, column 24 is wall\nRow 17: column 18 is wall, column 19 is open, column 20 is wall, column 21 is open, column 22 is wall, column 23 is open, column 24 is wall", "question": "The agent will attempt this plan in order: south, north, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3338e751d00cb1ac3e9426095feb6a6bb950ff3da0e1bff6722a91f5cad700d9"}891{"id": "blocked_pair-000890", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (9,4); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#...#...#\n#.#.###.#\n#.#...#..\n#.###.#.#\n#.#.A.#..\n#########\n#########\nXXXXXXXXX\nXXXXXXXXX", "question": "Are BOTH the cell one step east and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "e0f76b278b2ccd7bef72f1c696b9fdac1599696fc88fc4f39a7616e1a33d3b32"}892{"id": "snake_food-000891", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (4,6), travelling north, length 3.\n............\n............\n............\n............\n......H..*..\n......o.....\n......o.....\n............\n............\n............\n............\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "c45229c2a6a3521f5b7d309e2ccafc3a6402c30b1bcccf6311029addea47d74d"}893{"id": "first_move-000892", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,13), goal at (9,19).\n############################\n#.........................##\n#.#######################.##\n#.......................#.##\n#########################.##\n#.........................##\n#.#######################.##\n#.#..........A............##\n#.#.########################\n#.#.#.............#G......##\n#.#.#.###########.#######.##\n#.#.#.#.....#...........#.##\n#.#.#.#.###.#.###########.##\n#.#.#.#...#.#...........#.##\n#.#.#.#####.###########.#.##\n#.#.#.#.......#.......#.#.##\n#.#.#.#.#######.#####.#.#.##\n#.#.#.#.............#.#.#.##\n#.#.#.###############.#.#.##\n#.#.#.................#.#.##\n#.#.###################.#.##\n#.#.....................#.##\n#.#.#####################.##\n#.#.#.....................##\n#.###.######################\n#.........................##\n############################\n############################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "4800a2c51f73f640f9dbe87cdc6275b13cd6d22373ce15e20326da3084b554ae"}894{"id": "stochastic-000893", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (8,21); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#.#.#.#.\n.#.#.#.#.\n.#.#.#.#.\n.#.#.#.#.\n.#.#A#.#.\n.#.#.#.#.\n.#.###.#.\n.#.#...#.\n.#.#.#.#.", "question": "The actuator executes the intended move with probability 0.80; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.1333333333333333, 0.8666666666666667], "deterministic": false, "fingerprint": "8bf62729ffd7ebc8082d14c75f554fc782cedb2069b70b14f5b4480f36c59adf"}895{"id": "snake_food-000894", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,4), travelling south, length 3.\n..........\n..........\n..........\n..........\n....oo....\n....H....*\n..........\n..........\n..........\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "60e67bf742b6637e2d4790a574e9c8a917636878b8ceb40228aa8841200d579d"}896{"id": "snake_safe-000895", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,7), travelling north, length 3.\n..............\n..............\n..............\n..............\n..............\n.......H......\n.......o......\n.......o......\n..............\n..............\n..............\n..............\n..............\n............*.", "question": "If the snake moves north on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "f9cce3ad4421993a5215a5c4337f5d4d7a6a89f70ebb9b8764120c6170eb0608"}897{"id": "goal_bearing-000896", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,5), goal at (3,5).\n#########\n#.......#\n#.#####.#\n#.#..G#.#\n#.#.###.#\n#.#.#A#.#\n#.#.#.#.#\n#...#...#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "5bae3359767033e914888467782445bc11d28b9cce79283901cf31c2600c80b7"}898{"id": "move_two_step-000897", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 8. Rows increase to the south and columns to the east.\nRow 5: column 6 is open, column 7 is open, column 8 is open, column 9 is outside the maze, column 10 is outside the maze\nRow 6: column 6 is wall, column 7 is open, column 8 is open, column 9 is outside the maze, column 10 is outside the maze\nRow 7: column 6 is open, column 7 is open, column 8 is the agent, column 9 is outside the maze, column 10 is outside the maze\nRow 8: column 6 is wall, column 7 is open, column 8 is open, column 9 is outside the maze, column 10 is outside the maze\nRow 9: column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze", "question": "The agent plans to move south, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "36cf7dcc5c55fa575ba05902fedc912dbc64624901a9083f61facfb2a443ad7e"}899{"id": "plan_survives-000898", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 19, column 29. Rows increase to the south and columns to the east.\nRow 15: column 25 is open, column 26 is open, column 27 is open, column 28 is wall, column 29 is open, column 30 is open, column 31 is open, column 32 is wall, column 33 is open\nRow 16: column 25 is wall, column 26 is wall, column 27 is open, column 28 is wall, column 29 is wall, column 30 is wall, column 31 is open, column 32 is wall, column 33 is open\nRow 17: column 25 is open, column 26 is wall, column 27 is open, column 28 is open, column 29 is open, column 30 is wall, column 31 is open, column 32 is open, column 33 is open\nRow 18: column 25 is open, column 26 is wall, column 27 is wall, column 28 is wall, column 29 is open, column 30 is wall, column 31 is wall, column 32 is wall, column 33 is open\nRow 19: column 25 is open, column 26 is wall, column 27 is open, column 28 is wall, column 29 is the agent, column 30 is wall, column 31 is open, column 32 is open, column 33 is open\nRow 20: column 25 is open, column 26 is wall, column 27 is open, column 28 is wall, column 29 is wall, column 30 is wall, column 31 is open, column 32 is wall, column 33 is wall\nRow 21: column 25 is open, column 26 is open, column 27 is open, column 28 is open, column 29 is open, column 30 is wall, column 31 is open, column 32 is wall, column 33 is open\nRow 22: column 25 is open, column 26 is wall, column 27 is wall, column 28 is wall, column 29 is wall, column 30 is wall, column 31 is open, column 32 is wall, column 33 is open\nRow 23: column 25 is open, column 26 is wall, column 27 is open, column 28 is open, column 29 is open, column 30 is open, column 31 is open, column 32 is wall, column 33 is open", "question": "The agent will attempt this plan in order: north, east, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "6cf15606dbb657148dd79f7359dc5e8d8b9b9247bea300a23e043f9686f2654c"}900{"id": "move_two_step-000899", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 1. Rows increase to the south and columns to the east.\nRow -3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow -2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow -1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 0: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is wall, column 3 is wall, column 4 is open, column 5 is open\nRow 1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is the agent, column 2 is open, column 3 is open, column 4 is open, column 5 is wall\nRow 2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 4: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is wall, column 5 is open\nRow 5: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is wall", "question": "The agent plans to move north, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "1d00cdf2d6feb43ec868acb835ec9b0159e09ea03570e50772eac8f68674197b"}901{"id": "stochastic-000900", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[2,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"wall\",\"1,3\":\"open\",\"1,4\":\"open\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"agent\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"open\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\",\"5,4\":\"open\"}}", "question": "The actuator executes the intended move with probability 0.75; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.16666666666666663, 0.8333333333333334], "deterministic": false, "fingerprint": "8a5dbcaa224bf15ac440d83d8d904e687c5cdd138acb1c6ed9e3c93ed6c2aada"}902{"id": "cell_kind-000901", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,21); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\n#########\n.........\n.#.###.##\n.#.#A#.#.\n.###.###.\n...#.....\n##.######\n...#.#...", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b5a05e45dfb8edc4a2154ad823ae3fc65999a0f478f39177b6d6ea1d4b26e893"}903{"id": "snake_safe-000902", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,6), travelling east, length 3.\n............\n............\n............\n............\n............\n............\n....ooH.....\n............\n............\n............\n..........*.\n............", "question": "If the snake moves west on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "36af36fd51f50528b3399429450140dca3f438864e8efe3b213917c4810ba3e9"}904{"id": "move_two_step-000903", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (5,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.#.#XX\n..#.#XX\n#.###XX\n#..A#XX\n###.#XX\n....#XX\n#####XX", "question": "The agent plans to move west, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "03393f6e0f77ff2250acebb7991e03da1f00e28c2b95fb79d97b48fc279a2115"}905{"id": "distance_band-000904", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,4), goal at (0,0).\nG...####........\n..#..##.#.......\n#.......#.##....\n.........#.#....\n..#......#......\n#####...#.......\n..#.#..##......#\n....A.#.........\n##...#..#...#...\n......#.........\n.......##.....#.\n#..........###..\n..#....#..##..#.\n...#......###...\n...##.#.........\n..#.............", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "d9f7de666c3ff2c25da6fc17254e760596fef8998d4cc1c062608c8d4157916a"}906{"id": "move_legal-000905", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[33,34],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"30,31\":\"open\",\"30,32\":\"open\",\"30,33\":\"wall\",\"30,34\":\"wall\",\"30,35\":\"open\",\"30,36\":\"outside\",\"30,37\":\"outside\",\"31,31\":\"open\",\"31,32\":\"wall\",\"31,33\":\"open\",\"31,34\":\"open\",\"31,35\":\"open\",\"31,36\":\"outside\",\"31,37\":\"outside\",\"32,31\":\"open\",\"32,32\":\"open\",\"32,33\":\"open\",\"32,34\":\"wall\",\"32,35\":\"wall\",\"32,36\":\"outside\",\"32,37\":\"outside\",\"33,31\":\"open\",\"33,32\":\"open\",\"33,33\":\"open\",\"33,34\":\"agent\",\"33,35\":\"open\",\"33,36\":\"outside\",\"33,37\":\"outside\",\"34,31\":\"wall\",\"34,32\":\"open\",\"34,33\":\"open\",\"34,34\":\"open\",\"34,35\":\"open\",\"34,36\":\"outside\",\"34,37\":\"outside\",\"35,31\":\"open\",\"35,32\":\"open\",\"35,33\":\"open\",\"35,34\":\"wall\",\"35,35\":\"open\",\"35,36\":\"outside\",\"35,37\":\"outside\",\"36,31\":\"outside\",\"36,32\":\"outside\",\"36,33\":\"outside\",\"36,34\":\"outside\",\"36,35\":\"outside\",\"36,36\":\"outside\",\"36,37\":\"outside\"}}", "question": "If the agent attempts one cell west (col-1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "561afd0711491d5eff3a123e1d686a155a661d954143d407ba29d2c6e4e64692"}907{"id": "distance_band-000906", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (11,2), goal at (15,4).\n..#.##.#.......#\n#............#.#\n..............#.\n...#.#..........\n...#...#........\n......#..#...#..\n.#.#....#.##....\n##...#..........\n.#...#..#.......\n....#.......#...\n##...##.#...#.#.\n..A........#....\n......#..##.....\n.###....#......#\n.#.##.#...#....#\n....G#..#......#", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "09a75ab298b91115eb7d6e98a04a6861e0f10333bd1f9f085cca9ffc2002245d"}908{"id": "stochastic-000907", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 0, column 34. Rows increase to the south and columns to the east.\nRow -4: column 30 is outside the maze, column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze\nRow -3: column 30 is outside the maze, column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze\nRow -2: column 30 is outside the maze, column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze\nRow -1: column 30 is outside the maze, column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze\nRow 0: column 30 is open, column 31 is open, column 32 is open, column 33 is open, column 34 is the agent, column 35 is open, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze\nRow 1: column 30 is open, column 31 is open, column 32 is wall, column 33 is open, column 34 is open, column 35 is open, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze\nRow 2: column 30 is open, column 31 is wall, column 32 is open, column 33 is wall, column 34 is open, column 35 is open, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze\nRow 3: column 30 is open, column 31 is open, column 32 is open, column 33 is wall, column 34 is open, column 35 is open, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze\nRow 4: column 30 is open, column 31 is open, column 32 is wall, column 33 is open, column 34 is open, column 35 is wall, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze", "question": "The actuator executes the intended move with probability 0.80; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.06666666666666665, 0.9333333333333333], "deterministic": false, "fingerprint": "720adc71252705b42525cf9a5d25c4d87ba4d2a40f16db95bec5bc3308a8300d"}909{"id": "only_way-000908", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (0,0); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXX\nXXXXXXX\nXXXXXXX\nXXXA...\nXXX##.#\nXXX###.\nXXX##..", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "eec57aed57a477140aceec10b2107de2efbc42f57e49a1720eeb33a41c7af0e0"}910{"id": "first_move-000909", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,19), goal at (9,1).\n#####################\n#.......#.....#...#A#\n#.#####.#####.#.#.#.#\n#.#...#.#...#.#.#...#\n#.#.###.#.#.#.#.#####\n#.#...#...#.#.#.....#\n#.###.#####.#.#.###.#\n#.........#.#.#...#.#\n#######.###.#.#####.#\n#G#.....#...#.......#\n#.#.#####.#.#######.#\n#.#.#.....#.......#.#\n#.#.#.#######.#####.#\n#...#.#...#...#.....#\n#####.#.###.###.#####\n#.....#...#.#.#.....#\n#.#####.#.#.#.#####.#\n#.#...#.#.....#.....#\n#.#.#.#########.#####\n#...#...............#\n#####################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "310a7025a4a279316eaf1c6a74ac658ad5d7629f54ca9cb4be6b33ce774487d7"}911{"id": "snake_safe-000910", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (4,4), travelling east, length 3.\n.....*..\n........\n........\n........\n...oH...\n...o....\n........\n........", "question": "If the snake moves north on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "39a22176602fcaf57edd664ca4f09c654243c50b946856dddb1a658d285205e4"}912{"id": "first_move-000911", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,13), goal at (19,19).\n#####################\n#.......#...#.....#.#\n#.#####.#.#.###.#.#.#\n#.....#...#..A#.#...#\n#.###.#######.#.###.#\n#...#.#.........#...#\n###.#.###########.###\n#...#.....#...#.#...#\n#.#######.#.###.###.#\n#.#...#...#.#.....#.#\n#.#.#.#.###.###.###.#\n#...#.#...#...#.#...#\n#####.###.###.#.#.###\n#...#.#.#...#.#.....#\n#.#.#.#.###.#.#####.#\n#.#.#.....#.#.#...#.#\n#.#.###.###.#.#.#.#.#\n#.#...#.#...#...#.#.#\n#.#####.#.#######.#.#\n#.......#.........#G#\n#####################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "b2f8b1674bcec030692c9815ce2368f8952cea6e6d0f2266475061941806cd0f"}913{"id": "only_way-000912", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 19, column 7. Rows increase to the south and columns to the east.\nRow 15: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is open, column 9 is open, column 10 is open, column 11 is open\nRow 16: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall\nRow 17: column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open\nRow 18: column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open\nRow 19: column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is the agent, column 8 is wall, column 9 is open, column 10 is open, column 11 is open\nRow 20: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall\nRow 21: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 22: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 23: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "24a4e52307814cc81270241a20eef03552dad6af737e2c29afed264494dc8264"}914{"id": "first_move-000913", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,10), goal at (10,0).\n#....###..##\n....#.....A#\n#.....#..#.#\n........####\n.#..#......#\n.##........#\n...#.###..##\n..#........#\n.......#.#..\n###....#....\nG..#..##....\n#.....#..###", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "bc8cb90b111d8dd77706e242f3b8626775b8aeaa748f6d9950e28c36947d0fdb"}915{"id": "cell_kind-000914", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (15,15); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.#####\n#.....#\n#####.#\n#..A#.#\n#.###.#\n#.....#\n#######", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "40ca42a69602af2b9f892cf49d71d75980e321d2e656145499aa055843452278"}916{"id": "snake_safe-000915", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (2,3), travelling west, length 3.\n........\n.....*..\n...Ho...\n....o...\n........\n........\n........\n........", "question": "If the snake moves east on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "bef42c70b6a5f2c4aad5062337f255f6b799a95e81abe5e30bbaa5ebe7cd2dc7"}917{"id": "goal_bearing-000916", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (8,9), goal at (9,3).\n################\n#.............##\n#.###########.##\n#.#.........#.##\n#.#.#.#####.#.##\n#.#.#.#...#.#.##\n#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.##\n#.###.#.#A#.#.##\n#..G#.#.#.#.#.##\n#####.#.###.#.##\n#.....#.....#.##\n#.###.#######.##\n#...#.........##\n################\n################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 7, "answer": "south-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "8bbcfbdbcf5481bc9c6b4281ce6bfdebc309514634a960f824b9367cf5774be0"}918{"id": "distance_band-000917", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 40x40. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,8), goal at (1,13).\n########################################\n#............G#.......................##\n#.####...######.#.#.###.#.#.#.###.###.##\n#...............#.#.....#...#...#.#...##\n#.####.########.#.#.#.#.#.###.#.#...####\n#...#...A.....#.#.#.#.......#.........##\n###.#.###.###.#.#.#...#.#.#...#####.#.##\n#.#.#.........#.#.#.#...#.#.#...#.....##\n#...###.#.#.#.###.#.#.#####...#...###.##\n#.#.#...#.#.......#...#...#...#.#.....##\n#...#.###.#..####.#.###.#.#.#.#.##.##.##\n#.....#.............#...#.............##\n#.###.#.#.#.######..#.###.#..##.###.#.##\n#...#.........#.....#...#.....#.#...#.##\n#.#.#...#.#####.###.#.#######.#.#.#...##\n#...#.....#.......#.#.#.......#...#.#.##\n#.#.#...#.#.#.#.#.#.#.#.#...#.#.###.#.##\n#.#.#.......#.....#...#.#.#.#...#.#...##\n#...#.##.######.#.###.#.#.#.#.#.#.#.####\n#.#.......#...#.#...#...........#.....##\n#.#########.#.#.###...#.#.#.#.#.#####.##\n#.#.........#...#.............#...#...##\n#.#.#######.#.###.#.#.##.##.#.###.#.####\n#.......#...#.....#...#...#...#...#...##\n#.###.#.#.###.#####.###.#.#.###.###.#.##\n#.....#.......#...#.....#...........#.##\n#.#####...#.#.###.#.#####.....#######.##\n#.#...........#.....#.....#.#...#.....##\n#...#.#.###.###.###.#.#.###.###.#.###.##\n#.....#.......#...#.........#...#.....##\n#.#.#.###.###.###.###.#.#.#.#.###.#.####\n#...#...#...#.....#...#...#.#...#.....##\n#...###.#.#.#.#####.#####.#.###.#.###.##\n#.#.#.....#.......#...#.......#.#.#.#.##\n#.#.##.#..#.#.###.#.#.#.#####.#...#.#.##\n#...........#.#.............#.#.#.#...##\n#.#.#####.#.#.#.#.###.###.#.#.#.#.#.####\n#...........#...............#.....#...##\n########################################\n########################################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "8124a3390818c86d768c995d280637d27cf2fbbb522cc836e8cc4c05ad6ce452"}919{"id": "blocked_pair-000918", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 9, column 4. Rows increase to the south and columns to the east.\nRow 5: column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is open\nRow 6: column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall\nRow 7: column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall\nRow 8: column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall\nRow 9: column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is the agent, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 10: column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall\nRow 11: column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall\nRow 12: column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall\nRow 13: column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is open", "question": "Are BOTH the cell one step south and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "550c66395a1e55a6e84ca5eaa4c364550dc5e7ea980afa53bb458ba8231a1d5a"}920{"id": "cell_kind-000919", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 5, column 7. Rows increase to the south and columns to the east.\nRow 2: column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall\nRow 3: column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is open, column 9 is open, column 10 is wall\nRow 4: column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open, column 10 is wall\nRow 5: column 4 is wall, column 5 is open, column 6 is wall, column 7 is the agent, column 8 is wall, column 9 is open, column 10 is wall\nRow 6: column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall\nRow 7: column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall\nRow 8: column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "068be9751fabef8372d038a0ce5a917cb1948e2fecb3cc5b893ae8cd8d850b20"}921{"id": "stochastic-000920", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 11, column 3. Rows increase to the south and columns to the east.\nRow 7: column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open, column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open\nRow 8: column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open\nRow 9: column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is wall, column 6 is open, column 7 is open\nRow 10: column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open, column 6 is open, column 7 is wall\nRow 11: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is open, column 3 is the agent, column 4 is open, column 5 is wall, column 6 is open, column 7 is open\nRow 12: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is open, column 3 is open, column 4 is open, column 5 is wall, column 6 is wall, column 7 is open\nRow 13: column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is wall, column 6 is open, column 7 is open\nRow 14: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is open, column 3 is open, column 4 is open, column 5 is wall, column 6 is wall, column 7 is open\nRow 15: column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open, column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall", "question": "The actuator executes the intended move with probability 0.70; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.09999999999999998, 0.9], "deterministic": false, "fingerprint": "64d5ef04256bfa3546a039c7cae8fdd1a60483ec0f3e58dcd74a5ccf594a4376"}922{"id": "distance_band-000921", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (4,10), goal at (2,15).\n######...#..#...\n####...#...##...\n####..#.#.#...#G\n#####...#....#.#\n####......A.....\n###.........#..#\n####..#.....#.##\n####...###.#.###\n######.#.....#.#\n######....#.....\n#####..#...#...#\n####..#...##.#..\n###.........#.##\n####..#..##.....\n####...#..#.#...\n######.#..#....#", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 1, "answer": "5 to 9 moves", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "52f72e44da847a2af9178e1f1966f1b91e9242d99f2046c12bde13a604faa749"}923{"id": "goal_bearing-000922", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,33), goal at (33,1).\n####################################\n#.........#.......#.........#.....##\n#.#####.#.#.#.#.#.#.#.##.##.#.###.##\n#...#...........#.........#.......##\n###...#.#.#.#.########.##.#.#.#.#.##\n#...#.............#.....#.....#.#.##\n#.#.######.######.#.###.##.##.#.#.##\n#.#...............#.........#...#A##\n###.#.#.###.#######.##..#.#.#.###.##\n#...#...#.......#.#.#.....#.....#.##\n#.#######.#.#.#.#.#.#.#...####.##.##\n#.........#...#.#...#.#.#.......#.##\n###.##..###.###.#.#.#.#####.#.#.#.##\n#.#.........#.#.......#.....#.#.#.##\n#.#.###.#.#.#...#.#.###.#.#.#.#.#.##\n#...#.........#...........#.#.#...##\n###.#.###.#.#.#.#.###.###...#.######\n#...#.......#...#.........#.#...#.##\n#.#.##.######.#.####..#.#.###.#.#.##\n#...#.......#...........#.....#...##\n#.###.##.##.#.#.#####.#.#.###.###.##\n#.#...#...#...#.#.....#.#...#.....##\n#.#.#.#.#.#######.#.#.#.#.#.##.##.##\n#.#.....#...#...#...#.#.....#.....##\n#.#######.#.#.#.#.###...#.###.###.##\n#.#.......#...#...#...#.#.........##\n#.#.#.#####.#.###.#.###.##..###.####\n#.....#.......#.........#.....#...##\n#####...#.#.#.#.###.#.###.#.#.###.##\n#.........#.#.......#.#.#...#.#...##\n#.#####.#.#.###.#.###.#.#.#.....####\n#.#...#...#.....#.....#...#...#...##\n#.#.#.###.#####.#.##..#####.#.#.#.##\n#G............#.............#.....##\n####################################\n####################################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 7, "answer": "south-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "69ec02ec752835dc3796dec78ccb2dff86ed4dc6eddf99fadef970386db0b729"}924{"id": "move_legal-000923", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (3,4); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.....\n#####\n#.A..\n#####\n#....", "question": "If the agent attempts one cell east (col+1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "d79465e64c4e5aae3adf54e5106848191e057b2a876e961608139a2165108375"}925{"id": "move_legal-000924", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[15,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"11,-3\":\"outside\",\"11,-2\":\"outside\",\"11,-1\":\"outside\",\"11,0\":\"open\",\"11,1\":\"open\",\"11,2\":\"open\",\"11,3\":\"open\",\"11,4\":\"open\",\"11,5\":\"wall\",\"12,-3\":\"outside\",\"12,-2\":\"outside\",\"12,-1\":\"outside\",\"12,0\":\"open\",\"12,1\":\"open\",\"12,2\":\"open\",\"12,3\":\"open\",\"12,4\":\"open\",\"12,5\":\"open\",\"13,-3\":\"outside\",\"13,-2\":\"outside\",\"13,-1\":\"outside\",\"13,0\":\"wall\",\"13,1\":\"wall\",\"13,2\":\"open\",\"13,3\":\"open\",\"13,4\":\"open\",\"13,5\":\"open\",\"14,-3\":\"outside\",\"14,-2\":\"outside\",\"14,-1\":\"outside\",\"14,0\":\"open\",\"14,1\":\"open\",\"14,2\":\"open\",\"14,3\":\"open\",\"14,4\":\"open\",\"14,5\":\"open\",\"15,-3\":\"outside\",\"15,-2\":\"outside\",\"15,-1\":\"outside\",\"15,0\":\"open\",\"15,1\":\"agent\",\"15,2\":\"open\",\"15,3\":\"open\",\"15,4\":\"open\",\"15,5\":\"open\",\"16,-3\":\"outside\",\"16,-2\":\"outside\",\"16,-1\":\"outside\",\"16,0\":\"outside\",\"16,1\":\"outside\",\"16,2\":\"outside\",\"16,3\":\"outside\",\"16,4\":\"outside\",\"16,5\":\"outside\",\"17,-3\":\"outside\",\"17,-2\":\"outside\",\"17,-1\":\"outside\",\"17,0\":\"outside\",\"17,1\":\"outside\",\"17,2\":\"outside\",\"17,3\":\"outside\",\"17,4\":\"outside\",\"17,5\":\"outside\",\"18,-3\":\"outside\",\"18,-2\":\"outside\",\"18,-1\":\"outside\",\"18,0\":\"outside\",\"18,1\":\"outside\",\"18,2\":\"outside\",\"18,3\":\"outside\",\"18,4\":\"outside\",\"18,5\":\"outside\",\"19,-3\":\"outside\",\"19,-2\":\"outside\",\"19,-1\":\"outside\",\"19,0\":\"outside\",\"19,1\":\"outside\",\"19,2\":\"outside\",\"19,3\":\"outside\",\"19,4\":\"outside\",\"19,5\":\"outside\"}}", "question": "If the agent attempts one cell east (col+1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "a236164c18c7bf17c22bcfaacd9cf252d5d95503c8d7cb953a673a6ec304e17e"}926{"id": "reachable_within-000925", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,3), goal at (9,5).\n################\n#.#...........##\n#.#.###.#####.##\n#.#...#.....#.##\n#.###.#####.####\n#.....#...#...##\n#.#####.#.###.##\n#.......#.#...##\n###.#######.#.##\n#..A#G....#.#.##\n#.#######.#.####\n#...#...#.#...##\n###.#.#.#.###.##\n#.....#.#.....##\n################\n################", "question": "Can the agent reach the goal in at most 53 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "437d6138fe36ab29ee2ade8bd496952e6deda8954ad16d0b4e2592b7b45dc4e4"}927{"id": "first_move-000926", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (8,1), goal at (1,9).\n############\n#.......#G##\n#.#####.#.##\n#.....#.#.##\n#.###.#.#.##\n#...#.#.#.##\n#####.#.#.##\n#.....#.#.##\n#A#####.#.##\n#.#.......##\n############\n############", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "07669955191d95dfa0c8b4b6e39b7ca789a8bb357f1898cb15b44f7bb0152a0a"}928{"id": "only_way-000927", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (5,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.#####\n#......\n#######\n...A#..\n#####.#\n....#..\n###.###", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "40cb141a0992b069921395dbea7153e392b06705744f8b7927d18c7268a1ae3e"}929{"id": "move_legal-000928", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (0,23); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXX\nXXXXXXX\nXXXXXXX\n##.A...\n##.#...\n##...#.\n#...#..", "question": "If the agent attempts one cell east (col+1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "c2f1acd52a9e75a19c3b2234cd431a0f250ce15ce831a1dee94994fd34a05701"}930{"id": "distance_band-000929", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,9), goal at (13,13).\n################\n#.....#.......##\n#.#####.###.#.##\n#.#.........#.##\n#.###.###.###.##\n#.....#...#...##\n#.#####.###.#.##\n#.....#.......##\n#.###.#.#####.##\n#.............##\n###.####.####.##\n#.............##\n#.###.##########\n#........A...G##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "7d002dd51e204ae1724019d522cc7be2875478d4636f9ca2949b5d79156fb48e"}931{"id": "move_two_step-000930", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXX#####\nXX#....\nXX#.#.#\nXX#A#.#\nXX###.#\nXX#...#\nXX#.###", "question": "The agent plans to move east, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "86d6664abf8452e8f95516fc91766298df4cc90a771f41565d989c4d40c62b31"}932{"id": "snake_food-000931", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,11), travelling east, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n.........ooH..\n..............\n..............\n..............\n..............\n.......*......", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 7, "answer": "south-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "fc7ab531c202f7cb175485cd4f80830ca23bae76e4ac0abe2329b8ca77d78294"}933{"id": "move_two_step-000932", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (0,19); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX\n....A#XXX\n.#..##XXX\n....##XXX\n..####XXX\n....#.XXX", "question": "The agent plans to move east, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "fcbb51a4eb93c8444e1d5ee6194bea2790acacf105f67f7531eabc55f9db4fdd"}934{"id": "plan_survives-000933", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXX\nX#..#\nX#A..\nX#...\nX..#.", "question": "The agent will attempt this plan in order: east, east, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "ac6cbff6c031100910354dfe70a2406d6ebf860a422cabdc6d4f14176b3c10db"}935{"id": "plan_survives-000934", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (7,2); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX###.##\nX#...#.\nX#.###.\nX#.A.#.\nX#####.\nX#.....\nX#.####", "question": "The agent will attempt this plan in order: west, north, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "6d1f9be3d6003b6a5097286596f18d5684e5753c1c84d46678c808f69a5b5057"}936{"id": "move_legal-000935", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[1,13],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-3,9\":\"outside\",\"-3,10\":\"outside\",\"-3,11\":\"outside\",\"-3,12\":\"outside\",\"-3,13\":\"outside\",\"-3,14\":\"outside\",\"-3,15\":\"outside\",\"-3,16\":\"outside\",\"-3,17\":\"outside\",\"-2,9\":\"outside\",\"-2,10\":\"outside\",\"-2,11\":\"outside\",\"-2,12\":\"outside\",\"-2,13\":\"outside\",\"-2,14\":\"outside\",\"-2,15\":\"outside\",\"-2,16\":\"outside\",\"-2,17\":\"outside\",\"-1,9\":\"outside\",\"-1,10\":\"outside\",\"-1,11\":\"outside\",\"-1,12\":\"outside\",\"-1,13\":\"outside\",\"-1,14\":\"outside\",\"-1,15\":\"outside\",\"-1,16\":\"outside\",\"-1,17\":\"outside\",\"0,9\":\"wall\",\"0,10\":\"wall\",\"0,11\":\"wall\",\"0,12\":\"wall\",\"0,13\":\"wall\",\"0,14\":\"wall\",\"0,15\":\"wall\",\"0,16\":\"wall\",\"0,17\":\"wall\",\"1,9\":\"open\",\"1,10\":\"open\",\"1,11\":\"open\",\"1,12\":\"wall\",\"1,13\":\"agent\",\"1,14\":\"open\",\"1,15\":\"open\",\"1,16\":\"open\",\"1,17\":\"open\",\"2,9\":\"wall\",\"2,10\":\"wall\",\"2,11\":\"open\",\"2,12\":\"wall\",\"2,13\":\"wall\",\"2,14\":\"wall\",\"2,15\":\"wall\",\"2,16\":\"wall\",\"2,17\":\"wall\",\"3,9\":\"open\",\"3,10\":\"wall\",\"3,11\":\"open\",\"3,12\":\"open\",\"3,13\":\"open\",\"3,14\":\"open\",\"3,15\":\"open\",\"3,16\":\"open\",\"3,17\":\"open\",\"4,9\":\"open\",\"4,10\":\"wall\",\"4,11\":\"wall\",\"4,12\":\"wall\",\"4,13\":\"wall\",\"4,14\":\"wall\",\"4,15\":\"wall\",\"4,16\":\"wall\",\"4,17\":\"open\",\"5,9\":\"open\",\"5,10\":\"open\",\"5,11\":\"open\",\"5,12\":\"wall\",\"5,13\":\"open\",\"5,14\":\"open\",\"5,15\":\"open\",\"5,16\":\"wall\",\"5,17\":\"open\"}}", "question": "Is the cell one step north of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "9326b9d037b9638a92641da8a69a3c22c1ca87b92c27814876e143666f253eb1"}937{"id": "onward-000936", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 7. Rows increase to the south and columns to the east.\nRow -3: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow -2: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow -1: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 0: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 1: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is the agent, column 8 is open, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 2: column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 3: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 4: column 3 is open, column 4 is wall, column 5 is wall, column 6 is open, column 7 is open, column 8 is open, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 5: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze", "question": "Suppose the agent moves one cell east. From that new cell, would a further step north be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "f5c0528a65876836e5f39a2462934828abd756ef2f216e1d0c19b2cabcaaee5f"}938{"id": "onward-000937", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[8,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"open\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"wall\",\"5,4\":\"open\",\"6,-2\":\"outside\",\"6,-1\":\"outside\",\"6,0\":\"open\",\"6,1\":\"open\",\"6,2\":\"open\",\"6,3\":\"wall\",\"6,4\":\"wall\",\"7,-2\":\"outside\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"wall\",\"7,3\":\"open\",\"7,4\":\"open\",\"8,-2\":\"outside\",\"8,-1\":\"outside\",\"8,0\":\"wall\",\"8,1\":\"agent\",\"8,2\":\"open\",\"8,3\":\"wall\",\"8,4\":\"open\",\"9,-2\":\"outside\",\"9,-1\":\"outside\",\"9,0\":\"outside\",\"9,1\":\"outside\",\"9,2\":\"outside\",\"9,3\":\"outside\",\"9,4\":\"outside\",\"10,-2\":\"outside\",\"10,-1\":\"outside\",\"10,0\":\"outside\",\"10,1\":\"outside\",\"10,2\":\"outside\",\"10,3\":\"outside\",\"10,4\":\"outside\",\"11,-2\":\"outside\",\"11,-1\":\"outside\",\"11,0\":\"outside\",\"11,1\":\"outside\",\"11,2\":\"outside\",\"11,3\":\"outside\",\"11,4\":\"outside\"}}", "question": "Suppose the agent moves one cell north. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "2611a820b3ccc912bf81c9b3dfadcd6d10c2d9816b58d67becc0f8aa6f56f72a"}939{"id": "plan_survives-000938", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[2,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"agent\",\"2,4\":\"wall\",\"2,5\":\"open\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"open\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\"}}", "question": "The agent will attempt this plan in order: north, west, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "c814f228c121bf93715ffbdd9fbb3e96eb15d57a032487db7b1d060ad6965854"}940{"id": "first_move-000939", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,19), goal at (17,19).\n#####################\n#.#.................#\n#.#.#####.#.#######A#\n#...#.....#...#...#.#\n###.#.#.#.###.#.#.#.#\n#.......#.#...#.#...#\n#.#...###.#.###.#.#.#\n#.#.#.#...#...#...#.#\n#.#.#...###.#.#.....#\n#.#.#.......#...#.#.#\n#.#.#.##.#####.##.#.#\n#.#...#...........#.#\n#.###.#.#.#.###.#.#.#\n#.....#.#.......#...#\n#.###.#.#.#####.###.#\n#.......#.......#...#\n###.#.#...#.#####.###\n#...#...#.#.......#G#\n#.###.###.#########.#\n#...................#\n#####################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "3d331297441c3d9c4eaa980055833ff05e658c52ed777e674408e35c95f28767"}941{"id": "snake_safe-000940", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (3,4), travelling north, length 3.\n.*......\n........\n........\n....H...\n....o...\n....o...\n........\n........", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "337ee3010c505ef6526cf38933855a540886b3c1f80bda79b4312f614b7c5e10"}942{"id": "plan_survives-000941", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[13,6],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"9,2\":\"open\",\"9,3\":\"open\",\"9,4\":\"wall\",\"9,5\":\"open\",\"9,6\":\"open\",\"9,7\":\"open\",\"9,8\":\"open\",\"9,9\":\"open\",\"9,10\":\"wall\",\"10,2\":\"wall\",\"10,3\":\"wall\",\"10,4\":\"wall\",\"10,5\":\"open\",\"10,6\":\"wall\",\"10,7\":\"wall\",\"10,8\":\"wall\",\"10,9\":\"wall\",\"10,10\":\"wall\",\"11,2\":\"wall\",\"11,3\":\"open\",\"11,4\":\"open\",\"11,5\":\"open\",\"11,6\":\"wall\",\"11,7\":\"open\",\"11,8\":\"open\",\"11,9\":\"open\",\"11,10\":\"open\",\"12,2\":\"wall\",\"12,3\":\"open\",\"12,4\":\"wall\",\"12,5\":\"wall\",\"12,6\":\"wall\",\"12,7\":\"open\",\"12,8\":\"wall\",\"12,9\":\"wall\",\"12,10\":\"wall\",\"13,2\":\"open\",\"13,3\":\"open\",\"13,4\":\"wall\",\"13,5\":\"open\",\"13,6\":\"agent\",\"13,7\":\"open\",\"13,8\":\"open\",\"13,9\":\"open\",\"13,10\":\"wall\",\"14,2\":\"wall\",\"14,3\":\"wall\",\"14,4\":\"wall\",\"14,5\":\"wall\",\"14,6\":\"wall\",\"14,7\":\"wall\",\"14,8\":\"wall\",\"14,9\":\"wall\",\"14,10\":\"wall\",\"15,2\":\"wall\",\"15,3\":\"wall\",\"15,4\":\"wall\",\"15,5\":\"wall\",\"15,6\":\"wall\",\"15,7\":\"wall\",\"15,8\":\"wall\",\"15,9\":\"wall\",\"15,10\":\"wall\",\"16,2\":\"outside\",\"16,3\":\"outside\",\"16,4\":\"outside\",\"16,5\":\"outside\",\"16,6\":\"outside\",\"16,7\":\"outside\",\"16,8\":\"outside\",\"16,9\":\"outside\",\"16,10\":\"outside\",\"17,2\":\"outside\",\"17,3\":\"outside\",\"17,4\":\"outside\",\"17,5\":\"outside\",\"17,6\":\"outside\",\"17,7\":\"outside\",\"17,8\":\"outside\",\"17,9\":\"outside\",\"17,10\":\"outside\"}}", "question": "The agent will attempt this plan in order: east, south, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "aa52f8a47a3ce06b7986be2dcb610645d4142f8c14574ade6fc4e2c32dd94ba4"}943{"id": "open_count-000942", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\nX########\nX#.......\nX#.######\nX#.#A#...\nX#.#.#.##\nX#...#.#.\nX#####.##\nX#...#...", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 1, "answer": "1", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "8dcc3734f4ab8e0e27c088370c5eb741257cc9526dde2e8beb0161a7aa15b86d"}944{"id": "move_legal-000943", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 5, column 5. Rows increase to the south and columns to the east.\nRow 3: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open\nRow 4: column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 5: column 3 is open, column 4 is wall, column 5 is the agent, column 6 is wall, column 7 is open\nRow 6: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open\nRow 7: column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open", "question": "The agent wants to move north. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "2d699c1fc6ae5aeb3098fa0b75a105661f483d3384a4ab4e491b509c7f0f7fc8"}945{"id": "snake_food-000944", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (1,5), travelling north, length 3.\n..........\n.....H....\n....oo....\n........*.\n..........\n..........\n..........\n..........\n..........\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 6, "answer": "south-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "86787a6c7043bc0644dce493aea36e7b754806c4297eee0db4df7e24a97988ed"}946{"id": "blocked_pair-000945", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 2, column 1. Rows increase to the south and columns to the east.\nRow -2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow -1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 0: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open\nRow 3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open\nRow 4: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 5: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 6: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open", "question": "Are BOTH the cell one step south and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "a542a5aa0efd169c40f6b6c7ef5bd6d18acf169df7548ece5a072481f505d65f"}947{"id": "goal_bearing-000946", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,19), goal at (19,19).\n#####################\n#.....#.............#\n#.##..#.#.#.#.##.##.#\n#.......#.....#...#.#\n#.####.##.#####.#.#.#\n#.#.....#.....#.....#\n#.#####.##.##.###.###\n#...#.......#...#..A#\n###.#.#.#.#####.#.#.#\n#.......#...#...#...#\n#.#####.#.#.#.#.#####\n#...#...#.#.#.......#\n#.#.#.###...#.###.#.#\n#.#.#.#...#.........#\n#.#.#.#.#.#.#..##.#.#\n#.#...#...#.....#...#\n#.#####.#####.#.#.#.#\n#.#...........#...#.#\n#.##..#.#.###.#.#.#.#\n#.........#........G#\n#####################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 1, "answer": "south", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "af66682cc34dd117428f79716fa0b6a01c8dae328d4779967ec6df41e882e50f"}948{"id": "distance_band-000947", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,5), goal at (11,1).\n################\n#.........#...##\n#.#.###.#.#.#.##\n#.#...#.#...#.##\n#.#.#.#.#.###.##\n#.#.#...#.....##\n###.#.###.#..###\n#...#.#.#.#...##\n#.#.###.#.#.####\n#...#...#.....##\n###.#.###.###.##\n#G#...#.......##\n#.#####.###.#.##\n#....A......#.##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 1, "answer": "5 to 9 moves", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "20f65f365459d68953fb1304f9fc74470552e25ebeb8615f8a5a996eea08550a"}949{"id": "cell_kind-000948", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (5,0); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXX####\nXXX####\nXXX.###\nXXXA...\nXXX.#..\nXXX##..\nXXX....", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "a968148f420ed344c5b93bf6ffaa2b8db7ff8fa23a62c7d8b92d9cd880098f2c"}950{"id": "cell_kind-000949", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,9); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n######X\n....##X\n###.##X\n#..A##X\n######X\n....##X\n###.##X", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "978d76672c0db4b23091dc81b2897ec35ee2d13f9e7bff9d063350365e382078"}951{"id": "cell_kind-000950", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 3, column 29. Rows increase to the south and columns to the east.\nRow 0: column 26 is wall, column 27 is wall, column 28 is wall, column 29 is wall, column 30 is wall, column 31 is wall, column 32 is wall\nRow 1: column 26 is wall, column 27 is open, column 28 is open, column 29 is open, column 30 is open, column 31 is open, column 32 is open\nRow 2: column 26 is wall, column 27 is open, column 28 is wall, column 29 is wall, column 30 is wall, column 31 is open, column 32 is wall\nRow 3: column 26 is wall, column 27 is open, column 28 is open, column 29 is the agent, column 30 is wall, column 31 is open, column 32 is open\nRow 4: column 26 is wall, column 27 is wall, column 28 is wall, column 29 is wall, column 30 is wall, column 31 is open, column 32 is wall\nRow 5: column 26 is open, column 27 is open, column 28 is open, column 29 is open, column 30 is wall, column 31 is open, column 32 is open\nRow 6: column 26 is wall, column 27 is wall, column 28 is wall, column 29 is open, column 30 is wall, column 31 is wall, column 32 is wall", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "2d0dcc402b3fa29552ba6fe43b3be40af22fbf1cd4fd76bf16d1bf64123d61f9"}952{"id": "move_two_step-000951", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 5. Rows increase to the south and columns to the east.\nRow 4: column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 5: column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 6: column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 7: column 2 is wall, column 3 is open, column 4 is wall, column 5 is the agent, column 6 is wall, column 7 is open, column 8 is wall\nRow 8: column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 9: column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 10: column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall", "question": "The agent plans to move east, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "6a344aa882be8f752942597b63ef1ed97def83116c244ed22ede159e99a1938e"}953{"id": "first_move-000952", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,21), goal at (23,15).\n####################################\n#.....#...#.........#.......#.....##\n#.#####.#.#.#######.#.#.###.#####.##\n#.....#.#.#...#...#...#.#.#.....#.##\n#.###.#.#.###.#.#.#####.#.#####.#.##\n#.#.#...#...#.#.#...#...#.......#.##\n#.#.#####.###.#.###.#.###.#######.##\n#...#...#.....#...#.#.#...#.......##\n#####.#.#########.#.#.#.#####.###.##\n#.....#.........#.#..A#.....#...#.##\n#.#.###########.#.#########.#.###.##\n#.#.#...#.......#...#.....#.#.#...##\n#.###.#.#.#####.###.#.#.###.###.####\n#.#...#.#.#...#...#.#.#...#...#...##\n#.#.###.###.#.###.#.#####.###.###.##\n#...#.#.#...#...#.#.#.....#.#.....##\n#.###.#.#.#####.#.#.#.###.#.#####.##\n#...#.#.....#...#...#.#.....#...#.##\n###.#.#######.#######.#######.#.#.##\n#...#...#...#.......#...#.....#...##\n#.###.###.#.#######.#.#.#.##########\n#.#...#...#.........#.#.........#.##\n#.###.#.#####.#######.#########.#.##\n#.....#.#.#...#G#.......#...#.#.#.##\n#######.#.#.###.#########.#.#.#.#.##\n#.......#.....#.#.....#...#.#...#.##\n#.#############.#.###.#.###.###.#.##\n#.#.....#.....#...#.#...#.#...#...##\n#.#.###.#.###.#####.#####.###.###.##\n#...#...#...#.......#.......#.#.#.##\n#.###.#.###.###.###.#.###.###.#.#.##\n#.#...#.#...#.#.#...#.#.#...#.#...##\n#.#.#####.###.#.#####.#.###.#.#.####\n#.#...........#...........#...#...##\n####################################\n####################################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "047e5969445b4df05c30fd23397527909ec47253342afc2f783f5040a1602ae9"}954{"id": "stochastic-000953", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (5,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX#...\nX#.#.\nX#A#.\nX#.##\nX#.#.", "question": "The actuator executes the intended move with probability 0.90; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9333333333333333, 0.06666666666666665], "deterministic": false, "fingerprint": "a930849c07a8832c28bcbcd088eeacefa4b3cf8be407de0ec39c4664fabf0e17"}955{"id": "move_two_step-000954", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,33); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n...##\n.#.##\n.#A##\n##.##\n.#.##", "question": "The agent plans to move south, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "b9b52ee1ceb2a1e871125eb572b3ecc24fb306895b0d40fdd87ee806464a4123"}956{"id": "cell_kind-000955", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,20],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,17\":\"outside\",\"-2,18\":\"outside\",\"-2,19\":\"outside\",\"-2,20\":\"outside\",\"-2,21\":\"outside\",\"-2,22\":\"outside\",\"-2,23\":\"outside\",\"-1,17\":\"outside\",\"-1,18\":\"outside\",\"-1,19\":\"outside\",\"-1,20\":\"outside\",\"-1,21\":\"outside\",\"-1,22\":\"outside\",\"-1,23\":\"outside\",\"0,17\":\"wall\",\"0,18\":\"open\",\"0,19\":\"open\",\"0,20\":\"open\",\"0,21\":\"outside\",\"0,22\":\"outside\",\"0,23\":\"outside\",\"1,17\":\"open\",\"1,18\":\"open\",\"1,19\":\"open\",\"1,20\":\"agent\",\"1,21\":\"outside\",\"1,22\":\"outside\",\"1,23\":\"outside\",\"2,17\":\"open\",\"2,18\":\"open\",\"2,19\":\"open\",\"2,20\":\"open\",\"2,21\":\"outside\",\"2,22\":\"outside\",\"2,23\":\"outside\",\"3,17\":\"open\",\"3,18\":\"wall\",\"3,19\":\"open\",\"3,20\":\"open\",\"3,21\":\"outside\",\"3,22\":\"outside\",\"3,23\":\"outside\",\"4,17\":\"open\",\"4,18\":\"open\",\"4,19\":\"wall\",\"4,20\":\"open\",\"4,21\":\"outside\",\"4,22\":\"outside\",\"4,23\":\"outside\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "69ea595e4e59f1e79a017a2a7d12deda430c57a1bcb2ca37c28becc1536ab67a"}957{"id": "blocked_pair-000956", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[5,5],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"open\",\"2,8\":\"wall\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"3,6\":\"open\",\"3,7\":\"open\",\"3,8\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"open\",\"4,8\":\"wall\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"agent\",\"5,6\":\"open\",\"5,7\":\"open\",\"5,8\":\"wall\",\"6,2\":\"wall\",\"6,3\":\"wall\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"6,7\":\"wall\",\"6,8\":\"wall\",\"7,2\":\"open\",\"7,3\":\"open\",\"7,4\":\"open\",\"7,5\":\"open\",\"7,6\":\"open\",\"7,7\":\"open\",\"7,8\":\"wall\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"wall\",\"8,6\":\"wall\",\"8,7\":\"open\",\"8,8\":\"wall\"}}", "question": "Are BOTH the cell one step west and the cell one step south blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "03f386c16f6da35e407d172974145ba336e3758a1eb3cdf05c86ba97e0ef36a9"}958{"id": "open_count-000957", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,5\":\"outside\",\"-1,6\":\"outside\",\"-1,7\":\"outside\",\"-1,8\":\"outside\",\"-1,9\":\"outside\",\"-1,10\":\"outside\",\"-1,11\":\"outside\",\"-1,12\":\"outside\",\"-1,13\":\"outside\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"0,8\":\"wall\",\"0,9\":\"wall\",\"0,10\":\"wall\",\"0,11\":\"wall\",\"0,12\":\"outside\",\"0,13\":\"outside\",\"1,5\":\"open\",\"1,6\":\"open\",\"1,7\":\"open\",\"1,8\":\"open\",\"1,9\":\"open\",\"1,10\":\"wall\",\"1,11\":\"wall\",\"1,12\":\"outside\",\"1,13\":\"outside\",\"2,5\":\"wall\",\"2,6\":\"wall\",\"2,7\":\"wall\",\"2,8\":\"wall\",\"2,9\":\"open\",\"2,10\":\"wall\",\"2,11\":\"wall\",\"2,12\":\"outside\",\"2,13\":\"outside\",\"3,5\":\"open\",\"3,6\":\"open\",\"3,7\":\"open\",\"3,8\":\"wall\",\"3,9\":\"agent\",\"3,10\":\"wall\",\"3,11\":\"wall\",\"3,12\":\"outside\",\"3,13\":\"outside\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"open\",\"4,8\":\"wall\",\"4,9\":\"wall\",\"4,10\":\"wall\",\"4,11\":\"wall\",\"4,12\":\"outside\",\"4,13\":\"outside\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"open\",\"5,9\":\"open\",\"5,10\":\"wall\",\"5,11\":\"wall\",\"5,12\":\"outside\",\"5,13\":\"outside\",\"6,5\":\"open\",\"6,6\":\"wall\",\"6,7\":\"wall\",\"6,8\":\"wall\",\"6,9\":\"open\",\"6,10\":\"wall\",\"6,11\":\"wall\",\"6,12\":\"outside\",\"6,13\":\"outside\",\"7,5\":\"open\",\"7,6\":\"wall\",\"7,7\":\"open\",\"7,8\":\"open\",\"7,9\":\"open\",\"7,10\":\"wall\",\"7,11\":\"wall\",\"7,12\":\"outside\",\"7,13\":\"outside\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 1, "answer": "1", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "cf7169e30cadfb936d8a089b9979ab6a9cf985f3aa574f7e86cdead642465a05"}959{"id": "snake_safe-000958", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (1,4), travelling east, length 3.\n........\n...oH...\n...o....\n........\n........\n...*....\n........\n........", "question": "If the snake moves east on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "752000d7fdc6a68ba4be1970a9bdba42be119ea5b1a19d31686fcc73ad69373c"}960{"id": "goal_bearing-000959", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,4), goal at (9,9).\n############\n#.....#...##\n#.##.##.#.##\n#.....#...##\n#.###.#.#.##\n#.#.#.....##\n#.#.#.#.#.##\n#...#...#.##\n###.#####.##\n#...A...#G##\n############\n############", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "319a8b745ef399109226169a7f9fa2a4bbef086d24c62406598cb314a345649a"}961{"id": "stochastic-000960", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 2, column 14. Rows increase to the south and columns to the east.\nRow -2: column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze\nRow -1: column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze\nRow 0: column 10 is open, column 11 is open, column 12 is open, column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is open, column 18 is open\nRow 1: column 10 is open, column 11 is open, column 12 is wall, column 13 is open, column 14 is open, column 15 is wall, column 16 is open, column 17 is open, column 18 is open\nRow 2: column 10 is open, column 11 is open, column 12 is open, column 13 is wall, column 14 is the agent, column 15 is open, column 16 is open, column 17 is open, column 18 is open\nRow 3: column 10 is open, column 11 is open, column 12 is open, column 13 is wall, column 14 is open, column 15 is open, column 16 is open, column 17 is open, column 18 is open\nRow 4: column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall, column 16 is open, column 17 is open, column 18 is open\nRow 5: column 10 is wall, column 11 is wall, column 12 is wall, column 13 is open, column 14 is open, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall\nRow 6: column 10 is open, column 11 is wall, column 12 is open, column 13 is open, column 14 is open, column 15 is open, column 16 is wall, column 17 is open, column 18 is open", "question": "The actuator executes the intended move with probability 0.90; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.033333333333333326, 0.9666666666666667], "deterministic": false, "fingerprint": "565c15967ff9ef0d3f75d018a4f0a8488f855e8fa81c6c49aa1f190af12ce8a3"}962{"id": "snake_food-000961", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,4), travelling south, length 3.\n........\n........\n........\n........\n...oo*..\n....H...\n........\n........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "8470c24c1a62c18c889deae953615c1ec228fa291c2222bb057c6eb9c4ffcd8f"}963{"id": "snake_safe-000962", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,7), travelling north, length 3.\n..............\n...*..........\n..............\n..............\n..............\n..............\n.......H......\n......oo......\n..............\n..............\n..............\n..............\n..............\n..............", "question": "If the snake moves north on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "44895594c7f4eb29c0da90905256c0039aeabff1aebfdcb2a5f1d232515bb469"}964{"id": "cell_kind-000963", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[16,33],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"13,30\":\"wall\",\"13,31\":\"open\",\"13,32\":\"open\",\"13,33\":\"open\",\"13,34\":\"wall\",\"13,35\":\"wall\",\"13,36\":\"outside\",\"14,30\":\"wall\",\"14,31\":\"wall\",\"14,32\":\"wall\",\"14,33\":\"open\",\"14,34\":\"wall\",\"14,35\":\"wall\",\"14,36\":\"outside\",\"15,30\":\"open\",\"15,31\":\"open\",\"15,32\":\"wall\",\"15,33\":\"open\",\"15,34\":\"wall\",\"15,35\":\"wall\",\"15,36\":\"outside\",\"16,30\":\"wall\",\"16,31\":\"open\",\"16,32\":\"wall\",\"16,33\":\"agent\",\"16,34\":\"wall\",\"16,35\":\"wall\",\"16,36\":\"outside\",\"17,30\":\"wall\",\"17,31\":\"open\",\"17,32\":\"wall\",\"17,33\":\"open\",\"17,34\":\"wall\",\"17,35\":\"wall\",\"17,36\":\"outside\",\"18,30\":\"wall\",\"18,31\":\"open\",\"18,32\":\"wall\",\"18,33\":\"wall\",\"18,34\":\"wall\",\"18,35\":\"wall\",\"18,36\":\"outside\",\"19,30\":\"wall\",\"19,31\":\"open\",\"19,32\":\"open\",\"19,33\":\"open\",\"19,34\":\"wall\",\"19,35\":\"wall\",\"19,36\":\"outside\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "2af6fb1c090579aff5060c3176416dafe98dffa34343d315609e9e28b972313b"}965{"id": "goal_bearing-000964", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,3), goal at (9,9).\n############\n#.#.......##\n#.#.#####.##\n#...#.....##\n#.###.#.#.##\n#.....#...##\n###.#.#.#.##\n#.....#.#.##\n#.#.###.#.##\n#..A#...#G##\n############\n############", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "97b514ea7c53270dc53b0ae8a2e6c6eb1e280ca497c416cb43a2cee41bdf6431"}966{"id": "open_count-000965", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 2. Rows increase to the south and columns to the east.\nRow -3: column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze\nRow -2: column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze\nRow -1: column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze\nRow 0: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is open, column 3 is open, column 4 is wall, column 5 is open, column 6 is open\nRow 1: column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is the agent, column 3 is open, column 4 is wall, column 5 is open, column 6 is open\nRow 2: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is wall, column 4 is wall, column 5 is open, column 6 is open\nRow 3: column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is wall, column 3 is wall, column 4 is open, column 5 is open, column 6 is open\nRow 4: column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is wall, column 4 is open, column 5 is wall, column 6 is open\nRow 5: column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 4, "answer": "4", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "e05476b0a70a40608d5e6fe8ae5296092fe9ab6698bbfd14ad156598ca3ff969"}967{"id": "stochastic-000966", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 3, column 27. Rows increase to the south and columns to the east.\nRow -1: column 23 is outside the maze, column 24 is outside the maze, column 25 is outside the maze, column 26 is outside the maze, column 27 is outside the maze, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze, column 31 is outside the maze\nRow 0: column 23 is wall, column 24 is wall, column 25 is wall, column 26 is wall, column 27 is wall, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze, column 31 is outside the maze\nRow 1: column 23 is open, column 24 is open, column 25 is wall, column 26 is wall, column 27 is wall, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze, column 31 is outside the maze\nRow 2: column 23 is open, column 24 is open, column 25 is open, column 26 is wall, column 27 is open, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze, column 31 is outside the maze\nRow 3: column 23 is open, column 24 is open, column 25 is open, column 26 is open, column 27 is the agent, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze, column 31 is outside the maze\nRow 4: column 23 is open, column 24 is open, column 25 is open, column 26 is open, column 27 is open, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze, column 31 is outside the maze\nRow 5: column 23 is open, column 24 is open, column 25 is open, column 26 is open, column 27 is open, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze, column 31 is outside the maze\nRow 6: column 23 is open, column 24 is open, column 25 is open, column 26 is open, column 27 is wall, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze, column 31 is outside the maze\nRow 7: column 23 is open, column 24 is open, column 25 is wall, column 26 is open, column 27 is wall, column 28 is outside the maze, column 29 is outside the maze, column 30 is outside the maze, column 31 is outside the maze", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.1333333333333333, 0.8666666666666667], "deterministic": false, "fingerprint": "4cf20842855bd4783d619f30c3639a3f2a37a98b8926505a7bc663db26a17e51"}968{"id": "first_move-000967", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (12,11), goal at (19,21).\n############################\n#.....#...................##\n#.#####.#######.#######.#.##\n#.......#.....#...#.....#.##\n#########.###.#####.#####.##\n#...#...#.#.#.#...#.#.....##\n#.#.#.#.#.#.#.#.#.#.#.######\n#.#...#...#.#...#...#.#...##\n#.#########.#########.#.####\n#.#...#.........#.......#.##\n#.###.#.#.#######.#######.##\n#...#...#.#.......#.#.....##\n###.#####.#A#######.#.######\n#...#.....#.........#.....##\n#.###.#########.#########.##\n#.#.....#.......#.........##\n#.###.#.#.#######.##########\n#...#.#...#.....#.#.......##\n#.#.###.#####.###.#.#####.##\n#.#...#.......#...#.#G#...##\n#.###.#########.###.#.#.####\n#.#.#.........#.#...#.#...##\n#.#.#########.#.###.#.###.##\n#.#.....#.....#...#.#...#.##\n#.###.#.#.#######.#.#.#.#.##\n#.....#.#...........#.#...##\n############################\n############################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "13c429c6e9a6c0c1af8948f5b0b2ffffe4568adf675d6cc776e10dda34914c37"}969{"id": "open_count-000968", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,34],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,30\":\"outside\",\"-1,31\":\"outside\",\"-1,32\":\"outside\",\"-1,33\":\"outside\",\"-1,34\":\"outside\",\"-1,35\":\"outside\",\"-1,36\":\"outside\",\"-1,37\":\"outside\",\"-1,38\":\"outside\",\"0,30\":\"open\",\"0,31\":\"open\",\"0,32\":\"open\",\"0,33\":\"open\",\"0,34\":\"open\",\"0,35\":\"wall\",\"0,36\":\"outside\",\"0,37\":\"outside\",\"0,38\":\"outside\",\"1,30\":\"wall\",\"1,31\":\"open\",\"1,32\":\"wall\",\"1,33\":\"open\",\"1,34\":\"open\",\"1,35\":\"open\",\"1,36\":\"outside\",\"1,37\":\"outside\",\"1,38\":\"outside\",\"2,30\":\"wall\",\"2,31\":\"open\",\"2,32\":\"open\",\"2,33\":\"open\",\"2,34\":\"open\",\"2,35\":\"wall\",\"2,36\":\"outside\",\"2,37\":\"outside\",\"2,38\":\"outside\",\"3,30\":\"wall\",\"3,31\":\"wall\",\"3,32\":\"open\",\"3,33\":\"open\",\"3,34\":\"agent\",\"3,35\":\"open\",\"3,36\":\"outside\",\"3,37\":\"outside\",\"3,38\":\"outside\",\"4,30\":\"wall\",\"4,31\":\"open\",\"4,32\":\"open\",\"4,33\":\"open\",\"4,34\":\"open\",\"4,35\":\"wall\",\"4,36\":\"outside\",\"4,37\":\"outside\",\"4,38\":\"outside\",\"5,30\":\"wall\",\"5,31\":\"wall\",\"5,32\":\"open\",\"5,33\":\"wall\",\"5,34\":\"wall\",\"5,35\":\"open\",\"5,36\":\"outside\",\"5,37\":\"outside\",\"5,38\":\"outside\",\"6,30\":\"open\",\"6,31\":\"open\",\"6,32\":\"open\",\"6,33\":\"open\",\"6,34\":\"open\",\"6,35\":\"open\",\"6,36\":\"outside\",\"6,37\":\"outside\",\"6,38\":\"outside\",\"7,30\":\"open\",\"7,31\":\"open\",\"7,32\":\"open\",\"7,33\":\"wall\",\"7,34\":\"open\",\"7,35\":\"open\",\"7,36\":\"outside\",\"7,37\":\"outside\",\"7,38\":\"outside\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 4, "answer": "4", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "adea2aa1ad35a8bc68824873c7535921404109a9ec293cbc1bf0f64ae1fb52f5"}970{"id": "plan_survives-000969", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"0,8\":\"wall\",\"0,9\":\"outside\",\"0,10\":\"outside\",\"1,4\":\"open\",\"1,5\":\"open\",\"1,6\":\"open\",\"1,7\":\"open\",\"1,8\":\"wall\",\"1,9\":\"outside\",\"1,10\":\"outside\",\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"wall\",\"2,8\":\"wall\",\"2,9\":\"outside\",\"2,10\":\"outside\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"open\",\"3,7\":\"agent\",\"3,8\":\"wall\",\"3,9\":\"outside\",\"3,10\":\"outside\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"open\",\"4,8\":\"wall\",\"4,9\":\"outside\",\"4,10\":\"outside\",\"5,4\":\"open\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"wall\",\"5,9\":\"outside\",\"5,10\":\"outside\",\"6,4\":\"wall\",\"6,5\":\"open\",\"6,6\":\"wall\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"outside\",\"6,10\":\"outside\"}}", "question": "The agent will attempt this plan in order: west, west, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "30d22b56f83401b316bb8d1948a11350d7709b2561bd8f94024f8b4bb5ef266b"}971{"id": "only_way-000970", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (32,11); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.##.\n####.\n##A#.\n##.#.\n#..#.", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "35af30ca73fdbbdd6b9d93a344d9c7037dd9e7b108a2971f76ddeb02af5e1458"}972{"id": "onward-000971", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 0, column 0. Rows increase to the south and columns to the east.\nRow -3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze\nRow -2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze\nRow -1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze\nRow 0: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is the agent, column 1 is open, column 2 is wall, column 3 is open\nRow 1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is open, column 3 is open\nRow 2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is open, column 2 is open, column 3 is open\nRow 3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is open, column 1 is wall, column 2 is wall, column 3 is wall", "question": "Suppose the agent moves one cell east. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "86a006b06eaceade5e3ea546afa2f249c453853daa49594e285f6cb400ebf2a6"}973{"id": "reachable_within-000972", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (12,19), goal at (21,3).\n########################\n#.....#...............##\n#.##.##.#.#.###.#.###.##\n#...........#...#...#.##\n#.###########.###.#.#.##\n#...#.#...............##\n###.#.#.###.###.#####.##\n#.#.#.#.#...#...#.....##\n#.#.#.#.###.#.#.#.###.##\n#.....#...#...#...#...##\n#####.###.###.#...#.####\n#.......#...#.#...#...##\n#.#.#####.#.#.#.#.#A#.##\n#.....#.....#.#.#.#...##\n#...#.#.#####.#.#.#.#.##\n#...........#...#...#.##\n#.####.##.#########.#.##\n#.......#...#.....#...##\n#.#.#####.#.#.###.###.##\n#.#...#.......#.#.....##\n#.###.#.#####.#.####.###\n#..G#.................##\n########################\n########################", "question": "Can the agent reach the goal in at most 42 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "a91a6e6557746663f077a8117e0ebcce77d0995b5eae4e7bbf06750b93a87678"}974{"id": "reachable_within-000973", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (12,7), goal at (7,5).\n################\n#.............##\n#.###########.##\n#.#.........#.##\n#.#.#######.#.##\n#.#.#.......#.##\n#.#.#########.##\n#.#.#G........##\n#.#.############\n#.#...........##\n#.#.###.#####.##\n#.#...#.#.....##\n#.#####A#.######\n#.......#.....##\n################\n################", "question": "Can the agent reach the goal in at most 46 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "22363ccf06b913d6a476ce4a2d579b34080cf5f4516b0a83e4c2857d2719d50c"}975{"id": "reachable_within-000974", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,26), goal at (29,9).\n################################\n#.#.......................A...##\n#.#.##.##.#####.#.###.####.##.##\n#...#.....#.....#.....#.......##\n###.#.#.#.#.#.#.#.###.#.#..#####\n#...#.#.........#...#.#.....#.##\n#.#.#.#####.###...#...#####.#.##\n#.#.............#...#.....#...##\n#.#####.#.#...#####.#.###.#..###\n#.#.....#...#...#.....#...#...##\n#.#.###.##.####.###.#.#.###.#.##\n#...#...#.....#.......#.....#.##\n###.#.#.###.#####.###.#.#####.##\n#...........#.......#.#...#...##\n#.#######.#.#.###.###.#####.#.##\n#.#.........#...#...#.........##\n#.#.#.#.#.#####.#.#.#.#.#####.##\n#.#.#...#.........#.#.#...#...##\n#...#.###.####.####.#.###.#.#.##\n#.#.#...#.#.........#.....#.#.##\n#.#.###.#.#.###..##.#..##.#.#.##\n#...#.#...#.#.....#.......#...##\n#.###.#.#.#...#####.###.#.###.##\n#.#...#...#.#...#.......#.....##\n#.#.#.#.###.###.#.###.#.#...####\n#.#.........#...#.....#...#...##\n#.#.###...###.###.#.###.#.###.##\n#...#...#.........#.........#.##\n##.##.#####.###...####.##.#.#.##\n#........G#...................##\n################################\n################################", "question": "Can the agent reach the goal in at most 54 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "9fc3175bda30cf77f6fd863636584ca8f55a1c68cf9a0d8ac46eccceecd875eb"}976{"id": "snake_food-000975", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,6), travelling north, length 3.\n............\n............\n............\n............\n............\n......H.....\n.....oo*....\n............\n............\n............\n............\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 6, "answer": "south-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "41552d0f59be08c6e2354c7d9fa9a6c86ce7b871fae94a08539abeb7016325df"}977{"id": "snake_safe-000976", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (4,7), travelling north, length 3.\n........\n........\n........\n....*...\n.......H\n......oo\n........\n........", "question": "If the snake moves east on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "28fa059b4f68458a91443416de3e7d275c06e5d45c139da1832e6980854d61d2"}978{"id": "only_way-000977", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[8,6],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"open\",\"5,7\":\"open\",\"5,8\":\"wall\",\"5,9\":\"outside\",\"6,3\":\"wall\",\"6,4\":\"open\",\"6,5\":\"open\",\"6,6\":\"open\",\"6,7\":\"open\",\"6,8\":\"open\",\"6,9\":\"outside\",\"7,3\":\"wall\",\"7,4\":\"open\",\"7,5\":\"wall\",\"7,6\":\"open\",\"7,7\":\"open\",\"7,8\":\"open\",\"7,9\":\"outside\",\"8,3\":\"wall\",\"8,4\":\"open\",\"8,5\":\"wall\",\"8,6\":\"agent\",\"8,7\":\"wall\",\"8,8\":\"open\",\"8,9\":\"outside\",\"9,3\":\"outside\",\"9,4\":\"outside\",\"9,5\":\"outside\",\"9,6\":\"outside\",\"9,7\":\"outside\",\"9,8\":\"outside\",\"9,9\":\"outside\",\"10,3\":\"outside\",\"10,4\":\"outside\",\"10,5\":\"outside\",\"10,6\":\"outside\",\"10,7\":\"outside\",\"10,8\":\"outside\",\"10,9\":\"outside\",\"11,3\":\"outside\",\"11,4\":\"outside\",\"11,5\":\"outside\",\"11,6\":\"outside\",\"11,7\":\"outside\",\"11,8\":\"outside\",\"11,9\":\"outside\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b838ea99dedc6438432902d624f8562f6153721ed4f5d60048f5623d25880e1c"}979{"id": "blocked_pair-000978", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 8, column 4. Rows increase to the south and columns to the east.\nRow 4: column 0 is wall, column 1 is wall, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is open\nRow 5: column 0 is wall, column 1 is wall, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall\nRow 6: column 0 is wall, column 1 is wall, column 2 is open, column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is wall, column 8 is wall\nRow 7: column 0 is wall, column 1 is wall, column 2 is open, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is open, column 7 is wall, column 8 is wall\nRow 8: column 0 is wall, column 1 is wall, column 2 is open, column 3 is open, column 4 is the agent, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall\nRow 9: column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow 10: column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow 11: column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow 12: column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze", "question": "Are BOTH the cell one step south and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "8f7bf89f54a640f58ab8f52fca7757dc8cc757380d879772de31b9de4c5154ba"}980{"id": "cell_kind-000979", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 3, column 3. Rows increase to the south and columns to the east.\nRow 1: column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 2: column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 3: column 1 is open, column 2 is open, column 3 is the agent, column 4 is open, column 5 is open\nRow 4: column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 5: column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "8e27d6f142399d88725b509b9fa437ffdc109d59ccfe338c6895866ae3af04b3"}981{"id": "reachable_within-000980", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (21,7), goal at (9,7).\n########################\n#.....................##\n#.###################.##\n#.......#.....#.....#.##\n#########.#####.###.#.##\n#.................#.#.##\n#.#################.#.##\n#.#.......#.........#.##\n#.#.#####.#.#######.#.##\n#.#.#..G#.#.#.....#.#.##\n#.#.#.###.#.#.###.#.#.##\n#.#.#.....#.#.#.#.#.#.##\n#.#.#######.#.#.#.#.#.##\n#.#.......#.#...#.#.#.##\n#.#.#####.#.#####.#.#.##\n#.#.#...#.#.#.....#.#.##\n#.#.#.###.#.#.###.#.#.##\n#.#.#...#.#.#.#...#.#.##\n#.#.###.#.#.#.#####.#.##\n#.#.....#.#.#.......#.##\n#.#######.#.#########.##\n#......A..#...........##\n########################\n########################", "question": "Can the agent reach the goal in at most 39 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "ab59b93cb1ea42b312478a77643e8909ab58dc7d3dd706d108df36bc8eed61f3"}982{"id": "snake_safe-000981", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,7), travelling north, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n.......H......\n......oo......\n..............\n..............\n..............\n..............\n..............\n.*............", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "c8e8c5faa0cca6c8ae31a4f4612eb51c1771576a3ae4df61195ad5127888c6a5"}983{"id": "move_two_step-000982", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 4, column 3. Rows increase to the south and columns to the east.\nRow 1: column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open\nRow 2: column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall\nRow 3: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open, column 6 is wall\nRow 4: column 0 is wall, column 1 is open, column 2 is wall, column 3 is the agent, column 4 is wall, column 5 is open, column 6 is wall\nRow 5: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall\nRow 6: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall\nRow 7: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall", "question": "The agent plans to move east, then east. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "047b2dc9ebf27878da870d994614b8df7389fc881b168a0235cd563795fa7abc"}984{"id": "distance_band-000983", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,5), goal at (11,10).\n#####........###\n#####..#..#..###\n#####A####.#.#..\n######.##.....#.\n#####..#..##....\n.#....#.....#..#\n.....#......#.#.\n...#....###.....\n...#......##....\n..#...#.####....\n###...###.#.....\n####......G#..#.\n#####..#####....\n####...........#\n######........##\n#######..#..#...", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 3, "answer": "20 to 39 moves", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "a3b47d2d86db11c55030777702682525dda5b35259fd04705f7bc8743946780b"}985{"id": "goal_bearing-000984", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,9), goal at (3,3).\n############\n#.....#.#.##\n#.###.#.#.##\n#.#G#.#.#A##\n#.#.#.#.#.##\n#.#.#.#.#.##\n#.#.#.#.#.##\n#.#.#.#.#.##\n#.#.#.#.#.##\n#...#.....##\n############\n############", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "8d9e74b5fe43c99a61494b345229a1e171c60b9c3216f2d537134139a06ceace"}986{"id": "move_two_step-000985", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 5. Rows increase to the south and columns to the east.\nRow -1: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze\nRow 0: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall\nRow 1: column 3 is open, column 4 is open, column 5 is the agent, column 6 is wall, column 7 is open\nRow 2: column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open\nRow 3: column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open", "question": "The agent plans to move west, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "9c23bec4e3b4746aebeb395b315a1c5d05a7e5d0e15c63e27e882887467f83d9"}987{"id": "blocked_pair-000986", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,6],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-2,5\":\"outside\",\"-2,6\":\"outside\",\"-2,7\":\"outside\",\"-2,8\":\"outside\",\"-2,9\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"-1,6\":\"outside\",\"-1,7\":\"outside\",\"-1,8\":\"outside\",\"-1,9\":\"outside\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"0,8\":\"wall\",\"0,9\":\"outside\",\"1,3\":\"open\",\"1,4\":\"wall\",\"1,5\":\"open\",\"1,6\":\"agent\",\"1,7\":\"open\",\"1,8\":\"wall\",\"1,9\":\"outside\",\"2,3\":\"open\",\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"wall\",\"2,8\":\"wall\",\"2,9\":\"outside\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"open\",\"3,7\":\"open\",\"3,8\":\"wall\",\"3,9\":\"outside\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"open\",\"4,8\":\"wall\",\"4,9\":\"outside\"}}", "question": "Are BOTH the cell one step east and the cell one step west blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "8b4839cc20f60c7a7288a848aa8e5dc44b0d45a6989af2d303dda23673b2d6cf"}988{"id": "snake_safe-000987", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (4,4), travelling north, length 3.\n............\n............\n............\n............\n....H.......\n....oo......\n............\n........*...\n............\n............\n............\n............", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "139643e0a761f2f8b7ec15482ed4063253dba92432dfaeef056e983583ef4b3e"}989{"id": "distance_band-000988", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,8), goal at (1,9).\n################\n#.....#..G....##\n#.#####.#####.##\n#.......#.....##\n#.#####.#.###.##\n#...#...#.#.#.##\n#.#.#.#.#.#.#.##\n#.#...#.......##\n#.#.#######.####\n#.#.#...A.....##\n###.#.#.#.###.##\n#.#.......#...##\n#.#####.#.#.#.##\n#.......#.....##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "4055fbe04eefd0ce69149d851bde13237478819cca102537c0ecdd1c1429bab4"}990{"id": "blocked_pair-000989", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 9, column 9. Rows increase to the south and columns to the east.\nRow 7: column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall\nRow 8: column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall\nRow 9: column 7 is open, column 8 is wall, column 9 is the agent, column 10 is wall, column 11 is wall\nRow 10: column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall\nRow 11: column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall", "question": "Are BOTH the cell one step west and the cell one step south blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "a6611535d48574dcea470faf2845c941b5df816b4d9ef6ec187764c2432e6740"}991{"id": "blocked_pair-000990", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 3, column 7. Rows increase to the south and columns to the east.\nRow -1: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 0: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 1: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 2: column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 3: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is the agent, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 4: column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 5: column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 6: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 7: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze", "question": "Are BOTH the cell one step north and the cell one step west blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "1757ed0a798b4acf7437cffe8494ade8f15ea192ded446333009409803775ae9"}992{"id": "goal_bearing-000991", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (29,6), goal at (7,15).\n####################################\n#.......#.........................##\n#.#####.#.#######################.##\n#.#...#.#.#...#...#.......#.....#.##\n#.#.#.#.#.#.#.#.#.#.#.###.#.###.#.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.#G#.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.###.#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.....#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#######.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.........#.#.#.#.#.#.#.##\n#.#.#.#.#.###########.#.#.#.#.#.#.##\n#.#.#.#.#.............#.....#...#.##\n#.#.###.#########################.##\n#.#...............................##\n#.###############################.##\n#.........................#.....#.##\n#.#######################.#.###.#.##\n#.#.....................#.#.#...#.##\n#.#.#################.#.#.#.#.###.##\n#.#.................#.#.#.#.#...#.##\n#.#################.#.#.#.#.#####.##\n#.#...#...........#.#.#.#.#.......##\n#.#.#.###########.#.#.#.#.#.########\n#.#.#.............#.#.#.#.#.#...#.##\n#.#.###############.#.#.#.#.#.#.#.##\n#.#.................#.#.#.#.#.#.#.##\n#.#.#################.#.#.#.###.#.##\n#.#.#.A.............#.#.#.#.....#.##\n#.#.###############.#.#.#.#######.##\n#.#.................#.#.#.........##\n#.###################.#.############\n#.....................#...........##\n####################################\n####################################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a0a495cbd94350968d01fb8c890ae02d2097fa985d5b71f9d465acb3a5ce11fc"}993{"id": "snake_food-000992", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (2,1), travelling west, length 3.\n........\n........\n.Hoo....\n........\n........\n........\n........\n.*......", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 1, "answer": "south", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "03e9339a53605914874bc6088f324b0b6b49108bdcd0c2c3e30bfe648e5c6d53"}994{"id": "only_way-000993", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXX\n#####\n.#A#.\n.#.#.\n.#...", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3b84df8db2063a412dc7e7065f38544753f8b4d1cb037cd6ca00b7c4d814e94e"}995{"id": "reachable_within-000994", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,4), goal at (1,7).\n############\n#...#..G..##\n#.###.###.##\n#.........##\n#...###.####\n#.#.A.....##\n#.#.#.######\n#.#.......##\n#.###.###.##\n#.........##\n############\n############", "question": "Can the agent reach the goal in at most 6 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "41c43decd65fa9ee7ae1062c441dd1be1fdb9ee22f19ec4adce36b165df4370b"}996{"id": "move_two_step-000995", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,4],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"-1,6\":\"outside\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"agent\",\"1,5\":\"open\",\"1,6\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"2,6\":\"wall\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"3,6\":\"open\"}}", "question": "The agent plans to move west, then east. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "a35c62ac7a5140fd6b1386871d1d73519324c868596a636b60f697a1834386f8"}997{"id": "reachable_within-000996", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,1), goal at (7,13).\n################\n#.....#.......##\n#.###.#.###.####\n#...#.#.#.#...##\n###.#.#.#.###.##\n#...#.#.#...#.##\n#.###.#.###.#.##\n#A#...#.....#G##\n###.#.#####.####\n#...#...#...#.##\n#.#####.#.###.##\n#.#...#.#.#...##\n#.#.#.###.###.##\n#...#.........##\n################\n################", "question": "Can the agent reach the goal in at most 76 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "2d16a41e5e9a9ec9ca7a9ba08441667abd3a733c0d322f91a2b3b8d1c5e01593"}998{"id": "only_way-000997", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[15,13],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"13,11\":\"open\",\"13,12\":\"wall\",\"13,13\":\"open\",\"13,14\":\"wall\",\"13,15\":\"open\",\"14,11\":\"open\",\"14,12\":\"wall\",\"14,13\":\"wall\",\"14,14\":\"wall\",\"14,15\":\"wall\",\"15,11\":\"open\",\"15,12\":\"wall\",\"15,13\":\"agent\",\"15,14\":\"open\",\"15,15\":\"open\",\"16,11\":\"open\",\"16,12\":\"wall\",\"16,13\":\"wall\",\"16,14\":\"wall\",\"16,15\":\"wall\",\"17,11\":\"open\",\"17,12\":\"open\",\"17,13\":\"open\",\"17,14\":\"open\",\"17,15\":\"open\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "2e64699baace50a8ef8b2f9e53dade4b4938f304149eea777d7d82e6585d5031"}999{"id": "reachable_within-000998", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,6), goal at (9,5).\n############\n#.........##\n#.#####.#.##\n#.#...A...##\n#.#.#.###.##\n#.....#...##\n#####.#.#.##\n#.....#.#.##\n#.#.###.#.##\n#....G#...##\n############\n############", "question": "Can the agent reach the goal in at most 8 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "c971c1aeba53a85bbbbc971013f1b3acb8ab2270989d445362323202506afd54"}1000{"id": "blocked_pair-000999", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (15,16); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.#.#\n#.#..\n#.A.#\n#####\n....#", "question": "Are BOTH the cell one step north and the cell one step south blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "1847a7d1b1f9136b8e164541b271a3a280d9487db9cd6e546b9539f9b0cab531"}1001{"id": "snake_safe-001000", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,7), travelling south, length 3.\n..............\n.........*....\n..............\n..............\n..............\n..............\n..............\n......oo......\n.......H......\n..............\n..............\n..............\n..............\n..............", "question": "If the snake moves north on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "75a84290ecd8f56a074411802e0e5886e54285dae924e2b3d8845366cb6c5f75"}1002{"id": "first_move-001001", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (25,2), goal at (3,31).\n####################################\n#...#.#.....#.....#.....#.........##\n#.#...#.#.#.#.###.#.#.#.#.#.###.#.##\n#.#.#.....#.........#.#.#.#....G#.##\n#.#.#####.#.#.#######.#.#.#######.##\n#.#.....#.#...#.......#...#.....#.##\n#.#####.#.###.#.#######.###.#####.##\n#.....#.....#.#...#.....#...#.....##\n#.###.#####.#####.#.###.###.#.##.###\n#.....#...........#.#.#...#...#...##\n#.#.###.#.#########.#.###.#.#####.##\n#.#.....#.........#.......#.......##\n#.#.#############.###.#.#####.######\n#.....#.........#.#...#...#.......##\n###.#.#.#.#####.#.#.#####.#.#.#.#.##\n#.......#.....#...#.#...#.....#.#.##\n#.#.#.#####.#.#.#.#.#.###.###.#.#.##\n#.#.#.#.....#.#.#...#.......#.#.#.##\n#...#.#.#...#.#.#.###.#####.#.#.#.##\n#.#...#.......#.#...#.....#.#...#.##\n#.#####.###.#.#.#.###.##.##.##.##.##\n#.#...#.....#...#.#...#.....#.....##\n#.#.#.#.#.#####.#.#.###.###.#.###.##\n#...#.#.........#.....#.#.#.#.#...##\n##.##.#.###.#####.#####.#...#.#.#.##\n#.A.#.....#...#.....#...#.#...#...##\n#.###.###.###.#.#.###.#.#.#.#####.##\n#...#...#.#.....#.#...#.#.........##\n###.###.#.#.###.#.#.######.##.#.####\n#.....#...#...#...#.#...#.....#...##\n#.#########.#.###.#.#.#.#.#..####.##\n#.#.....#...#...#...#.#...#...#...##\n#.#.###.#.###.#####...#####.#.#.#.##\n#.....#.............#.......#.....##\n####################################\n####################################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "06bc061143867052a4383ae89e81c9f660450e53f060f84851df843980965401"}1003{"id": "onward-001002", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[8,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"wall\",\"4,8\":\"wall\",\"4,9\":\"wall\",\"4,10\":\"wall\",\"4,11\":\"wall\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"wall\",\"5,9\":\"open\",\"5,10\":\"open\",\"5,11\":\"open\",\"6,3\":\"wall\",\"6,4\":\"wall\",\"6,5\":\"open\",\"6,6\":\"wall\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"open\",\"6,10\":\"wall\",\"6,11\":\"wall\",\"7,3\":\"open\",\"7,4\":\"open\",\"7,5\":\"open\",\"7,6\":\"wall\",\"7,7\":\"open\",\"7,8\":\"wall\",\"7,9\":\"open\",\"7,10\":\"open\",\"7,11\":\"open\",\"8,3\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"open\",\"8,6\":\"wall\",\"8,7\":\"agent\",\"8,8\":\"wall\",\"8,9\":\"wall\",\"8,10\":\"wall\",\"8,11\":\"open\",\"9,3\":\"open\",\"9,4\":\"open\",\"9,5\":\"open\",\"9,6\":\"wall\",\"9,7\":\"open\",\"9,8\":\"open\",\"9,9\":\"open\",\"9,10\":\"wall\",\"9,11\":\"open\",\"10,3\":\"wall\",\"10,4\":\"wall\",\"10,5\":\"open\",\"10,6\":\"wall\",\"10,7\":\"wall\",\"10,8\":\"wall\",\"10,9\":\"open\",\"10,10\":\"wall\",\"10,11\":\"open\",\"11,3\":\"open\",\"11,4\":\"wall\",\"11,5\":\"open\",\"11,6\":\"wall\",\"11,7\":\"open\",\"11,8\":\"wall\",\"11,9\":\"open\",\"11,10\":\"wall\",\"11,11\":\"open\",\"12,3\":\"open\",\"12,4\":\"wall\",\"12,5\":\"open\",\"12,6\":\"wall\",\"12,7\":\"open\",\"12,8\":\"wall\",\"12,9\":\"open\",\"12,10\":\"wall\",\"12,11\":\"wall\"}}", "question": "Suppose the agent moves one cell south. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "455410d1559e762d0df57167fa47fa29293d8e8d366cd4c9e703f179b32c292a"}1004{"id": "move_two_step-001003", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,33); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n...##\n.#.##\n.#A##\n##.##\n.#.##", "question": "The agent plans to move north, then north. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "ff70171d44fd37ae7f5bc4176a5103a122058c505b0fae98343b4cc9317f39c2"}1005{"id": "stochastic-001004", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (0,14); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXX\nXXXXX\n#.A.X\n....X\n.#.#X", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.6, 0.4], "deterministic": false, "fingerprint": "f221bea6eeea1247df79f555355a6dc8ebf7aab2130871082399120f1962de5c"}1006{"id": "cell_kind-001005", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 5, column 7. Rows increase to the south and columns to the east.\nRow 3: column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is open\nRow 4: column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall\nRow 5: column 5 is open, column 6 is wall, column 7 is the agent, column 8 is open, column 9 is open\nRow 6: column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open\nRow 7: column 5 is open, column 6 is wall, column 7 is open, column 8 is open, column 9 is open", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "e6a04f9868de68fdd13ad8bf73ea063ad708258a31d4f995a8c484ad5388fcc2"}1007{"id": "onward-001006", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (31,16); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n....#....\n#########\n#.....#..\n#.#####.#\n..#.A...#\n#.#.#####\n..#.#....\n#########\n#########", "question": "Suppose the agent moves one cell west. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "c46cf735b72224d9d1ce119de19facef1e71df1beb356bac63483a24114b689b"}1008{"id": "snake_food-001007", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (2,3), travelling west, length 4.\n........\n........\n...Hoo*.\n.....o..\n........\n........\n........\n........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "bb6a93a99d9985f1ce8f3b4cf0d7d30beca249f1b42497979422272c83b2a8d5"}1009{"id": "blocked_pair-001008", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[0,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-3,-2\":\"outside\",\"-3,-1\":\"outside\",\"-3,0\":\"outside\",\"-3,1\":\"outside\",\"-3,2\":\"outside\",\"-3,3\":\"outside\",\"-3,4\":\"outside\",\"-2,-2\":\"outside\",\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"open\",\"0,1\":\"agent\",\"0,2\":\"wall\",\"0,3\":\"open\",\"0,4\":\"open\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"open\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"wall\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"open\",\"2,3\":\"open\",\"2,4\":\"open\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"open\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"open\"}}", "question": "Are BOTH the cell one step north and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "43839817578c9149076313274510653e828ad306476c7a9576a2a74e07f08ad5"}1010{"id": "first_move-001009", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,11), goal at (5,17).\n#####################\n#.........#.........#\n#.#################.#\n#.#........A........#\n#.#.#####.#########.#\n#.#.#...#.#......G#.#\n#.#.###.#.#.#######.#\n#.#.....#.#.......#.#\n#.#####.#.#######.#.#\n#.#...#.#.#.....#.#.#\n#.#.#.#.#.#.###.#.#.#\n#.#.#.#.#.#.#.#.#.#.#\n#.#.###.#.#.#.#.#.#.#\n#.#.....#.#.#.#.#.#.#\n#.#.#####.#.#.#.#.#.#\n#.#.#...#.#.#.....#.#\n#.#.###.#.#.#######.#\n#.#.....#.#.#.......#\n#.#######.#.#.#######\n#.........#.........#\n#####################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a3796917558333941094e46f825b39956e2a58b0cb1bf01116125e1c2e521c67"}1011{"id": "stochastic-001010", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 20, column 5. Rows increase to the south and columns to the east.\nRow 18: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open\nRow 19: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open\nRow 20: column 3 is open, column 4 is wall, column 5 is the agent, column 6 is wall, column 7 is open\nRow 21: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open\nRow 22: column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open", "question": "The actuator executes the intended move with probability 0.70; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.7999999999999999, 0.20000000000000004], "deterministic": false, "fingerprint": "919d480b92a4d3b8856ee5cfb674ec8bd32f4c3c05624c2f97e68bf3e0ff340d"}1012{"id": "move_legal-001011", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 2, column 1. Rows increase to the south and columns to the east.\nRow -2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow -1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 0: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 1: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open\nRow 2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall\nRow 3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 4: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 5: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 6: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open", "question": "The agent wants to move east. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "987f641b26ef2c507006c2dcc92c9424ec3f038815084512a19a8b61f555d328"}1013{"id": "snake_safe-001012", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (4,4), travelling north, length 3.\n............\n............\n............\n............\n....H.......\n....oo......\n............\n........*...\n............\n............\n............\n............", "question": "If the snake moves north on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "162d9590de1b1f0e945799973400930c40c016a379479b5d2bbb15664164e7a1"}1014{"id": "distance_band-001013", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (19,5), goal at (1,21).\n################################\n#...................#G..#.....##\n#.##.#.######.#.###.###.#.#.####\n#.#.............#.....#.......##\n#.##..#.#.#####.#.###.#.###.#.##\n#.#.....#.#.........#.#...#...##\n#.#.#####...#####.#.#.###.#.#.##\n#...#.......#.....#.#.#...#.#.##\n#.#.####..#.#######.#.#.###.#.##\n#...#.......#.......#.#...#.#.##\n#.###.#######.###.#.#.###.#.#.##\n#...#.#.......#.#.#.#.........##\n#.#.#.#.#####.#.#.#.#.###.#.####\n#.#.#.#...........#.#...#.#...##\n#.#.#.#####.###.###.###.#####.##\n#...#.....#...........#.......##\n###.#.###.#.#.####.##.#######.##\n#.#.............#...........#.##\n#.###.###.###.#.#.###.#.#####.##\n#...#A#.#...#.#...#.#.#.#.....##\n#.#.#...#.#.###...#.#.#.#.###.##\n#.#.#.#.#.#.....#...#.....#...##\n#.#.#.#.#.###.#.###.#.#.###.#.##\n#.#...#.#...#.#.....#.......#.##\n#.#####.###.#...###.#.#######.##\n#.#.....#...#.#.#...#...#.....##\n#.#.#.####..#.#.#.#####...###.##\n#.#.#.........#.#.......#.#...##\n#.#.###.#.#.###.##.######.###.##\n#.#.......#...................##\n################################\n################################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 4, "answer": "40 moves or more", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "b99d011e9b35cd349b2f2b68638ec558e90a78221375d46ea57e5fb40daf9d1a"}1015{"id": "move_legal-001014", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[33,34],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"30,31\":\"open\",\"30,32\":\"open\",\"30,33\":\"wall\",\"30,34\":\"wall\",\"30,35\":\"open\",\"30,36\":\"outside\",\"30,37\":\"outside\",\"31,31\":\"open\",\"31,32\":\"wall\",\"31,33\":\"open\",\"31,34\":\"open\",\"31,35\":\"open\",\"31,36\":\"outside\",\"31,37\":\"outside\",\"32,31\":\"open\",\"32,32\":\"open\",\"32,33\":\"open\",\"32,34\":\"wall\",\"32,35\":\"wall\",\"32,36\":\"outside\",\"32,37\":\"outside\",\"33,31\":\"open\",\"33,32\":\"open\",\"33,33\":\"open\",\"33,34\":\"agent\",\"33,35\":\"open\",\"33,36\":\"outside\",\"33,37\":\"outside\",\"34,31\":\"wall\",\"34,32\":\"open\",\"34,33\":\"open\",\"34,34\":\"open\",\"34,35\":\"open\",\"34,36\":\"outside\",\"34,37\":\"outside\",\"35,31\":\"open\",\"35,32\":\"open\",\"35,33\":\"open\",\"35,34\":\"wall\",\"35,35\":\"open\",\"35,36\":\"outside\",\"35,37\":\"outside\",\"36,31\":\"outside\",\"36,32\":\"outside\",\"36,33\":\"outside\",\"36,34\":\"outside\",\"36,35\":\"outside\",\"36,36\":\"outside\",\"36,37\":\"outside\"}}", "question": "The agent wants to move east. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "c106380cc7effbf75cadf23ad1ee74ed851225eb7d49dedbd8a08af3256fbcdf"}1016{"id": "goal_bearing-001015", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,1), goal at (5,7).\n#########\n#.......#\n#.#.#.#.#\n#.#.#.#.#\n#.#.#.###\n#A..#..G#\n#.###.#.#\n#.......#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "36a293b0c36ec9299e66605dd2badaf7f5beebe4a0633dff5ca65ecd2726174d"}1017{"id": "stochastic-001016", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 8, column 9. Rows increase to the south and columns to the east.\nRow 4: column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze\nRow 5: column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze\nRow 6: column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze\nRow 7: column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze\nRow 8: column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is the agent, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze\nRow 9: column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze\nRow 10: column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze\nRow 11: column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze\nRow 12: column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze", "question": "The actuator executes the intended move with probability 0.85; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9, 0.10000000000000002], "deterministic": false, "fingerprint": "960b0f0f42f5e3e7c4776d097e6120879961e9622e9c24e4c1a1eb4b96539d20"}1018{"id": "move_legal-001017", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (5,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXX#.....\nXXX#.####\nXXX#.#.#.\nXXX#.#.#.\nXXX#A#.#.\nXXX###.##\nXXX#.....\nXXX######\nXXXXXXXXX", "question": "The agent wants to move north. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "b9bdcbc27e1cc3bc2b42e4540455fc4d2387284f7ba32edd3a452328ee1a6468"}1019{"id": "open_count-001018", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 3, column 33. Rows increase to the south and columns to the east.\nRow 1: column 31 is open, column 32 is open, column 33 is open, column 34 is wall, column 35 is wall\nRow 2: column 31 is wall, column 32 is wall, column 33 is open, column 34 is wall, column 35 is wall\nRow 3: column 31 is open, column 32 is open, column 33 is the agent, column 34 is wall, column 35 is wall\nRow 4: column 31 is open, column 32 is wall, column 33 is open, column 34 is wall, column 35 is wall\nRow 5: column 31 is open, column 32 is wall, column 33 is open, column 34 is wall, column 35 is wall", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 3, "answer": "3", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "f90d133e387ba40d7a187cb0ef59449eccc8b98abf07a177d41d639575a1a762"}1020{"id": "goal_bearing-001019", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (27,24), goal at (15,17).\n####################################\n#...#.......#.......#.............##\n###.#.#.###.#.#####.#.#####.########\n#...#.#.#...#...#.#.#.....#.#.....##\n#.###.#.#.#####.#.#.#.###.###.###.##\n#.....#.#.......#...#...#.......#.##\n#.#####.#########.###############.##\n#.....#.#...#...#.......#.......#.##\n#######.#.#.#.#.#######.#.#####.#.##\n#.......#.#...#.......#...#...#...##\n#.#######.###########.#####.#.###.##\n#...#.#...#.........#.....#.#...#.##\n###.#.#.#####.###.#######.#.###.#.##\n#...#.#.#...#...#.......#...#...#.##\n#.###.#.#.#.###.###.#####.###.#.#.##\n#...#.....#...#..G#.#...#.#...#.#.##\n#.#.#########.#####.#.#.###.#.###.##\n#.#.#.......#.......#.#...#.#.#...##\n#.#.#.#####.###.#####.###.#.#.#.####\n#.#...#...#.#...#.....#.....#.#...##\n#.#####.#.#.#####.#####.#########.##\n#.#.....#.#.....#...#...#.........##\n#.#.#.#########.###.#####.##########\n#.#.#...#.......#...#...#...#.....##\n#.#####.#.#######.#.#.#.###.#####.##\n#.......#.......#.#.#.#...........##\n#######.#######.#.#.#.###########.##\n#.....#...#...#.#.#.#.#.A.#.....#.##\n#.#.#.###.#.###.#.###.#.###.###.####\n#.#.#.#.....#...#...#.#.#...#.#...##\n#.#.###.#####.#####.#.#.#.###.###.##\n#.#...#.#.....#...#...#...#.....#.##\n#.###.#.#.#####.#.#########.###.#.##\n#...#...#.......#.............#...##\n####################################\n####################################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "4a35ac65e5730a4a622a80b9f9ae89e6f24b2dfa01c32bbfc7d3176bdfe9b7f1"}1021{"id": "only_way-001020", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 6, column 1. Rows increase to the south and columns to the east.\nRow 4: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall\nRow 5: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open\nRow 6: column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is wall, column 3 is wall\nRow 7: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open\nRow 8: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 5, "answer": "more than one of them", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "9899a4933e1d2ca9c8949c7c943278d2b5f7814b94c176171d14572108e8894c"}1022{"id": "stochastic-001021", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (2,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#####\n...##\n##A##\n...##\n##.##", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move north. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.2666666666666667, 0.7333333333333333], "deterministic": false, "fingerprint": "08746392d5bd92de3bc8f289936ea37a071801583f63942a0785ebdd5ccb91d1"}1023{"id": "onward-001022", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 0, column 2. Rows increase to the south and columns to the east.\nRow -2: column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze\nRow -1: column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze\nRow 0: column 0 is open, column 1 is open, column 2 is the agent, column 3 is open, column 4 is open\nRow 1: column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is wall\nRow 2: column 0 is open, column 1 is open, column 2 is open, column 3 is open, column 4 is wall", "question": "Suppose the agent moves one cell south. From that new cell, would a further step south be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "269c7b2805153c9a11f51f6480dcd779d9c4a42d018bfea334226143ca6c3155"}1024{"id": "only_way-001023", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 9. Rows increase to the south and columns to the east.\nRow 5: column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is wall\nRow 6: column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall\nRow 7: column 7 is open, column 8 is wall, column 9 is the agent, column 10 is wall, column 11 is wall\nRow 8: column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall\nRow 9: column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is wall", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "9820f338028dff40c121fa22276b049a3f681af339df41204a7d884a58d8605d"}1025{"id": "stochastic-001024", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 15, column 6. Rows increase to the south and columns to the east.\nRow 12: column 3 is wall, column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is wall\nRow 13: column 3 is wall, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall\nRow 14: column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is wall\nRow 15: column 3 is wall, column 4 is wall, column 5 is open, column 6 is the agent, column 7 is wall, column 8 is wall, column 9 is wall\nRow 16: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze\nRow 17: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze\nRow 18: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze", "question": "The actuator executes the intended move with probability 0.75; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.16666666666666663, 0.8333333333333334], "deterministic": false, "fingerprint": "da402b7b98213ddf2f7fe56583a987a3ce54c035ce584bbf6dd988c2526feb64"}1026{"id": "goal_bearing-001025", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,7), goal at (7,7).\n#########\n#.......#\n#.#####A#\n#.....#.#\n#.###.###\n#.#.#...#\n#.#.#####\n#......G#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 1, "answer": "south", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "1a70ec32ee57fa0b4321d08566aa6dd06507259b4d23b97b319a71b330f22e9f"}1027{"id": "distance_band-001026", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,6), goal at (13,13).\n################\n#.............##\n#.#########.####\n#.#.#.....#...##\n#.#.#.#.#.###.##\n#.#.......#...##\n#.###.#.###.#.##\n#.....#.#...#.##\n#.#.#.#...#.####\n#.....A.#.#...##\n###.#.#.###.#.##\n#.....#...#.#.##\n#.###.#.#.###.##\n#.......#....G##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "5bad7eac342f6187d6b975fb22da95ae2a872a6e75964eb48eb584a1b278700a"}1028{"id": "onward-001027", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[2,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"agent\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"open\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\",\"5,4\":\"wall\"}}", "question": "Suppose the agent moves one cell north. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "4ac05db2cc5eb03e128961f90cf9490b11c403097d0fc5ac6bd34c8ac89f6d3e"}1029{"id": "snake_food-001028", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (1,6), travelling north, length 3.\n........\n......H.\n.....oo.\n........\n........\n........\n........\n.*......", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 7, "answer": "south-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "3f943ad481dc5a86802e6bfc482b591aa6f08bd95d795a53b05d73643a70ee2e"}1030{"id": "move_legal-001029", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 3, column 3. Rows increase to the south and columns to the east.\nRow 1: column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 2: column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open\nRow 3: column 1 is open, column 2 is wall, column 3 is the agent, column 4 is wall, column 5 is open\nRow 4: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 5: column 1 is open, column 2 is open, column 3 is open, column 4 is wall, column 5 is open", "question": "If the agent attempts one cell south (row+1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "93f4f83be7aa574b108c8f1ad3e7c0698159dabee1d2bc3d7d94e14ce392072d"}1031{"id": "move_two_step-001030", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,5\":\"open\",\"1,6\":\"wall\",\"1,7\":\"open\",\"1,8\":\"wall\",\"1,9\":\"outside\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"open\",\"2,8\":\"wall\",\"2,9\":\"outside\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"agent\",\"3,8\":\"wall\",\"3,9\":\"outside\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"open\",\"4,8\":\"wall\",\"4,9\":\"outside\",\"5,5\":\"open\",\"5,6\":\"open\",\"5,7\":\"open\",\"5,8\":\"wall\",\"5,9\":\"outside\"}}", "question": "The agent plans to move south, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "a89db315f933b992893f7b498327de7fd0f1f2ea95b0e05e222a2f26f84743b0"}1032{"id": "plan_survives-001031", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[2,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"0,8\":\"wall\",\"0,9\":\"outside\",\"1,5\":\"open\",\"1,6\":\"open\",\"1,7\":\"open\",\"1,8\":\"wall\",\"1,9\":\"outside\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"agent\",\"2,8\":\"wall\",\"2,9\":\"outside\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"3,8\":\"wall\",\"3,9\":\"outside\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"wall\",\"4,8\":\"wall\",\"4,9\":\"outside\"}}", "question": "The agent will attempt this plan in order: north, south, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "206c270f1edbc1e7f1038be722cfef02e774a399d68485e559ea068d878a905a"}1033{"id": "move_legal-001032", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (2,5); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXX\n#######\n..#....\n#.#A###\n#.#.#..\n#.#.#.#\n#.#.#.#", "question": "Is the cell one step south of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "7aaaee1de5987e52086c17b6f108596941532352a333d22b7f8ccfcd5b75aeb5"}1034{"id": "distance_band-001033", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (8,9), goal at (13,13).\n..#......#...#..\n......#.###.#...\n.....###.#..#.#.\n.#...#.#..#...##\n###..#.#.......#\n..........#.....\n.............##.\n.....#..#.##...#\n..#......A......\n.......#....#...\n.....##....#..#.\n#........#.##...\n....#.........##\n....#.....#..G##\n#.#.....#...####\n.....#.#...#####", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 1, "answer": "5 to 9 moves", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b7ab9f3d13239b7149391f196ca1fa5f3ceab71d897d43fc6ff08ff70daea4b0"}1035{"id": "cell_kind-001034", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[7,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"wall\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"7,1\":\"open\",\"7,2\":\"open\",\"7,3\":\"agent\",\"7,4\":\"wall\",\"7,5\":\"open\",\"8,1\":\"wall\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"wall\",\"9,1\":\"outside\",\"9,2\":\"outside\",\"9,3\":\"outside\",\"9,4\":\"outside\",\"9,5\":\"outside\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "299266a9f2a13c1d040e86b091559a1fa3f0927a01dfe47bb0717c2f87b6e469"}1036{"id": "only_way-001035", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[7,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"4,4\":\"wall\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"4,7\":\"open\",\"4,8\":\"wall\",\"4,9\":\"outside\",\"4,10\":\"outside\",\"5,4\":\"open\",\"5,5\":\"open\",\"5,6\":\"open\",\"5,7\":\"open\",\"5,8\":\"wall\",\"5,9\":\"outside\",\"5,10\":\"outside\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"6,7\":\"wall\",\"6,8\":\"wall\",\"6,9\":\"outside\",\"6,10\":\"outside\",\"7,4\":\"open\",\"7,5\":\"open\",\"7,6\":\"open\",\"7,7\":\"agent\",\"7,8\":\"wall\",\"7,9\":\"outside\",\"7,10\":\"outside\",\"8,4\":\"wall\",\"8,5\":\"wall\",\"8,6\":\"wall\",\"8,7\":\"wall\",\"8,8\":\"wall\",\"8,9\":\"outside\",\"8,10\":\"outside\",\"9,4\":\"outside\",\"9,5\":\"outside\",\"9,6\":\"outside\",\"9,7\":\"outside\",\"9,8\":\"outside\",\"9,9\":\"outside\",\"9,10\":\"outside\",\"10,4\":\"outside\",\"10,5\":\"outside\",\"10,6\":\"outside\",\"10,7\":\"outside\",\"10,8\":\"outside\",\"10,9\":\"outside\",\"10,10\":\"outside\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "11feefbc7d7872895e980d9ae596b263c955bb6d84e3461fcf4258d78f9cd26c"}1037{"id": "distance_band-001036", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,8), goal at (1,15).\n..##...####....#\n........A#..#..G\n#.#....#........\n..#....##..#####\n#.#.#...#...#...\n##...#..#.##....\n##...#.####...#.\n##......#....#.#\n#.....#..#.##...\n##.##.........##\n###.#..###..##.#\n..#.....#..#....\n#.#..#........##\n.....###.....#.#\n..#..##...#.....\n#.#..#.........#", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 1, "answer": "5 to 9 moves", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "40876883b932aef3ac69e8b42f927d6235c1ccd0ef7957d0f7669d84fb216b8a"}1038{"id": "blocked_pair-001037", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 24, column 1. Rows increase to the south and columns to the east.\nRow 20: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open\nRow 21: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is wall, column 5 is open\nRow 22: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 23: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open\nRow 24: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open\nRow 25: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 26: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 27: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 28: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze", "question": "Are BOTH the cell one step east and the cell one step south blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "1f054a6bb7a61b0a229d8c683051f1e18990ee9fef14462ff88eb8e558fb3ab2"}1039{"id": "onward-001038", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 12, column 1. Rows increase to the south and columns to the east.\nRow 8: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 9: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open\nRow 10: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall\nRow 11: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open\nRow 12: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is open\nRow 13: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is wall, column 5 is open\nRow 14: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 15: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 16: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze", "question": "Suppose the agent moves one cell north. From that new cell, would a further step east be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "2a94253c245d9cefd269dba88bd38be6bd64a12427f0865443d3a88668d738d0"}1040{"id": "distance_band-001039", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (10,5), goal at (1,1).\n################\n#G....#.......##\n#####.#.#####.##\n#.....#.....#.##\n#.#.###.#.###.##\n#.......#.#...##\n#.#.###.#...####\n#.....#...#.#.##\n###.#.#..##.#.##\n#.#...#...#.#.##\n#.#.#A#.#####.##\n#...#.........##\n#.#####.#..##.##\n#.............##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "0258931d8b52b82af937da772172114d12cd44f5bdaa0478f40bbe5b633f0353"}1041{"id": "move_legal-001040", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[13,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"9,3\":\"open\",\"9,4\":\"wall\",\"9,5\":\"open\",\"9,6\":\"wall\",\"9,7\":\"open\",\"9,8\":\"open\",\"9,9\":\"open\",\"9,10\":\"open\",\"9,11\":\"open\",\"10,3\":\"wall\",\"10,4\":\"wall\",\"10,5\":\"wall\",\"10,6\":\"wall\",\"10,7\":\"open\",\"10,8\":\"wall\",\"10,9\":\"open\",\"10,10\":\"wall\",\"10,11\":\"open\",\"11,3\":\"open\",\"11,4\":\"open\",\"11,5\":\"open\",\"11,6\":\"open\",\"11,7\":\"open\",\"11,8\":\"wall\",\"11,9\":\"open\",\"11,10\":\"open\",\"11,11\":\"open\",\"12,3\":\"open\",\"12,4\":\"wall\",\"12,5\":\"wall\",\"12,6\":\"wall\",\"12,7\":\"wall\",\"12,8\":\"wall\",\"12,9\":\"wall\",\"12,10\":\"wall\",\"12,11\":\"open\",\"13,3\":\"open\",\"13,4\":\"wall\",\"13,5\":\"open\",\"13,6\":\"open\",\"13,7\":\"agent\",\"13,8\":\"open\",\"13,9\":\"open\",\"13,10\":\"open\",\"13,11\":\"open\",\"14,3\":\"wall\",\"14,4\":\"wall\",\"14,5\":\"wall\",\"14,6\":\"wall\",\"14,7\":\"wall\",\"14,8\":\"wall\",\"14,9\":\"wall\",\"14,10\":\"wall\",\"14,11\":\"wall\",\"15,3\":\"wall\",\"15,4\":\"wall\",\"15,5\":\"wall\",\"15,6\":\"wall\",\"15,7\":\"wall\",\"15,8\":\"wall\",\"15,9\":\"wall\",\"15,10\":\"wall\",\"15,11\":\"wall\",\"16,3\":\"outside\",\"16,4\":\"outside\",\"16,5\":\"outside\",\"16,6\":\"outside\",\"16,7\":\"outside\",\"16,8\":\"outside\",\"16,9\":\"outside\",\"16,10\":\"outside\",\"16,11\":\"outside\",\"17,3\":\"outside\",\"17,4\":\"outside\",\"17,5\":\"outside\",\"17,6\":\"outside\",\"17,7\":\"outside\",\"17,8\":\"outside\",\"17,9\":\"outside\",\"17,10\":\"outside\",\"17,11\":\"outside\"}}", "question": "Is the cell one step south of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "b13e49f6506155e686d3eb6d8ad94ca7a5e5f01562b1f5836f22e9a47da85425"}1042{"id": "first_move-001041", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (8,14), goal at (0,13).\n..#######...#G..\n.....###.#.##...\n.#..###.....#...\n.#.###..#..###..\n..#.......####..\n...###....###...\n..#....#...####.\n#...##......###.\n.#............A.\n..#...#.##...#..\n................\n..........##.#..\n..##.#...######.\n..#....#....###.\n.###....##..####\n####.#...#...###", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "7425add946bedc2c160e80f11259c9e4a776de4813ab850a661aae80012f9c84"}1043{"id": "only_way-001042", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (7,9); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n...#.##XX\n.###.##XX\n.....##XX\n.######XX\n.#..A##XX\n####.##XX\n.....##XX\n#######XX\n#######XX", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 5, "answer": "more than one of them", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "fc95fa8cbb12dabf2d272609616deeefacdf81fe0c60e8dee11e68d0b4e3490f"}1044{"id": "snake_safe-001043", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,8), travelling east, length 3.\n..............\n..............\n..............\n..............\n..............\n.......oH.*...\n.......o......\n..............\n..............\n..............\n..............\n..............\n..............\n..............", "question": "If the snake moves west on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "3054eb031e7761420915976974d9c4396376bb4afef9192cf51ea30782026e7b"}1045{"id": "snake_food-001044", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,4), travelling west, length 3.\n..............\n..............\n..............\n..............\n..............\n..*...........\n..............\n..............\n....Hoo.......\n..............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "72e70219e9793d192a79aafb7547896ede9be76f3772a642d65bbd0049224a26"}1046{"id": "goal_bearing-001045", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 36x36. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (24,31), goal at (7,31).\n####################################\n#.......#.............#.....#.....##\n#.#######.#######.###.#.###.#.#.####\n#.#.....#.#.....#...#.#...#...#...##\n#.#.###.#.#.###.###.#.###.#######.##\n#.#...#...#.#.....#.#.#...#.......##\n#.###.#####.#.#####.#.#.###.########\n#.#...#.....#...#...#.#...#.#..G#.##\n#.#.###.#.#####.#.#######.#.#.###.##\n#.#...#.#...#...#.......#.#...#...##\n#.###.#####.#.#########.#.#####.#.##\n#...#...#...#.........#...#.....#.##\n#.#####.#.###########.#######.#.####\n#.......#...#.........#.....#.#...##\n#.#########.#.#########.###.#####.##\n#.......#...#.....#.....#...#...#.##\n#######.#.#######.#.#####.###.#.#.##\n#...#...#.....#...#...#...#...#...##\n#.#.#.###.###.###.#.#.#.###.#####.##\n#.#.#...#.#.#...#.#.#.#...#.....#.##\n#.#.###.#.#.###.#.###.###.#####.#.##\n#.#...#.#...#...#...#.#...#.....#.##\n#.#.#.#.#####.#####.#.#.###.#####.##\n#.#.#.#.#.....#...#...#...#...#.#.##\n###.###.#.#####.#.#######.###.#A#.##\n#...#...#.......#.#.....#.#...#.#.##\n#.#.#.###########.#.#.#.#.#.###.#.##\n#.#.#.............#.#.#.#.#.#.....##\n#.#.#################.###.#.########\n#.#.#.....#...........#...#.......##\n#.###.#.###.#######.#.#.#########.##\n#.....#.#...#.....#.#.#.....#.....##\n#.#####.#.#####.#.#.#######.#.###.##\n#.....#.........#.#...........#...##\n####################################\n####################################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3fce6a70f12dfaa98e5233fa02299363cd30f2b2e1346a28dcfd6f238cfbfc5c"}1047{"id": "cell_kind-001046", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[3,11],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,9\":\"open\",\"1,10\":\"open\",\"1,11\":\"open\",\"1,12\":\"open\",\"1,13\":\"open\",\"2,9\":\"open\",\"2,10\":\"wall\",\"2,11\":\"wall\",\"2,12\":\"wall\",\"2,13\":\"open\",\"3,9\":\"open\",\"3,10\":\"wall\",\"3,11\":\"agent\",\"3,12\":\"wall\",\"3,13\":\"open\",\"4,9\":\"open\",\"4,10\":\"wall\",\"4,11\":\"open\",\"4,12\":\"wall\",\"4,13\":\"open\",\"5,9\":\"open\",\"5,10\":\"wall\",\"5,11\":\"open\",\"5,12\":\"wall\",\"5,13\":\"open\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 0, "answer": "dead end: at most one way out", "answer_distribution": [1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "641f2141f46664098174090fe9557a88bfea6daad1948c35fcd9de08d6d32e80"}1048{"id": "snake_safe-001047", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,8), travelling east, length 3.\n............\n............\n............\n............\n............\n.....*......\n............\n............\n......ooH...\n............\n............\n............", "question": "If the snake moves west on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "f80400a42baa1ff4fbfc0e231510d0dbcd7cabec808e3b33f53b8a82631852b5"}1049{"id": "blocked_pair-001048", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 5, column 19. Rows increase to the south and columns to the east.\nRow 3: column 17 is open, column 18 is wall, column 19 is open, column 20 is open, column 21 is open\nRow 4: column 17 is open, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is wall\nRow 5: column 17 is open, column 18 is open, column 19 is the agent, column 20 is wall, column 21 is open\nRow 6: column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is open\nRow 7: column 17 is open, column 18 is open, column 19 is open, column 20 is open, column 21 is open", "question": "Are BOTH the cell one step south and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "14000030cff5bb1b79504d44bee10b251cb66f177e7b606582b789ce9571f7f9"}1050{"id": "blocked_pair-001049", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (19,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX#.......\nX#######.\nX#...#.#.\nX#.#.#.##\nX#.#A..#.\nX#.#####.\nX#.......\nX########\nX#.....#.", "question": "Are BOTH the cell one step south and the cell one step west blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "2274de539c2f1bf5bf9fd7ad4a543f8ba05368b1f8c6712adbfbc1301f9425bd"}1051{"id": "move_two_step-001050", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (3,0); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXX###.\nXXX.###\nXXX.###\nXXXA.#.\nXXX....\nXXX....\nXXX##..", "question": "The agent plans to move west, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "7200fda95b54d575348f8719df7f7b5df2f15772365b35dc137e52c2dd382ffa"}1052{"id": "first_move-001051", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,9), goal at (25,11).\n############################\n#........A#.....#.........##\n###.#.#.#.#####.#.#.#.###.##\n#...............#...#.....##\n#.###.#####.###.#.#.###.#.##\n#...#.............#.......##\n###.#.#.#####.###.#######.##\n#...#...........#.#.#.....##\n#.#.#.###.#.#..##.#.#.###.##\n#.....#...#.......#.#.#.#.##\n#.#.###.###.##.####...#.#.##\n#.........................##\n#####.#.#.#####.#.###.#.#.##\n#...#.....#...#...#...#...##\n#.#.#####.#.#.#####.###.#.##\n#.#.......#.#...#...#...#.##\n#.#########.###.#.###.#.#.##\n#.#.....#...#.#...#.....#.##\n#.###.#.#.###.#####.###.#.##\n#.....#...#.....#...#.#.#.##\n#.#.###.#.#.#.#.#.#.#.#...##\n#.............#...#...#.#.##\n#.#####.#.###.#######.#.#.##\n#.#.....#...#.............##\n#.#.#####.#.#.#.#..##.#.#.##\n#..........G..............##\n############################\n############################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "55a31c0890bad472de4e4b1b9da4b42889ffbae1d1d3416e26ad1a2b77a8fc68"}1053{"id": "blocked_pair-001052", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[24,23],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"22,21\":\"open\",\"22,22\":\"wall\",\"22,23\":\"wall\",\"22,24\":\"wall\",\"22,25\":\"open\",\"23,21\":\"open\",\"23,22\":\"wall\",\"23,23\":\"open\",\"23,24\":\"open\",\"23,25\":\"open\",\"24,21\":\"open\",\"24,22\":\"wall\",\"24,23\":\"agent\",\"24,24\":\"wall\",\"24,25\":\"open\",\"25,21\":\"open\",\"25,22\":\"open\",\"25,23\":\"open\",\"25,24\":\"wall\",\"25,25\":\"open\",\"26,21\":\"wall\",\"26,22\":\"wall\",\"26,23\":\"wall\",\"26,24\":\"wall\",\"26,25\":\"wall\"}}", "question": "Are BOTH the cell one step north and the cell one step south blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "326f49d59ceb1c7c7a9ae4b755d7a971ff517a68e076eb064240cb30a7083f13"}1054{"id": "onward-001053", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[26,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"24,7\":\"wall\",\"24,8\":\"wall\",\"24,9\":\"wall\",\"24,10\":\"wall\",\"24,11\":\"open\",\"25,7\":\"open\",\"25,8\":\"wall\",\"25,9\":\"open\",\"25,10\":\"wall\",\"25,11\":\"open\",\"26,7\":\"open\",\"26,8\":\"wall\",\"26,9\":\"agent\",\"26,10\":\"wall\",\"26,11\":\"wall\",\"27,7\":\"open\",\"27,8\":\"wall\",\"27,9\":\"open\",\"27,10\":\"wall\",\"27,11\":\"open\",\"28,7\":\"open\",\"28,8\":\"wall\",\"28,9\":\"open\",\"28,10\":\"wall\",\"28,11\":\"open\"}}", "question": "Suppose the agent moves one cell south. From that new cell, would a further step east be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "f9cf533284e68fd618e4037576a90d9e86130fa32f68c23c7b35c380028bff87"}1055{"id": "snake_safe-001054", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (4,6), travelling north, length 3.\n............\n............\n............\n............\n......H.....\n......o.....\n......o.....\n..*.........\n............\n............\n............\n............", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "d56ae6138f59dfacc7ebce2eb9a84d0f30ca99920c5cc6d09962246e29ca0180"}1056{"id": "distance_band-001055", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (29,26), goal at (7,23).\n################################\n#.#.............#.............##\n#.#.###########.#.###########.##\n#.#.#.........#.#.#.........#.##\n#.#.#########.#.#.#.#######.#.##\n#.............#.#.#.#.....#.#.##\n#############.#.#.#.#.###.#.#.##\n#.......#...#.#.#.#.#.#G#.#.#.##\n#.#####.#.###.#.#.#.#.#.#.#.#.##\n#.#...#.#.....#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#####.#.#.#.#.#.#.#.##\n#.#.#.#.#.....#.#.#.#.#.#.#.#.##\n#.#.#.#.#######.#.#.#.#.#.#.#.##\n#.#.#.#.#.......#.#.#.#.#.#.#.##\n#.#.#.#.#.#####.#.#.#.#.#.#.#.##\n#.#.#.#.#.#...#.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.###.#.#.#.#.#.#.#.##\n#.#.#.#.#.#.....#.#.#...#.#.#.##\n#.#.#.#.#.#######.#.#####.#.#.##\n#.#.#.#.#.........#.......#.#.##\n#.#.#.#.#.#################.#.##\n#.#.#.#.#...................#.##\n#.#.#.#.###################.#.##\n#.#.#.#.....#.............#.#.##\n#.#.#####################.#.#.##\n#.#.......................#.#.##\n#.#.#######.###############.#.##\n#.#.......#.................#.##\n#.###########################.##\n#.........................A...##\n################################\n################################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 4, "answer": "40 moves or more", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "621c325ef5dda5417e6b390d0040d660302db1012a663698b3ca20361880b185"}1057{"id": "distance_band-001056", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (11,11), goal at (17,3).\n################################\n#.....#.......................##\n#.###.#.##.#####.##########.#.##\n#.#.#...#...#.............#...##\n#.#.#####.#.###.#.#.#######.#.##\n#.........#...#.....#.......#.##\n#####.#######.#.###.#.####..####\n#.....#...#...#...#.#...#.....##\n#.###.#.#.#.###.#.#.###.#.###.##\n#...#...#.#.#...#.....#...#.#.##\n#.#.##.####.###.#.#.#####.#.#.##\n#...#......A..#...#.#...#...#.##\n#.#.#.###.###.#.###.#.#.###...##\n#...#.#.....#.#.#...#.#...#.#.##\n#.#.#.#.#.###.#.###.#.###.###.##\n#.#.#...#...#.#.#.....#.......##\n#.#.#.#.###.#.....###.#.###.####\n#.#G#.......#...#.#.....#.....##\n#.#####.#.###.#.#.#.#####.###.##\n#...#...#.....#.#.#.....#.#.#.##\n#.#.#.#.#.#####.#.#####.#.#.#.##\n#.#...#.........#...........#.##\n#.#####.##.####.#####.#######.##\n#.#...#.#.....#.#.........#...##\n#.#.#.#.#.#.###.#.#.#.#####.#.##\n#.#.#...#.#.#...#.#.#.......#.##\n#.#.##.####.#.###.#.###.##.##.##\n#.#.....#...#.#.#.......#.....##\n#.#####.#.###.#.#.###.#.#.######\n#.......#.....#...#.....#.....##\n################################\n################################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 3, "answer": "20 to 39 moves", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "065aa4867abe32ef6fb27c1aba80b5e46007b61b24ff9b546fff7bdb9bd6f036"}1058{"id": "only_way-001057", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 0, column 11. Rows increase to the south and columns to the east.\nRow -2: column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze\nRow -1: column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze, column 12 is outside the maze, column 13 is outside the maze\nRow 0: column 9 is open, column 10 is open, column 11 is the agent, column 12 is outside the maze, column 13 is outside the maze\nRow 1: column 9 is wall, column 10 is open, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze\nRow 2: column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze, column 13 is outside the maze", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "8e528e607dd37fe388be2c7e221f6a11b4bf26f067ff37230377fe6ccbc2f5e0"}1059{"id": "plan_survives-001058", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (13,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#.#.#.#.\n.#.#.#.#.\n.#.#.#.#.\n.#.#.#.#.\n.#.#A#.#.\n.#.#.#.##\n.#.#.#.#.\n.#####.##\n.........", "question": "The agent will attempt this plan in order: south, north, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "19352c982e4f32a7aac5fde841b114653eb72030f0f241a4e5f1735591d852e8"}1060{"id": "distance_band-001059", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,3), goal at (13,13).\n################\n#.............##\n#.###########.##\n#...#.........##\n#.###.#######.##\n#.#...#.....#.##\n###.###.#####.##\n#.#.#.....#...##\n#.#.###.#.#.####\n#.#A..#.#...#.##\n#.###.###.###.##\n#...#...#.#...##\n###.###.#.#.#.##\n#.......#...#G##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 3, "answer": "20 to 39 moves", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "34bffa71b89dd9667568e3a14fc076213a44df247dde2e3c74fb475f889eee51"}1061{"id": "open_count-001060", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[35,2],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"33,0\":\"wall\",\"33,1\":\"wall\",\"33,2\":\"open\",\"33,3\":\"wall\",\"33,4\":\"open\",\"34,0\":\"wall\",\"34,1\":\"open\",\"34,2\":\"open\",\"34,3\":\"open\",\"34,4\":\"wall\",\"35,0\":\"wall\",\"35,1\":\"open\",\"35,2\":\"agent\",\"35,3\":\"open\",\"35,4\":\"open\",\"36,0\":\"outside\",\"36,1\":\"outside\",\"36,2\":\"outside\",\"36,3\":\"outside\",\"36,4\":\"outside\",\"37,0\":\"outside\",\"37,1\":\"outside\",\"37,2\":\"outside\",\"37,3\":\"outside\",\"37,4\":\"outside\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 3, "answer": "3", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "d30137f7699659cbf87f82722b20b2ef02be286541855e92fdb014602ea18d70"}1062{"id": "snake_food-001061", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (2,5), travelling north, length 3.\n........\n........\n.....H.*\n.....o..\n.....o..\n........\n........\n........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "ad32a3b909a8d1d1ce4f6e2f87ba7ca62a0eb1cef8f85b026134552cd82ef31c"}1063{"id": "move_legal-001062", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[11,6],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"9,4\":\"open\",\"9,5\":\"open\",\"9,6\":\"open\",\"9,7\":\"open\",\"9,8\":\"open\",\"10,4\":\"wall\",\"10,5\":\"wall\",\"10,6\":\"wall\",\"10,7\":\"wall\",\"10,8\":\"wall\",\"11,4\":\"open\",\"11,5\":\"open\",\"11,6\":\"agent\",\"11,7\":\"open\",\"11,8\":\"open\",\"12,4\":\"wall\",\"12,5\":\"wall\",\"12,6\":\"wall\",\"12,7\":\"wall\",\"12,8\":\"wall\",\"13,4\":\"open\",\"13,5\":\"open\",\"13,6\":\"open\",\"13,7\":\"open\",\"13,8\":\"open\"}}", "question": "The agent wants to move west. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "c5656f1d6bd86900d06243ee1f4f1a77b4edcb1b05038a5ec68dd015c7b80bc6"}1064{"id": "only_way-001063", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 8, column 17. Rows increase to the south and columns to the east.\nRow 5: column 14 is wall, column 15 is open, column 16 is wall, column 17 is wall, column 18 is open, column 19 is open, column 20 is wall\nRow 6: column 14 is wall, column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is wall\nRow 7: column 14 is wall, column 15 is wall, column 16 is open, column 17 is wall, column 18 is open, column 19 is open, column 20 is open\nRow 8: column 14 is open, column 15 is open, column 16 is wall, column 17 is the agent, column 18 is wall, column 19 is open, column 20 is wall\nRow 9: column 14 is wall, column 15 is open, column 16 is wall, column 17 is open, column 18 is wall, column 19 is open, column 20 is open\nRow 10: column 14 is open, column 15 is open, column 16 is wall, column 17 is open, column 18 is open, column 19 is open, column 20 is wall\nRow 11: column 14 is open, column 15 is open, column 16 is wall, column 17 is wall, column 18 is open, column 19 is wall, column 20 is wall", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "c8a5889ad7c2fc4c593f7f3d0a9a22e706af217a841982b63f6ff515df30f178"}1065{"id": "snake_food-001064", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,6), travelling west, length 4.\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n........o.....\n.*....Hoo.....\n..............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "6f5a99c4dfb8b379bff1c5c473bf269ea51143e1ba40fcdc59b57264371e65a8"}1066{"id": "reachable_within-001065", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,7), goal at (7,11).\n################\n#.....#...#...##\n#.###.#.#.#.#.##\n#...#..A#...#.##\n###########.#.##\n#.........#.#.##\n#.#######.###.##\n#...#.....#G#.##\n###.#.#.###.#.##\n#.#.#.#.#...#.##\n#.#.#.###.###.##\n#...#.....#...##\n#.#########.#.##\n#...........#.##\n################\n################", "question": "Can the agent reach the goal in at most 73 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "735100a2342c61adff3bcea916cf254523a905ea757dbfe6b56f83400698341e"}1067{"id": "blocked_pair-001066", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,2); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXX\nXXXXXXX\nX######\nX#.A.#.\nX#.###.\nX#.....\nX######", "question": "Are BOTH the cell one step west and the cell one step north blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "56d161e91803a4352ee51098e94a1e7e9ac4360ba898c8ed70b5211b1d7ca546"}1068{"id": "cell_kind-001067", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[10,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"6,5\":\"open\",\"6,6\":\"open\",\"6,7\":\"open\",\"6,8\":\"open\",\"6,9\":\"wall\",\"6,10\":\"wall\",\"6,11\":\"open\",\"6,12\":\"outside\",\"6,13\":\"outside\",\"7,5\":\"open\",\"7,6\":\"open\",\"7,7\":\"open\",\"7,8\":\"open\",\"7,9\":\"open\",\"7,10\":\"open\",\"7,11\":\"open\",\"7,12\":\"outside\",\"7,13\":\"outside\",\"8,5\":\"open\",\"8,6\":\"open\",\"8,7\":\"wall\",\"8,8\":\"open\",\"8,9\":\"open\",\"8,10\":\"open\",\"8,11\":\"wall\",\"8,12\":\"outside\",\"8,13\":\"outside\",\"9,5\":\"wall\",\"9,6\":\"open\",\"9,7\":\"wall\",\"9,8\":\"open\",\"9,9\":\"wall\",\"9,10\":\"wall\",\"9,11\":\"wall\",\"9,12\":\"outside\",\"9,13\":\"outside\",\"10,5\":\"wall\",\"10,6\":\"wall\",\"10,7\":\"open\",\"10,8\":\"open\",\"10,9\":\"agent\",\"10,10\":\"open\",\"10,11\":\"open\",\"10,12\":\"outside\",\"10,13\":\"outside\",\"11,5\":\"wall\",\"11,6\":\"wall\",\"11,7\":\"wall\",\"11,8\":\"open\",\"11,9\":\"wall\",\"11,10\":\"open\",\"11,11\":\"open\",\"11,12\":\"outside\",\"11,13\":\"outside\",\"12,5\":\"outside\",\"12,6\":\"outside\",\"12,7\":\"outside\",\"12,8\":\"outside\",\"12,9\":\"outside\",\"12,10\":\"outside\",\"12,11\":\"outside\",\"12,12\":\"outside\",\"12,13\":\"outside\",\"13,5\":\"outside\",\"13,6\":\"outside\",\"13,7\":\"outside\",\"13,8\":\"outside\",\"13,9\":\"outside\",\"13,10\":\"outside\",\"13,11\":\"outside\",\"13,12\":\"outside\",\"13,13\":\"outside\",\"14,5\":\"outside\",\"14,6\":\"outside\",\"14,7\":\"outside\",\"14,8\":\"outside\",\"14,9\":\"outside\",\"14,10\":\"outside\",\"14,11\":\"outside\",\"14,12\":\"outside\",\"14,13\":\"outside\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "7dc566f58697c45543bc16f052cced1def353dd3b2f50be429569bda6a7b18ed"}1069{"id": "onward-001068", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (9,5); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#.......\n.########\n.........\n#########\n....A..#.\n.#######.\n.#.......\n.########\n.........", "question": "Suppose the agent moves one cell west. From that new cell, would a further step north be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "dc50f7d1f6e2969f00906cd36716a71990e56715103cbdc04ecbb482f02ae922"}1070{"id": "move_legal-001069", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (3,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\n######XXX\n.....#XXX\n.###.#XXX\n...#A#XXX\n##.###XXX\n.#...#XXX\n####.#XXX\n.....#XXX", "question": "If the agent attempts one cell west (col-1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "e4daf50f19122a50672e11d28b904822856c13e33248300122ae874b7be995c9"}1071{"id": "stochastic-001070", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[7,4],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"open\",\"5,5\":\"open\",\"5,6\":\"open\",\"6,2\":\"wall\",\"6,3\":\"wall\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"7,2\":\"open\",\"7,3\":\"open\",\"7,4\":\"agent\",\"7,5\":\"open\",\"7,6\":\"open\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"wall\",\"8,6\":\"wall\",\"9,2\":\"outside\",\"9,3\":\"outside\",\"9,4\":\"outside\",\"9,5\":\"outside\",\"9,6\":\"outside\"}}", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.7333333333333334, 0.26666666666666666], "deterministic": false, "fingerprint": "2fe8ee01780e712d9f65b39966f0be6fc412eda853244be9eb8cd9963eb215e7"}1072{"id": "only_way-001071", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (9,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n###.#.#\n#...#.#\n#.###.#\n#.#A#.#\n#.#.#.#\n#.#.#.#\n#.#.#.#", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "58a2da2d00387f718f5b6db346ed539861b87fe00a23627d35ffe560d805a071"}1073{"id": "goal_bearing-001072", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (6,17), goal at (5,5).\n#####################\n#.......#...........#\n#####.#.#.#########.#\n#.....#.#.#.#.....#.#\n#.#####.#.#.#.###.#.#\n#...#G..#...#.#.#.#.#\n#.#.#####.###.#.#A#.#\n#.#...#...#...#.#.#.#\n#.###.#.###.###.###.#\n#...#.....#...#.....#\n#############.#.#####\n#...#.......#.#.....#\n#.#.#.###.###.#####.#\n#.#...#...#...#...#.#\n#.#####.###.###.#.#.#\n#.....#...#.#...#...#\n#####.###.#.#.#####.#\n#...#.#.#...#.#...#.#\n#.###.#.#####.#.#.#.#\n#.............#.#...#\n#####################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b96ea4027430aa0ce4c7f7f25fd8fbb6ab25e253c6f02c292370ab3ca9247009"}1074{"id": "move_legal-001073", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[8,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"6,-1\":\"outside\",\"6,0\":\"open\",\"6,1\":\"open\",\"6,2\":\"open\",\"6,3\":\"wall\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"open\",\"7,3\":\"wall\",\"8,-1\":\"outside\",\"8,0\":\"open\",\"8,1\":\"agent\",\"8,2\":\"open\",\"8,3\":\"wall\",\"9,-1\":\"outside\",\"9,0\":\"outside\",\"9,1\":\"outside\",\"9,2\":\"outside\",\"9,3\":\"outside\",\"10,-1\":\"outside\",\"10,0\":\"outside\",\"10,1\":\"outside\",\"10,2\":\"outside\",\"10,3\":\"outside\"}}", "question": "Is the cell one step south of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "95732ccd6f0c95c092d4a97abf578f843272102e3cc06e13b997971cd07ebb3e"}1075{"id": "move_two_step-001074", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (11,9); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.....\n.....\n..A..\nXXXXX\nXXXXX", "question": "The agent plans to move south, then south. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "1985298f78cf4908615ddc1ece0ef24b5dc855be0c0065470d2235100ee28f46"}1076{"id": "onward-001075", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (0,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXX\nXXXXX\n#.A..\n...##\n..##.", "question": "Suppose the agent moves one cell east. From that new cell, would a further step east be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "be43e5335148070f45c7269ab50bff66e65bf5f213dee8a5d11feb48599e34ed"}1077{"id": "snake_food-001076", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (8,7), travelling west, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n........o.....\n.......Ho.....\n*.............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 7, "answer": "south-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "c43955c5e7b07bd516f4a9bb8fd6dae97e25ac5f9d3f0323b5325f7249e9afbe"}1078{"id": "distance_band-001077", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 40x40. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (15,33), goal at (31,25).\n########################################\n#...#.......................#.......#.##\n#.#.#.###############.#####.#.#.###.#.##\n#.#.#.....#...#.....#.....#.#.#.#.#...##\n#.#######.###.#.###.#####.#.#.#.#.###.##\n#...#...#.....#.#.......#.#.#.#.#...#.##\n#.#.#.#.#####.#.#######.#.###.#.#.#.#.##\n#.#...#.#.....#.....#.#.#.....#.#.#...##\n#.#####.#.#########.#.#.#.#####.#.######\n#.#...#...#...#...#.#.#.#.#.....#.#...##\n###.#.#.###.###.#.#.#.#.#.#.#####.#.#.##\n#...#.#.....#...#...#...#.#.#...#...#.##\n#.###.###.###.#######.###.#.#.#.########\n#.#.....#.#...#.#.....#.#.#...#.#.....##\n#.#####.###.###.#.#####.#.#####.#.###.##\n#.....#.....#...#.#.#...#...#.#..A..#.##\n#.###.#######.###.#.#.#.###.#.#######.##\n#.#...#...#.......#.#.#.....#...#.....##\n###.###.###.#######.#.#########.#.###.##\n#...#.#...#...#.....#.....#.....#...#.##\n#.###.#.#.###.#.#.#######.#.#######.####\n#.#.#...#.....#.#.#...#...#.......#...##\n###.#.###########.#.#.#.#######.#.###.##\n#...#.#.........#...#.#.#.......#.#...##\n#.###.#.#######.#.###.#.#.#######.#.#.##\n#.....#.....#...#...#.#...#.....#.#.#.##\n#.#####.###.#.#####.#.#######.###.#.####\n#...#...#.#.#.....#.#.........#...#...##\n###.#.###.#.#####.#.###.#####.#.#####.##\n#...#.#...#...#.#.#...#.....#.#.#...#.##\n#.###.#.#.###.#.#.###.#####.###.#.#.#.##\n#...#.#.#...#.#.#.#......G#...#...#.#.##\n###.#.#####.#.#.#.#########.#.###.###.##\n#...#.......#.#.#.........#.#.....#...##\n#.#########.#.#.#########.#.#####.#.####\n#.....#...#.#.#.....#.....#.#...#.#...##\n#####.#.#.###.#.#.###.#######.#.#####.##\n#.......#.....#.#.............#.......##\n########################################\n########################################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 4, "answer": "40 moves or more", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "77abebcdb55b2f73d89610973f859b007d67ba501a3d36586e71b7a6d63568a2"}1079{"id": "goal_bearing-001078", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (25,1), goal at (21,1).\n############################\n#.........#...............##\n#####.#####.###.#########.##\n#.....#.....#.#.#...#.....##\n#.#####.#####.#.#.#.#.######\n#.#...#.#.....#.#.#.#...#.##\n#.#.#.#.#.#####.#.#.###.#.##\n#...#.#...#...#.#.#.....#.##\n#.###.#.###.#.#.#####.###.##\n#...#...#...#...#...#.....##\n###.#####.#######.#.#####.##\n#.#.#.....#...#...#.....#.##\n#.#.#.#####.#.#.#######.####\n#...#.......#...#.....#...##\n#.###########.#####.#.###.##\n#...#...#...#.#.....#.#.#.##\n###.#.###.#.#.#.#####.#.#.##\n#...#.#...#.#...#...#...#.##\n#.###.#.###.#####.#.#####.##\n#.....#...#.#.....#.....#.##\n#####.###.#.#.#########.#.##\n#G......#.#.#...#.....#...##\n#########.#.###.#.###.###.##\n#...#...#.#.....#.#.....#.##\n#.#.#.#.#.#######.#.#####.##\n#A#...#...#.......#.......##\n############################\n############################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "6efe5bf06ddb15cb48dd093c4b4a725a1aae85ea067c6d3f70ec28786d8bbd91"}1080{"id": "snake_safe-001079", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,9), travelling east, length 3.\n........*.....\n..............\n..............\n..............\n..............\n..............\n..............\n.......ooH....\n..............\n..............\n..............\n..............\n..............\n..............", "question": "If the snake moves south on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "5b46cca8d98e9d690b2c5e916e7684479350e7ee5df9f2f9241bad10e1a96fd4"}1081{"id": "distance_band-001080", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,13), goal at (3,3).\n################\n#.............##\n#.##############\n#.#G..........##\n#.###########.##\n#.......#...#.##\n#.#####.###.#.##\n#.#.........#.##\n#.###########.##\n#............A##\n#.##############\n#.#...........##\n#.###########.##\n#.............##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "e46aff384b6d953ac2048d5aaef4a3951f8bb9bf3f413b2c5dc35056d46b7c8e"}1082{"id": "plan_survives-001081", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 3, column 16. Rows increase to the south and columns to the east.\nRow -1: column 12 is outside the maze, column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze, column 20 is outside the maze\nRow 0: column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall\nRow 1: column 12 is wall, column 13 is open, column 14 is open, column 15 is open, column 16 is wall, column 17 is open, column 18 is open, column 19 is open, column 20 is wall\nRow 2: column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall\nRow 3: column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is the agent, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall\nRow 4: column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is open, column 20 is wall\nRow 5: column 12 is open, column 13 is open, column 14 is wall, column 15 is open, column 16 is open, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall\nRow 6: column 12 is wall, column 13 is wall, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open, column 18 is open, column 19 is open, column 20 is wall\nRow 7: column 12 is open, column 13 is open, column 14 is open, column 15 is open, column 16 is wall, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall", "question": "The agent will attempt this plan in order: east, north, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "6b90f87587e33766735cce6b9e64dbf81e4fd456cb0d96ea281c30b433d5006e"}1083{"id": "only_way-001082", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (9,11); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n...#.\n##.#.\n.#A#.\n.###.\n.....", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3d810b0df6a208afb0ed025011c0ae0ea33f303b0e897658b4716ae5f01d92ae"}1084{"id": "plan_survives-001083", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 1, column 6. Rows increase to the south and columns to the east.\nRow -1: column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze\nRow 0: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall\nRow 1: column 4 is open, column 5 is open, column 6 is the agent, column 7 is open, column 8 is wall\nRow 2: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall\nRow 3: column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall", "question": "The agent will attempt this plan in order: west, west, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "bb186318b0c1408f244011aa3bb2b4af93a3b598ca2c9cbd06944f8f664bc6f4"}1085{"id": "cell_kind-001084", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[0,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-4,-1\":\"outside\",\"-4,0\":\"outside\",\"-4,1\":\"outside\",\"-4,2\":\"outside\",\"-4,3\":\"outside\",\"-4,4\":\"outside\",\"-4,5\":\"outside\",\"-4,6\":\"outside\",\"-4,7\":\"outside\",\"-3,-1\":\"outside\",\"-3,0\":\"outside\",\"-3,1\":\"outside\",\"-3,2\":\"outside\",\"-3,3\":\"outside\",\"-3,4\":\"outside\",\"-3,5\":\"outside\",\"-3,6\":\"outside\",\"-3,7\":\"outside\",\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-2,5\":\"outside\",\"-2,6\":\"outside\",\"-2,7\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"-1,6\":\"outside\",\"-1,7\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"open\",\"0,3\":\"agent\",\"0,4\":\"open\",\"0,5\":\"open\",\"0,6\":\"open\",\"0,7\":\"wall\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"wall\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"wall\",\"1,5\":\"open\",\"1,6\":\"wall\",\"1,7\":\"open\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"wall\",\"2,2\":\"wall\",\"2,3\":\"open\",\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"open\",\"2,7\":\"open\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"wall\",\"3,2\":\"wall\",\"3,3\":\"wall\",\"3,4\":\"open\",\"3,5\":\"open\",\"3,6\":\"open\",\"3,7\":\"wall\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"wall\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"open\",\"4,5\":\"open\",\"4,6\":\"open\",\"4,7\":\"open\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "eb829191d0cece896a3b090cfc8a1e603e583ac7bb069a18ff1ddb7160226416"}1086{"id": "reachable_within-001085", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,1), goal at (1,1).\n############\n#G#.......##\n#.#.###.####\n#.#.#.#...##\n#.#.#.###.##\n#.#.#...#.##\n#.#.###.####\n#.#...#...##\n#.###.#.#.##\n#A......#.##\n############\n############", "question": "Can the agent reach the goal in at most 7 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "30baaf57330c3fabb3baaed14e9b268ef1847288ff767c923a6ebc0b53e90c26"}1087{"id": "move_legal-001086", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[2,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-2,5\":\"outside\",\"-2,6\":\"outside\",\"-2,7\":\"outside\",\"-2,8\":\"outside\",\"-2,9\":\"outside\",\"-2,10\":\"outside\",\"-2,11\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"-1,6\":\"outside\",\"-1,7\":\"outside\",\"-1,8\":\"outside\",\"-1,9\":\"outside\",\"-1,10\":\"outside\",\"-1,11\":\"outside\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"0,8\":\"wall\",\"0,9\":\"wall\",\"0,10\":\"wall\",\"0,11\":\"wall\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"1,6\":\"wall\",\"1,7\":\"open\",\"1,8\":\"wall\",\"1,9\":\"open\",\"1,10\":\"wall\",\"1,11\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"agent\",\"2,8\":\"wall\",\"2,9\":\"open\",\"2,10\":\"wall\",\"2,11\":\"wall\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"3,8\":\"open\",\"3,9\":\"open\",\"3,10\":\"wall\",\"3,11\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"wall\",\"4,8\":\"wall\",\"4,9\":\"open\",\"4,10\":\"wall\",\"4,11\":\"wall\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"open\",\"5,9\":\"open\",\"5,10\":\"wall\",\"5,11\":\"wall\",\"6,3\":\"wall\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"wall\",\"6,10\":\"wall\",\"6,11\":\"wall\"}}", "question": "Is the cell one step west of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "359df2f07bbb3a82708eb4276ee293fe52de94ce5da2565462293845a54ea0dd"}1088{"id": "goal_bearing-001087", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 21x21. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (19,11), goal at (17,11).\n#####################\n#.....#.............#\n###.#.#.#####.#####.#\n#...#.#.....#.#...#.#\n#.###.#####.#.#.###.#\n#.#.......#.#.#.#...#\n#.#####.###.#.#.#.###\n#.#...#.#...#.#.#.#.#\n#.#.#.###.###.#.#.#.#\n#...#.#...#...#...#.#\n#.###.#.###.###.###.#\n#...#.#.#.#...#.....#\n###.#.#.#.###.#####.#\n#.#.#...#...#.....#.#\n#.#.#####.#######.#.#\n#.#...#...#.....#.#.#\n#.###.#.#.#.###.#.#.#\n#...#.#.#.#G#...#.#.#\n#.###.#.#.###.###.#.#\n#.......#..A#.....#.#\n#####################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "f8340dd4a88ecf4d3959c5c49d802a43ec6533e2cf2e3eba03b302dee289b80d"}1089{"id": "distance_band-001088", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 32x32. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (18,17), goal at (17,11).\n################################\n#...#...........#.......#.....##\n#.#.#.###.###.###.###.###.#.#.##\n#.#.#.#...#...#.....#...#.#.#.##\n#.#.###.#.#####.#######.#.#.####\n#.#.....#.#.......#...#...#...##\n#.#####.###.#######.#.#######.##\n#.....#.#...#...#...#.....#...##\n#.###.###.###.#.#.#######.#.#.##\n#...#...#.#...#.#.#.....#.#.#.##\n###.###.#.#.#.#.#.#.#####.#.#.##\n#.#.#.#...#.#.#.#.#...#...#.#.##\n#.#.#.#####.#.#.#.###.#.###.####\n#...#...#...#.#.#.#...#.#.....##\n#.###.#.###.#.###.#.###.#####.##\n#.#...#...#.#...#.#...#...#...##\n#.###.###.#.###.#.#.#.###.#.#.##\n#...#.#.#.#G#...#.#.#.....#.#.##\n###.#.#.#.###.###A#####.###.####\n#.#...#.#.#...#.#.....#...#...##\n#.#####.#.#.#.#.#.###.###.###.##\n#.....#...#.#.#.#.#...#.#.....##\n###.###.###.###.#.#.###.#####.##\n#...#...#...#...#.#.#.#.......##\n#.###.###.#.#.#####.#.#.########\n#...#...#.#.#...#...#.#.#.....##\n###.###.###.###.#.###.#.#.###.##\n#.....#.#.....#.#.#.#...#.#...##\n#.#####.#.###.#.#.#.#.###.#.####\n#.........#...#...#.......#...##\n################################\n################################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 4, "answer": "40 moves or more", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "509e804c11fc562a3af17a24a858862cc15907692d30b7a2aa313f033e3b5434"}1090{"id": "only_way-001089", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 3, column 9. Rows increase to the south and columns to the east.\nRow 0: column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 1: column 6 is open, column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 2: column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 3: column 6 is open, column 7 is open, column 8 is wall, column 9 is the agent, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 4: column 6 is wall, column 7 is open, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 5: column 6 is wall, column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 6: column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b5f70319d07f280f5fd8011413881bf78512a7746ce8bccea6d9c824274c04f3"}1091{"id": "stochastic-001090", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 35, column 32. Rows increase to the south and columns to the east.\nRow 32: column 29 is open, column 30 is wall, column 31 is wall, column 32 is open, column 33 is open, column 34 is open, column 35 is open\nRow 33: column 29 is open, column 30 is open, column 31 is open, column 32 is open, column 33 is open, column 34 is open, column 35 is open\nRow 34: column 29 is open, column 30 is wall, column 31 is open, column 32 is open, column 33 is open, column 34 is open, column 35 is wall\nRow 35: column 29 is open, column 30 is open, column 31 is open, column 32 is the agent, column 33 is open, column 34 is wall, column 35 is wall\nRow 36: column 29 is outside the maze, column 30 is outside the maze, column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze\nRow 37: column 29 is outside the maze, column 30 is outside the maze, column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze\nRow 38: column 29 is outside the maze, column 30 is outside the maze, column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze", "question": "The actuator executes the intended move with probability 0.90; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9, 0.09999999999999998], "deterministic": false, "fingerprint": "6b67cb2b118bd8d79c24f7fa050a87a9cae96adc134b4009746fdc60528125c3"}1092{"id": "first_move-001091", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,7), goal at (1,5).\n#########\n#...#G#.#\n#.#.#.#.#\n#.....#.#\n#.###.#.#\n#...#..A#\n#.#.###.#\n#.#.....#\n#########", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "a4ebc656b115d8aa9a628d0e21c3e0b638cb483d07cba3b9860e54f9ca9442e0"}1093{"id": "snake_food-001092", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (3,1), travelling west, length 3.\n........\n.*......\n........\n.Hoo....\n........\n........\n........\n........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "fc1bf18b5578c2cbfdc8d8818f3034dd6ac865557e4b40783885ca3ce75f760e"}1094{"id": "snake_food-001093", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,5), travelling south, length 3.\n..........\n..........\n..........\n..........\n..........\n....oo....\n.....H...*\n..........\n..........\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "24bb2d052e35c454fdc8d42ef85dc4bc45472013a5567f73dfe6f5c1c407c065"}1095{"id": "reachable_within-001094", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,5), goal at (9,3).\n############\n#.#.......##\n#.#.###.#.##\n#...#...#.##\n#.###.######\n#...#.....##\n###.#####.##\n#...#A#...##\n#.###.#.#.##\n#..G#...#.##\n############\n############", "question": "Can the agent reach the goal in at most 44 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "86fd9fb6533a9de473cc99594894f3af5004edfdb84818252b960aa0fccc02d1"}1096{"id": "cell_kind-001095", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (16,3); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#...#..\n#.#####\n#.#.#..\n#.#A#.#\n#.#...#\n#.#####\n#......", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "cd02e963b857e573c5c75a2e866dbc6916a3e85d931ba987c00084281fb0b496"}1097{"id": "goal_bearing-001096", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,5), goal at (1,5).\n############\n#....G#...##\n#.#####.#.##\n#.#.....#.##\n#...#####.##\n#...#.....##\n#...#.###.##\n#.#.#...#.##\n#.#.#...#.##\n#...#A#...##\n############\n############", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "81412d42c79ea77346e24b629ef6fa313040126599f4613375d4ca2f0ffff331"}1098{"id": "blocked_pair-001097", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[25,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"21,-3\":\"outside\",\"21,-2\":\"outside\",\"21,-1\":\"outside\",\"21,0\":\"wall\",\"21,1\":\"wall\",\"21,2\":\"open\",\"21,3\":\"open\",\"21,4\":\"open\",\"21,5\":\"open\",\"22,-3\":\"outside\",\"22,-2\":\"outside\",\"22,-1\":\"outside\",\"22,0\":\"wall\",\"22,1\":\"wall\",\"22,2\":\"wall\",\"22,3\":\"open\",\"22,4\":\"open\",\"22,5\":\"wall\",\"23,-3\":\"outside\",\"23,-2\":\"outside\",\"23,-1\":\"outside\",\"23,0\":\"wall\",\"23,1\":\"wall\",\"23,2\":\"wall\",\"23,3\":\"open\",\"23,4\":\"open\",\"23,5\":\"open\",\"24,-3\":\"outside\",\"24,-2\":\"outside\",\"24,-1\":\"outside\",\"24,0\":\"open\",\"24,1\":\"open\",\"24,2\":\"wall\",\"24,3\":\"open\",\"24,4\":\"wall\",\"24,5\":\"wall\",\"25,-3\":\"outside\",\"25,-2\":\"outside\",\"25,-1\":\"outside\",\"25,0\":\"open\",\"25,1\":\"agent\",\"25,2\":\"wall\",\"25,3\":\"wall\",\"25,4\":\"open\",\"25,5\":\"open\",\"26,-3\":\"outside\",\"26,-2\":\"outside\",\"26,-1\":\"outside\",\"26,0\":\"open\",\"26,1\":\"open\",\"26,2\":\"open\",\"26,3\":\"open\",\"26,4\":\"wall\",\"26,5\":\"open\",\"27,-3\":\"outside\",\"27,-2\":\"outside\",\"27,-1\":\"outside\",\"27,0\":\"open\",\"27,1\":\"open\",\"27,2\":\"open\",\"27,3\":\"open\",\"27,4\":\"wall\",\"27,5\":\"wall\",\"28,-3\":\"outside\",\"28,-2\":\"outside\",\"28,-1\":\"outside\",\"28,0\":\"open\",\"28,1\":\"wall\",\"28,2\":\"open\",\"28,3\":\"open\",\"28,4\":\"wall\",\"28,5\":\"wall\",\"29,-3\":\"outside\",\"29,-2\":\"outside\",\"29,-1\":\"outside\",\"29,0\":\"wall\",\"29,1\":\"open\",\"29,2\":\"open\",\"29,3\":\"open\",\"29,4\":\"open\",\"29,5\":\"open\"}}", "question": "Are BOTH the cell one step south and the cell one step east blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "56bd0b1dea9249a89b72f7c1d7e34b64eb843c7b259a75a81c00b10ab608d40e"}1099{"id": "onward-001098", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (5,5); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.......#X\n.#######X\n.#.....#X\n.#.###.#X\n.#..A#.#X\n.#####.#X\n.......#X\n########X\nXXXXXXXXX", "question": "Suppose the agent moves one cell west. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "c021cde3eadb7a9585304bbc2db2903341767e090050346433bcd74d0548e773"}1100{"id": "move_legal-001099", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[8,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"6,-1\":\"outside\",\"6,0\":\"open\",\"6,1\":\"open\",\"6,2\":\"open\",\"6,3\":\"wall\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"open\",\"7,3\":\"wall\",\"8,-1\":\"outside\",\"8,0\":\"open\",\"8,1\":\"agent\",\"8,2\":\"open\",\"8,3\":\"wall\",\"9,-1\":\"outside\",\"9,0\":\"outside\",\"9,1\":\"outside\",\"9,2\":\"outside\",\"9,3\":\"outside\",\"10,-1\":\"outside\",\"10,0\":\"outside\",\"10,1\":\"outside\",\"10,2\":\"outside\",\"10,3\":\"outside\"}}", "question": "If the agent attempts one cell east (col+1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "0a025eebf8aac089f9bffda720fa501aad9c0a97f71fcff5d79a666874b9fb37"}1101{"id": "plan_survives-001100", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 16, column 5. Rows increase to the south and columns to the east.\nRow 12: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open\nRow 13: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open\nRow 14: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open\nRow 15: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open\nRow 16: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is the agent, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open\nRow 17: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open\nRow 18: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open\nRow 19: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open\nRow 20: column 1 is open, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open", "question": "The agent will attempt this plan in order: north, south, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "096a902ca169f67eef997de95a3f1f64c9c068a238fe723858696db668ba0003"}1102{"id": "snake_food-001101", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,7), travelling east, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n..............\n.....ooH..*...\n..............\n..............\n..............\n..............\n..............\n..............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 2, "answer": "east", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "abaec472b37457893aae02e1e31cfa62ec124018bbe6f9fb075e63ea810595ea"}1103{"id": "stochastic-001102", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[1,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-3,-3\":\"outside\",\"-3,-2\":\"outside\",\"-3,-1\":\"outside\",\"-3,0\":\"outside\",\"-3,1\":\"outside\",\"-3,2\":\"outside\",\"-3,3\":\"outside\",\"-3,4\":\"outside\",\"-3,5\":\"outside\",\"-2,-3\":\"outside\",\"-2,-2\":\"outside\",\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-2,5\":\"outside\",\"-1,-3\":\"outside\",\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"0,-3\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"1,-3\":\"outside\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"agent\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"2,-3\":\"outside\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"open\",\"3,-3\":\"outside\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"4,-3\":\"outside\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"5,-3\":\"outside\",\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"open\",\"5,5\":\"open\"}}", "question": "The actuator executes the intended move with probability 0.85; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9, 0.10000000000000002], "deterministic": false, "fingerprint": "33b22f69322eafdef3a4c1b6ce0e455eeac4d3a7674a4db287e497c71a8c5938"}1104{"id": "snake_safe-001103", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 8x8 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,5), travelling west, length 3.\n........\n........\n........\n.......*\n........\n......o.\n.....Ho.\n........", "question": "If the snake moves east on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "18dc170d0f41c6732238b8d56ab2734822464835a4f26cc3361203af049c8d31"}1105{"id": "snake_safe-001104", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,5), travelling north, length 3.\n....*.......\n............\n............\n............\n............\n.....H......\n.....oo.....\n............\n............\n............\n............\n............", "question": "If the snake moves west on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "9af89034a8b975c841fab6634e8a7ba690a920ad02ea6c71cab4415fc2089599"}1106{"id": "distance_band-001105", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,13), goal at (9,1).\n################\n#...........#.##\n#.###.#####.#.##\n#.....#.......##\n#.#.###.########\n#.#.....#.....##\n#.###.#.##.##.##\n#...#.#.......##\n#.###.#...##.###\n#G#...#.#.....##\n###.###.#####.##\n#...#...#.....##\n#.####.##.######\n#............A##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 3, "answer": "20 to 39 moves", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "71752bfde5d4d9ab66cbf4dade0259c397648a54a17fd34c3abef9cc7ca0cbd1"}1107{"id": "blocked_pair-001106", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[19,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"15,-1\":\"outside\",\"15,0\":\"wall\",\"15,1\":\"open\",\"15,2\":\"open\",\"15,3\":\"open\",\"15,4\":\"open\",\"15,5\":\"open\",\"15,6\":\"open\",\"15,7\":\"open\",\"16,-1\":\"outside\",\"16,0\":\"wall\",\"16,1\":\"wall\",\"16,2\":\"wall\",\"16,3\":\"wall\",\"16,4\":\"wall\",\"16,5\":\"wall\",\"16,6\":\"wall\",\"16,7\":\"wall\",\"17,-1\":\"outside\",\"17,0\":\"wall\",\"17,1\":\"open\",\"17,2\":\"open\",\"17,3\":\"open\",\"17,4\":\"open\",\"17,5\":\"open\",\"17,6\":\"open\",\"17,7\":\"open\",\"18,-1\":\"outside\",\"18,0\":\"wall\",\"18,1\":\"open\",\"18,2\":\"wall\",\"18,3\":\"wall\",\"18,4\":\"wall\",\"18,5\":\"wall\",\"18,6\":\"wall\",\"18,7\":\"wall\",\"19,-1\":\"outside\",\"19,0\":\"wall\",\"19,1\":\"open\",\"19,2\":\"wall\",\"19,3\":\"agent\",\"19,4\":\"open\",\"19,5\":\"open\",\"19,6\":\"open\",\"19,7\":\"open\",\"20,-1\":\"outside\",\"20,0\":\"wall\",\"20,1\":\"open\",\"20,2\":\"wall\",\"20,3\":\"wall\",\"20,4\":\"wall\",\"20,5\":\"wall\",\"20,6\":\"wall\",\"20,7\":\"wall\",\"21,-1\":\"outside\",\"21,0\":\"wall\",\"21,1\":\"open\",\"21,2\":\"open\",\"21,3\":\"open\",\"21,4\":\"open\",\"21,5\":\"open\",\"21,6\":\"open\",\"21,7\":\"open\",\"22,-1\":\"outside\",\"22,0\":\"wall\",\"22,1\":\"open\",\"22,2\":\"wall\",\"22,3\":\"wall\",\"22,4\":\"wall\",\"22,5\":\"wall\",\"22,6\":\"wall\",\"22,7\":\"wall\",\"23,-1\":\"outside\",\"23,0\":\"wall\",\"23,1\":\"open\",\"23,2\":\"wall\",\"23,3\":\"open\",\"23,4\":\"open\",\"23,5\":\"open\",\"23,6\":\"open\",\"23,7\":\"open\"}}", "question": "Are BOTH the cell one step east and the cell one step south blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "91fe1b47a4e79ccdb3ba771838a3403be476765cbc1aff19a8f5600207496bbd"}1108{"id": "plan_survives-001107", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 35, column 35. Rows increase to the south and columns to the east.\nRow 31: column 31 is open, column 32 is open, column 33 is wall, column 34 is open, column 35 is open, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze, column 39 is outside the maze\nRow 32: column 31 is open, column 32 is wall, column 33 is open, column 34 is open, column 35 is open, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze, column 39 is outside the maze\nRow 33: column 31 is open, column 32 is open, column 33 is open, column 34 is wall, column 35 is open, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze, column 39 is outside the maze\nRow 34: column 31 is open, column 32 is open, column 33 is open, column 34 is open, column 35 is wall, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze, column 39 is outside the maze\nRow 35: column 31 is wall, column 32 is open, column 33 is open, column 34 is open, column 35 is the agent, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze, column 39 is outside the maze\nRow 36: column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze, column 39 is outside the maze\nRow 37: column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze, column 39 is outside the maze\nRow 38: column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze, column 39 is outside the maze\nRow 39: column 31 is outside the maze, column 32 is outside the maze, column 33 is outside the maze, column 34 is outside the maze, column 35 is outside the maze, column 36 is outside the maze, column 37 is outside the maze, column 38 is outside the maze, column 39 is outside the maze", "question": "The agent will attempt this plan in order: west, north, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "6df952230fc2baad34e8e535ebc36b66d89b9170de76ae93ef6ab1626fd448bb"}1109{"id": "open_count-001108", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (0,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXX\nXXXXXXX\nXXXXXXX\n..#A.XX\n.#.#.XX\n.....XX\n#.#..XX", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 1, "answer": "1", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3ea04572c776a320053e1750f8c8052f9d5dfb86adcbc4a6addd2e88e5ffce60"}1110{"id": "goal_bearing-001109", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (5,5), goal at (1,5).\n#########\n#...#G..#\n#.#.###.#\n#.#.....#\n#.#####.#\n#....A#.#\n#.#.###.#\n#.......#\n#########", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "3be7f5ac1b24faac58e006242488ba3927bc2874f0357934bd9163978525502e"}1111{"id": "distance_band-001110", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (21,5), goal at (21,1).\n########################\n#.......#.............##\n#.#####.###.##..#.#.#.##\n#.#.......#.....#...#.##\n#.#.#####.####.##.###.##\n#...#...#.......#.....##\n#.#...#.#######.#.###.##\n#.#.#.#.#.......#...#.##\n#.###...#.###.#.#.###.##\n#.....#.......#.#.#...##\n#.#####.#.##.##.#...####\n#...#...#.......#.#...##\n#.#.#.###.#.#.#...###.##\n#.#.#...#.#.......#...##\n#.#.###.###.###.###.#.##\n#.#.#...............#.##\n#.#...###.#.#####.#.#.##\n#.....#...#.........#.##\n#.###...###.#.#.###.#.##\n#.....#...#...#.#.....##\n#####.###.#.#.#.#.###.##\n#G...A..#.....#...#...##\n########################\n########################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "970c2780ad3354164fc6f6eddbf72a757e777a4f5b2cd206d262ee541511940e"}1112{"id": "snake_safe-001111", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,6), travelling east, length 3.\n............\n............\n............\n............\n............\n............\n....ooH.....\n.....*......\n............\n............\n............\n............", "question": "If the snake moves west on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "8edaf76a00d389682dc4dd1c995cfc9f1cb74b2c29e7fb6f7acdddd154346c19"}1113{"id": "stochastic-001112", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (13,35); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#.XX\n...XX\n..AXX\n##.XX\n###XX", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.6, 0.4], "deterministic": false, "fingerprint": "c03d416be1d993a536988fec2b74b79b502f6ba5d9d405a4e802bcfcc90bc17f"}1114{"id": "snake_food-001113", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (7,11), travelling east, length 3.\n............\n............\n............\n............\n............\n............\n............\n.*.......ooH\n............\n............\n............\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "06fa267a571b5de0f8a2d1fd9e16f06f39882af292933c91941cbeb587fdacea"}1115{"id": "open_count-001114", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[10,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"7,6\":\"wall\",\"7,7\":\"open\",\"7,8\":\"open\",\"7,9\":\"open\",\"7,10\":\"wall\",\"7,11\":\"open\",\"7,12\":\"outside\",\"8,6\":\"wall\",\"8,7\":\"wall\",\"8,8\":\"open\",\"8,9\":\"open\",\"8,10\":\"open\",\"8,11\":\"open\",\"8,12\":\"outside\",\"9,6\":\"open\",\"9,7\":\"wall\",\"9,8\":\"open\",\"9,9\":\"open\",\"9,10\":\"wall\",\"9,11\":\"open\",\"9,12\":\"outside\",\"10,6\":\"open\",\"10,7\":\"open\",\"10,8\":\"open\",\"10,9\":\"agent\",\"10,10\":\"open\",\"10,11\":\"open\",\"10,12\":\"outside\",\"11,6\":\"open\",\"11,7\":\"open\",\"11,8\":\"open\",\"11,9\":\"open\",\"11,10\":\"open\",\"11,11\":\"open\",\"11,12\":\"outside\",\"12,6\":\"outside\",\"12,7\":\"outside\",\"12,8\":\"outside\",\"12,9\":\"outside\",\"12,10\":\"outside\",\"12,11\":\"outside\",\"12,12\":\"outside\",\"13,6\":\"outside\",\"13,7\":\"outside\",\"13,8\":\"outside\",\"13,9\":\"outside\",\"13,10\":\"outside\",\"13,11\":\"outside\",\"13,12\":\"outside\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 4, "answer": "4", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "d1beb3679c1ad971e230b21c16b6f8c014bfb24271f68ff9eb577de55044918c"}1116{"id": "only_way-001115", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 17, column 17. Rows increase to the south and columns to the east.\nRow 13: column 13 is open, column 14 is open, column 15 is open, column 16 is open, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze\nRow 14: column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze\nRow 15: column 13 is open, column 14 is open, column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is wall, column 21 is outside the maze\nRow 16: column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is outside the maze\nRow 17: column 13 is open, column 14 is open, column 15 is open, column 16 is wall, column 17 is the agent, column 18 is open, column 19 is open, column 20 is wall, column 21 is outside the maze\nRow 18: column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze\nRow 19: column 13 is open, column 14 is open, column 15 is open, column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is wall, column 21 is outside the maze\nRow 20: column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall, column 18 is wall, column 19 is wall, column 20 is wall, column 21 is outside the maze\nRow 21: column 13 is outside the maze, column 14 is outside the maze, column 15 is outside the maze, column 16 is outside the maze, column 17 is outside the maze, column 18 is outside the maze, column 19 is outside the maze, column 20 is outside the maze, column 21 is outside the maze", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "68b0c538dab246649d9ff0e29c04408910a2ae725eb7920cf1602fb890487e0c"}1117{"id": "reachable_within-001116", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,12), goal at (3,1).\n################\n#.............##\n#.#.#...###.#.##\n#G..#.#.....#.##\n###.#.###.###.##\n#...#.....#...##\n#.###.###.#.####\n#...#.....#.#.##\n###.#.#####.#.##\n#.........#.A.##\n#.#.#####.###.##\n#.#.....#.#...##\n#.###..##...####\n#.........#...##\n################\n################", "question": "Can the agent reach the goal in at most 24 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "efba9cb4eaebd024902b92168bacc6e8821970676a10fbbaef2a04753a175b6a"}1118{"id": "plan_survives-001117", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[7,6],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"4,3\":\"open\",\"4,4\":\"open\",\"4,5\":\"open\",\"4,6\":\"open\",\"4,7\":\"open\",\"4,8\":\"open\",\"4,9\":\"outside\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"wall\",\"5,6\":\"open\",\"5,7\":\"wall\",\"5,8\":\"wall\",\"5,9\":\"outside\",\"6,3\":\"wall\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"6,7\":\"wall\",\"6,8\":\"wall\",\"6,9\":\"outside\",\"7,3\":\"open\",\"7,4\":\"open\",\"7,5\":\"open\",\"7,6\":\"agent\",\"7,7\":\"open\",\"7,8\":\"wall\",\"7,9\":\"outside\",\"8,3\":\"wall\",\"8,4\":\"open\",\"8,5\":\"open\",\"8,6\":\"wall\",\"8,7\":\"open\",\"8,8\":\"open\",\"8,9\":\"outside\",\"9,3\":\"outside\",\"9,4\":\"outside\",\"9,5\":\"outside\",\"9,6\":\"outside\",\"9,7\":\"outside\",\"9,8\":\"outside\",\"9,9\":\"outside\",\"10,3\":\"outside\",\"10,4\":\"outside\",\"10,5\":\"outside\",\"10,6\":\"outside\",\"10,7\":\"outside\",\"10,8\":\"outside\",\"10,9\":\"outside\"}}", "question": "The agent will attempt this plan in order: north, south, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "7d2c5aeefc4e612828b2fa2774115d7adde7bd1fab2acfb28c420f732c4de5bd"}1119{"id": "stochastic-001118", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (15,11); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#..##....\n.........\n......#.#\n....#.#..\n.#..A...#\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX", "question": "The actuator executes the intended move with probability 0.60; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.2666666666666667, 0.7333333333333333], "deterministic": false, "fingerprint": "832cc5e0e194ffd6647801c59a54eaec061c8990d4beb744a1654b934513c591"}1120{"id": "move_legal-001119", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 1, column 7. Rows increase to the south and columns to the east.\nRow -2: column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze\nRow -1: column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze\nRow 0: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze\nRow 1: column 4 is open, column 5 is open, column 6 is open, column 7 is the agent, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze\nRow 2: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze\nRow 3: column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze\nRow 4: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze", "question": "The agent wants to move west. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "ece807238dcf30600790b332c4dffea8bdcb3ee1d794b296ec50a2297ba08cb5"}1121{"id": "open_count-001120", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[15,19],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"11,15\":\"wall\",\"11,16\":\"open\",\"11,17\":\"open\",\"11,18\":\"open\",\"11,19\":\"wall\",\"11,20\":\"wall\",\"11,21\":\"outside\",\"11,22\":\"outside\",\"11,23\":\"outside\",\"12,15\":\"wall\",\"12,16\":\"open\",\"12,17\":\"open\",\"12,18\":\"wall\",\"12,19\":\"open\",\"12,20\":\"wall\",\"12,21\":\"outside\",\"12,22\":\"outside\",\"12,23\":\"outside\",\"13,15\":\"wall\",\"13,16\":\"open\",\"13,17\":\"wall\",\"13,18\":\"open\",\"13,19\":\"open\",\"13,20\":\"open\",\"13,21\":\"outside\",\"13,22\":\"outside\",\"13,23\":\"outside\",\"14,15\":\"wall\",\"14,16\":\"open\",\"14,17\":\"open\",\"14,18\":\"open\",\"14,19\":\"wall\",\"14,20\":\"open\",\"14,21\":\"outside\",\"14,22\":\"outside\",\"14,23\":\"outside\",\"15,15\":\"open\",\"15,16\":\"open\",\"15,17\":\"open\",\"15,18\":\"open\",\"15,19\":\"agent\",\"15,20\":\"open\",\"15,21\":\"outside\",\"15,22\":\"outside\",\"15,23\":\"outside\",\"16,15\":\"open\",\"16,16\":\"open\",\"16,17\":\"wall\",\"16,18\":\"wall\",\"16,19\":\"open\",\"16,20\":\"open\",\"16,21\":\"outside\",\"16,22\":\"outside\",\"16,23\":\"outside\",\"17,15\":\"open\",\"17,16\":\"open\",\"17,17\":\"open\",\"17,18\":\"open\",\"17,19\":\"open\",\"17,20\":\"open\",\"17,21\":\"outside\",\"17,22\":\"outside\",\"17,23\":\"outside\",\"18,15\":\"open\",\"18,16\":\"open\",\"18,17\":\"open\",\"18,18\":\"open\",\"18,19\":\"open\",\"18,20\":\"open\",\"18,21\":\"outside\",\"18,22\":\"outside\",\"18,23\":\"outside\",\"19,15\":\"open\",\"19,16\":\"open\",\"19,17\":\"open\",\"19,18\":\"open\",\"19,19\":\"wall\",\"19,20\":\"wall\",\"19,21\":\"outside\",\"19,22\":\"outside\",\"19,23\":\"outside\"}}", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 3, "answer": "3", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "b68d3289e4d22dac49850f3bc89c3411fd7714e7678b3fa35bbeab6ac9e47d15"}1122{"id": "snake_safe-001121", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 14x14 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,7), travelling north, length 3.\n..............\n..............\n..............\n..............\n..............\n..............\n.......H......\n......oo......\n..............\n..............\n..............\n..............\n..............\n.*............", "question": "If the snake moves west on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "58aecbcbc23e5bea232b94a016f2aba3d57584096c6e0600a4e84bb07ce1d217"}1123{"id": "plan_survives-001122", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[7,10],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"4,7\":\"wall\",\"4,8\":\"wall\",\"4,9\":\"wall\",\"4,10\":\"wall\",\"4,11\":\"open\",\"4,12\":\"wall\",\"4,13\":\"wall\",\"5,7\":\"open\",\"5,8\":\"open\",\"5,9\":\"open\",\"5,10\":\"wall\",\"5,11\":\"open\",\"5,12\":\"wall\",\"5,13\":\"open\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"wall\",\"6,10\":\"wall\",\"6,11\":\"open\",\"6,12\":\"wall\",\"6,13\":\"open\",\"7,7\":\"open\",\"7,8\":\"wall\",\"7,9\":\"open\",\"7,10\":\"agent\",\"7,11\":\"open\",\"7,12\":\"wall\",\"7,13\":\"open\",\"8,7\":\"open\",\"8,8\":\"wall\",\"8,9\":\"wall\",\"8,10\":\"wall\",\"8,11\":\"wall\",\"8,12\":\"wall\",\"8,13\":\"open\",\"9,7\":\"open\",\"9,8\":\"wall\",\"9,9\":\"open\",\"9,10\":\"open\",\"9,11\":\"open\",\"9,12\":\"wall\",\"9,13\":\"open\",\"10,7\":\"open\",\"10,8\":\"wall\",\"10,9\":\"open\",\"10,10\":\"wall\",\"10,11\":\"wall\",\"10,12\":\"wall\",\"10,13\":\"open\"}}", "question": "The agent will attempt this plan in order: north, south, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "1e16b9903b748b5b03dd07862851bde7b55be400c1e8715755ec9b99fd871c28"}1124{"id": "onward-001123", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 30, column 33. Rows increase to the south and columns to the east.\nRow 26: column 29 is open, column 30 is wall, column 31 is wall, column 32 is wall, column 33 is open, column 34 is wall, column 35 is wall, column 36 is outside the maze, column 37 is outside the maze\nRow 27: column 29 is open, column 30 is wall, column 31 is open, column 32 is open, column 33 is open, column 34 is wall, column 35 is wall, column 36 is outside the maze, column 37 is outside the maze\nRow 28: column 29 is open, column 30 is wall, column 31 is open, column 32 is wall, column 33 is open, column 34 is wall, column 35 is wall, column 36 is outside the maze, column 37 is outside the maze\nRow 29: column 29 is open, column 30 is open, column 31 is open, column 32 is open, column 33 is open, column 34 is wall, column 35 is wall, column 36 is outside the maze, column 37 is outside the maze\nRow 30: column 29 is wall, column 30 is open, column 31 is wall, column 32 is wall, column 33 is the agent, column 34 is wall, column 35 is wall, column 36 is outside the maze, column 37 is outside the maze\nRow 31: column 29 is open, column 30 is open, column 31 is open, column 32 is open, column 33 is open, column 34 is wall, column 35 is wall, column 36 is outside the maze, column 37 is outside the maze\nRow 32: column 29 is wall, column 30 is wall, column 31 is open, column 32 is wall, column 33 is open, column 34 is wall, column 35 is wall, column 36 is outside the maze, column 37 is outside the maze\nRow 33: column 29 is open, column 30 is open, column 31 is open, column 32 is wall, column 33 is open, column 34 is wall, column 35 is wall, column 36 is outside the maze, column 37 is outside the maze\nRow 34: column 29 is wall, column 30 is wall, column 31 is wall, column 32 is wall, column 33 is wall, column 34 is wall, column 35 is wall, column 36 is outside the maze, column 37 is outside the maze", "question": "Suppose the agent moves one cell north. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "6c9c3f124845bc32628e74844515c756d4a6421d471afe1d1aa59bfa448bedb9"}1125{"id": "open_count-001124", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (25,15); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.....#...\n#########\n...#.....\n.#.#.###.\n....A....\n#########\n#########\nXXXXXXXXX\nXXXXXXXXX", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 3, "answer": "3", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "6bde8712ee20838866c25d33c8ff864f7401458625948caf54da8b79c55bcb0e"}1126{"id": "only_way-001125", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (29,15); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#.#.#.#\n....#..\n#######\n..#A...\n#.#####\n#......\n#######", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "5c147f5549e492f24f5c361cf16e437e08a2772e76fa4ed66ba58c3cf4bba9f7"}1127{"id": "first_move-001126", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 9x9. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,1), goal at (3,1).\n#########\n#A....#.#\n#####...#\n#G#.....#\n#.#.#.#.#\n#.#.#...#\n#.#.###.#\n#.......#\n#########", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 1, "answer": "east", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "c61adccb03239edb62fca955e20e09aedfc46a3f88f9be1c5f95c56171f06d53"}1128{"id": "goal_bearing-001127", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,12), goal at (3,3).\n################\n#.#.....#.....##\n#.#.#..##.######\n#..G#.........##\n#.###.#.###.#.##\n#.....#...#.#.##\n#####.###.###.##\n#.......#...#.##\n#.#.###.###.#.##\n#.#.#.........##\n#.#.#.#.#####.##\n#...#.#.#.....##\n###...###.######\n#...#.......A.##\n################\n################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 5, "answer": "north-west", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "b1e3b0aa63caa5d25dc2e3036f7faef99684938c192b5d11f65ca938c15492d3"}1129{"id": "snake_food-001128", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (5,2), travelling north, length 3.\n..........\n..*.......\n..........\n..........\n..........\n..H.......\n..oo......\n..........\n..........\n..........", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "249091be1063f332eb8cfa959234e64c964a63c01da9d2ed79244f84bac5aed9"}1130{"id": "move_legal-001129", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[3,4],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"1,6\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"2,6\":\"wall\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"agent\",\"3,5\":\"open\",\"3,6\":\"open\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"4,5\":\"wall\",\"4,6\":\"wall\",\"5,2\":\"open\",\"5,3\":\"open\",\"5,4\":\"open\",\"5,5\":\"open\",\"5,6\":\"open\"}}", "question": "If the agent attempts one cell north (row-1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "051f0d0242f02bf4b98aac9a164f9b3b136f474f62ad4932a893a329b3ac57d8"}1131{"id": "plan_survives-001130", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (5,11); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n...#.\n.###.\n.#A..\n#####\n.....", "question": "The agent will attempt this plan in order: south, west, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "d49dd1f2d2fafab11a324b8cd89212c1b1a51feb1c4a1785aebb3b5710ff2f89"}1132{"id": "reachable_within-001131", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,6), goal at (3,11).\n################\n#.#...........##\n#.#.#########.##\n#.#.#......G#.##\n#.#.#.#######.##\n#.#.#.........##\n#.#.############\n#.#...........##\n#.#.#########.##\n#.#.#.......#.##\n#.#.#######.#.##\n#.#.........#.##\n#.###########.##\n#.....A.......##\n################\n################", "question": "Can the agent reach the goal in at most 63 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "fadb96b2f2b8c88c9b811add904d5ad01ce1a0c21c3dc947435e0957b1a2ad13"}1133{"id": "goal_bearing-001132", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (6,1), goal at (9,13).\n################\n#.......#.....##\n#.###.###.#.#.##\n#.#...#...#...##\n#.#.#.#.#.###.##\n#...#...#.#...##\n#A#.#.###.#.#.##\n#.#.#.#.#.....##\n#.#.#.#.#.###.##\n#.#.#.......#G##\n#.#.#####.#.#.##\n#.....#.....#.##\n#.###.#.#.#.#.##\n#.......#.....##\n################\n################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 6, "answer": "south-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "b0cd47d6b30ff027c1cfad23a12292a34f81ce487a78f47fa74fdef0e617c885"}1134{"id": "move_legal-001133", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 22, column 31. Rows increase to the south and columns to the east.\nRow 19: column 28 is wall, column 29 is open, column 30 is open, column 31 is open, column 32 is wall, column 33 is open, column 34 is wall\nRow 20: column 28 is wall, column 29 is wall, column 30 is wall, column 31 is wall, column 32 is wall, column 33 is open, column 34 is wall\nRow 21: column 28 is open, column 29 is open, column 30 is wall, column 31 is open, column 32 is open, column 33 is open, column 34 is wall\nRow 22: column 28 is wall, column 29 is open, column 30 is wall, column 31 is the agent, column 32 is wall, column 33 is open, column 34 is wall\nRow 23: column 28 is wall, column 29 is open, column 30 is wall, column 31 is open, column 32 is wall, column 33 is open, column 34 is wall\nRow 24: column 28 is wall, column 29 is open, column 30 is wall, column 31 is wall, column 32 is wall, column 33 is open, column 34 is wall\nRow 25: column 28 is wall, column 29 is open, column 30 is wall, column 31 is open, column 32 is open, column 33 is open, column 34 is wall", "question": "Is the cell one step west of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "d9875f1e9afd685ae67a981ba8b7f2c9e0655d92abe7909370da33791e82ad04"}1135{"id": "first_move-001134", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (15,1), goal at (3,1).\n#.##..#...###...##..........\n.....#.....#.#..#...###....#\n..#..#..#.#....#.#..###..#..\n#G#.....##.#.#.#.#..####..#.\n#######..#....#......##..#.#\n######.......###.##.###....#\n#######...#.......#.##..####\n#######.#####....#..#.######\n######....#...#.#........###\n#####...#.#....###.......###\n######..#..#..#######.#..###\n########..##.########.##..#.\n##.######..#.####.###.......\n#..######.#..###..#..#.#....\n....####.####..#.###...###.#\n.A.#####.#..#.......#....##.\n..#####.....###.#...#...#...\n....###.##..###.....#.#.##..\n#...#.......#####...#...##..\n...#..#...##..#.#.....####..\n.....#.#..#..#.........##...\n#...#..#..##..#......#####..\n...#..#......##.#.##........\n........#####.#......#####.#\n....#..#..###...#.#.#..#....\n...##...#.##...##..##..#...#\n.#.#......#.........#..#...#\n.##....#.....#..#.#..#......", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "4907b2d49f4fc78d4b887db843783c17a0c75a77a1740116d14c1fdbbb62f524"}1136{"id": "onward-001135", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 8, column 9. Rows increase to the south and columns to the east.\nRow 5: column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 6: column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 7: column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 8: column 6 is wall, column 7 is open, column 8 is wall, column 9 is the agent, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 9: column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 10: column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is outside the maze\nRow 11: column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is wall, column 12 is outside the maze", "question": "Suppose the agent moves one cell north. From that new cell, would a further step north be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "d688b613849c03a827dce7f18f4842e5d3d84907737600657c0fca275d8a5b0f"}1137{"id": "first_move-001136", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (10,1), goal at (0,3).\n...G#.#...#.\n..##....#.#.\n..#.........\n#......##...\n#.....##..##\n..##.....#.#\n###.........\n##.##.#...##\n#..#.#......\n........#..#\n.A#.#.###...\n...###....#.", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "d63169907bcd3bed4437220a5033c187734ec0590ce3ddc7dc8e65e34e4283db"}1138{"id": "plan_survives-001137", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (20,19); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n....X\n..#.X\n..A.X\nXXXXX\nXXXXX", "question": "The agent will attempt this plan in order: west, east, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "8156ac2401e93f7c4a295cd6366b7d4da67038c151ba81dd27f1bfd64310b501"}1139{"id": "blocked_pair-001138", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (21,15); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#######\n#......\n#.#####\n#.#A..#\n#.###.#\n#.....#\n#######", "question": "Are BOTH the cell one step east and the cell one step south blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "5e4defe471a8944ac1d2e960b2b70651232d166003a972ce402954faf78e7f51"}1140{"id": "move_two_step-001139", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (2,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXX\nXX.##..\nXX..###\nXX.A.#.\nXX....#\nXX...##\nXX.#...", "question": "The agent plans to move north, then east. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "b839361462209a3aa99f9ffacea1a14c839996613ed7d8392660460d8493d1e3"}1141{"id": "only_way-001140", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 5, column 7. Rows increase to the south and columns to the east.\nRow 2: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is wall, column 10 is wall\nRow 3: column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is open, column 9 is open, column 10 is wall\nRow 4: column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open, column 10 is wall\nRow 5: column 4 is open, column 5 is open, column 6 is open, column 7 is the agent, column 8 is wall, column 9 is open, column 10 is wall\nRow 6: column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open, column 10 is wall\nRow 7: column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall\nRow 8: column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a3265cc80b6777a87b1a7486500c7dcbdc065d2ee12a53be4be60c36b0301b88"}1142{"id": "only_way-001141", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[13,13],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"9,9\":\"open\",\"9,10\":\"open\",\"9,11\":\"open\",\"9,12\":\"open\",\"9,13\":\"open\",\"9,14\":\"wall\",\"9,15\":\"wall\",\"9,16\":\"outside\",\"9,17\":\"outside\",\"10,9\":\"open\",\"10,10\":\"wall\",\"10,11\":\"wall\",\"10,12\":\"wall\",\"10,13\":\"open\",\"10,14\":\"wall\",\"10,15\":\"wall\",\"10,16\":\"outside\",\"10,17\":\"outside\",\"11,9\":\"open\",\"11,10\":\"wall\",\"11,11\":\"open\",\"11,12\":\"open\",\"11,13\":\"open\",\"11,14\":\"wall\",\"11,15\":\"wall\",\"11,16\":\"outside\",\"11,17\":\"outside\",\"12,9\":\"open\",\"12,10\":\"wall\",\"12,11\":\"wall\",\"12,12\":\"wall\",\"12,13\":\"wall\",\"12,14\":\"wall\",\"12,15\":\"wall\",\"12,16\":\"outside\",\"12,17\":\"outside\",\"13,9\":\"open\",\"13,10\":\"open\",\"13,11\":\"open\",\"13,12\":\"open\",\"13,13\":\"agent\",\"13,14\":\"wall\",\"13,15\":\"wall\",\"13,16\":\"outside\",\"13,17\":\"outside\",\"14,9\":\"wall\",\"14,10\":\"wall\",\"14,11\":\"wall\",\"14,12\":\"wall\",\"14,13\":\"wall\",\"14,14\":\"wall\",\"14,15\":\"wall\",\"14,16\":\"outside\",\"14,17\":\"outside\",\"15,9\":\"wall\",\"15,10\":\"wall\",\"15,11\":\"wall\",\"15,12\":\"wall\",\"15,13\":\"wall\",\"15,14\":\"wall\",\"15,15\":\"wall\",\"15,16\":\"outside\",\"15,17\":\"outside\",\"16,9\":\"outside\",\"16,10\":\"outside\",\"16,11\":\"outside\",\"16,12\":\"outside\",\"16,13\":\"outside\",\"16,14\":\"outside\",\"16,15\":\"outside\",\"16,16\":\"outside\",\"16,17\":\"outside\",\"17,9\":\"outside\",\"17,10\":\"outside\",\"17,11\":\"outside\",\"17,12\":\"outside\",\"17,13\":\"outside\",\"17,14\":\"outside\",\"17,15\":\"outside\",\"17,16\":\"outside\",\"17,17\":\"outside\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a95c561a52e92a0ac06a96d83a97894dcf40b04d3aeddad6431c28c0543d706b"}1143{"id": "move_legal-001142", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "Agent coordinate: (9,7); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#####.#\n....#.#\n###.#.#\n..#A..#\n#######\n#######\nXXXXXXX", "question": "The agent wants to move west. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "09b0ac55cf776f9dc6927ac0b20e85c66954e9e5f1f9601775b8e0cf372f9182"}1144{"id": "plan_survives-001143", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (35,15); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n...####..\n..#####..\n......###\n..#..##..\n..#.A##..\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX\nXXXXXXXXX", "question": "The agent will attempt this plan in order: north, north, south. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "a46f5b602dfdca38c9df7b94864e6a8cad32c5b08ce80c135e36667ab3ecf974"}1145{"id": "onward-001144", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,27],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-3,23\":\"outside\",\"-3,24\":\"outside\",\"-3,25\":\"outside\",\"-3,26\":\"outside\",\"-3,27\":\"outside\",\"-3,28\":\"outside\",\"-3,29\":\"outside\",\"-3,30\":\"outside\",\"-3,31\":\"outside\",\"-2,23\":\"outside\",\"-2,24\":\"outside\",\"-2,25\":\"outside\",\"-2,26\":\"outside\",\"-2,27\":\"outside\",\"-2,28\":\"outside\",\"-2,29\":\"outside\",\"-2,30\":\"outside\",\"-2,31\":\"outside\",\"-1,23\":\"outside\",\"-1,24\":\"outside\",\"-1,25\":\"outside\",\"-1,26\":\"outside\",\"-1,27\":\"outside\",\"-1,28\":\"outside\",\"-1,29\":\"outside\",\"-1,30\":\"outside\",\"-1,31\":\"outside\",\"0,23\":\"open\",\"0,24\":\"wall\",\"0,25\":\"wall\",\"0,26\":\"open\",\"0,27\":\"open\",\"0,28\":\"outside\",\"0,29\":\"outside\",\"0,30\":\"outside\",\"0,31\":\"outside\",\"1,23\":\"open\",\"1,24\":\"wall\",\"1,25\":\"wall\",\"1,26\":\"wall\",\"1,27\":\"agent\",\"1,28\":\"outside\",\"1,29\":\"outside\",\"1,30\":\"outside\",\"1,31\":\"outside\",\"2,23\":\"open\",\"2,24\":\"open\",\"2,25\":\"wall\",\"2,26\":\"open\",\"2,27\":\"open\",\"2,28\":\"outside\",\"2,29\":\"outside\",\"2,30\":\"outside\",\"2,31\":\"outside\",\"3,23\":\"open\",\"3,24\":\"open\",\"3,25\":\"open\",\"3,26\":\"open\",\"3,27\":\"wall\",\"3,28\":\"outside\",\"3,29\":\"outside\",\"3,30\":\"outside\",\"3,31\":\"outside\",\"4,23\":\"open\",\"4,24\":\"wall\",\"4,25\":\"wall\",\"4,26\":\"open\",\"4,27\":\"open\",\"4,28\":\"outside\",\"4,29\":\"outside\",\"4,30\":\"outside\",\"4,31\":\"outside\",\"5,23\":\"open\",\"5,24\":\"open\",\"5,25\":\"wall\",\"5,26\":\"wall\",\"5,27\":\"open\",\"5,28\":\"outside\",\"5,29\":\"outside\",\"5,30\":\"outside\",\"5,31\":\"outside\"}}", "question": "Suppose the agent moves one cell north. From that new cell, would a further step west be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "e8c964c9253f51f22588fc52796847f06ba44513bdf3f5de7736ea9a86d53c3a"}1146{"id": "only_way-001145", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 7, column 13. Rows increase to the south and columns to the east.\nRow 5: column 11 is open, column 12 is open, column 13 is open, column 14 is wall, column 15 is wall\nRow 6: column 11 is wall, column 12 is wall, column 13 is wall, column 14 is wall, column 15 is wall\nRow 7: column 11 is open, column 12 is wall, column 13 is the agent, column 14 is wall, column 15 is wall\nRow 8: column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall\nRow 9: column 11 is open, column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 2, "answer": "south", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "907aaab244903f3e590ef27462ff5882dcdf7c31d29a51a67d7bb7b1a5ac4d82"}1147{"id": "move_two_step-001146", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,5],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-2,5\":\"outside\",\"-2,6\":\"outside\",\"-2,7\":\"outside\",\"-2,8\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"-1,6\":\"outside\",\"-1,7\":\"outside\",\"-1,8\":\"outside\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"0,6\":\"wall\",\"0,7\":\"wall\",\"0,8\":\"wall\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"agent\",\"1,6\":\"wall\",\"1,7\":\"open\",\"1,8\":\"wall\",\"2,2\":\"open\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"wall\",\"2,7\":\"open\",\"2,8\":\"wall\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"3,8\":\"wall\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"open\",\"4,7\":\"open\",\"4,8\":\"wall\"}}", "question": "The agent plans to move east, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "63487b6674e9f0550d0386ed3511ed66feb143fd0e2e6f3a556990aa731bc678"}1148{"id": "distance_band-001147", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (20,5), goal at (16,0).\n#..#...#...##.##########\n.#.#.....#....#####.##.#\n.#....#....#....#...##.#\n......#.##..#.....#.....\n.#...#.#.#..........#..#\n#...#....#..###.#.###..#\n..#....#.....###........\n..##.#..#..#.#...#...#.#\n##........#..#....##....\n.#....#....#..##....#.#.\n.##...#.......#.#..#..#.\n.....####.#......##.#...\n##...##...........#..#..\n#...#..##.#.#...####...#\n..#.#........#.#####.#..\n###........#.########...\nG##..###....#########...\n.###.###....#########.#.\n...#.......#.#.#####....\n.#.#....##.....####.#...\n##...A...#.##........#..\n###..#.#.##...##..##....\n###...##.##.....#.##.#..\n###.#...##......#..##..#", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 1, "answer": "5 to 9 moves", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "74565a91efef0e2332b929ba35280d70876135d241f083fb43d1cc672eff4149"}1149{"id": "goal_bearing-001148", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (4,5), goal at (1,5).\n############################\n#....G#.....#...#.......#.##\n###.###.###.#.#.#.#####.#.##\n#...#...#.#...#.#...#.....##\n#.###A###.#####.#.#.#####.##\n#...#.#.....#...#.#...#...##\n###.###.#####.###.###.######\n#...#.......#.#.....#.....##\n#.###.#####.#.#.#######.#.##\n#.......#...#.#.#.....#.#.##\n#########.#.#.###.###.#.#.##\n#.........#.#.....#.#.#.#.##\n#.#########.#######.#.###.##\n#.#.....#.........#.#.....##\n#.###.###.#######.#.#####.##\n#...#.....#...#.......#...##\n###.#.#####.#.###.#####.####\n#...#.#.....#...#.#...#...##\n#.###.#.#######.###.#.###.##\n#...#.#.....#.#.....#.....##\n#.#.#######.#.##############\n#.#.#.....#.#.....#.......##\n#.#.#.###.#.#.###.#.#.###.##\n#.#...#...#.#.#.#.#.#...#.##\n#.#####.###.#.#.#.#####.#.##\n#.....#.......#.........#.##\n############################\n############################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "da0fd18f99ca70084f41fc3a3ba14fbe2a713bff0689c554bce6c5e6a7388309"}1150{"id": "goal_bearing-001149", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (20,9), goal at (1,15).\n############################\n#.............#G..#.......##\n#######.#.#######.#.#.###.##\n#.......#.#.......#.#.#.#.##\n#.#########.#########.#.#.##\n#...#.......#...#.....#.#.##\n###.#.#######.###.#.###.#.##\n#...#.#...#.......#.#...#.##\n#.###.#.#.#.#######.#.###.##\n#.#...#.#...#.#.....#.....##\n#.#.###.#####.#.#####.######\n#.#...#.#.....#.....#.#...##\n#.###.#.###.#.#####.#.###.##\n#...#.#...#.#...#...#.#...##\n#.#.#.###.#.#.###.###.#.#.##\n#.#.#...#...#.#...#.....#.##\n#.#.###.#####.#.#########.##\n#.#...#.#.....#.#.....#...##\n#.#.###.#.#####.#.###.#.####\n#.#.#...#.#...#.....#.#...##\n#.#.#.###A#.#.#####.#.######\n#.#...#...#.#...#...#.....##\n#.#########.###.#########.##\n#...#...#...#...#.....#...##\n###.#.#.#.###.###.###.#.#.##\n#.....#...#.......#.....#.##\n############################\n############################", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 4, "answer": "north-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "5fca3568fe46aca070256a8e67a3950694d0129890a9ca564e02128f3d4e16e6"}1151{"id": "first_move-001150", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (11,7), goal at (1,7).\n################\n#.....#G#.....##\n#.#.###.#.###.##\n#.#.....#...#.##\n#.###########.##\n#.#.........#.##\n#.#.#######.#.##\n#.#...#...#.#.##\n#.###.###.#.#.##\n#.....#...#...##\n#######.#.###.##\n#......A#.#...##\n#.###.#####.####\n#...#.........##\n################\n################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "24a052c9a120b75c03569a2a8012d0dab3835b311079b32e8c4f94b4946ae74f"}1152{"id": "distance_band-001151", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (7,7), goal at (1,1).\n################\n#G............##\n#.###.#####.#.##\n#.#...#.....#.##\n#.#.###.#.#.#.##\n#.......#.....##\n#####.###.#.#.##\n#...#..A..#...##\n#.#.#.#.#.#.####\n#.#...#.....#.##\n#.#########.#.##\n#.......#...#.##\n#.#####.#.###.##\n#.....#.......##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "ce18f12d6b9f2e671b9c9a708efc63cfbadc00403ef5e4b0dc9019e01257b3f4"}1153{"id": "reachable_within-001152", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,7), goal at (7,13).\n################\n#.............##\n#.#.#.#.#.#.#.##\n#.#.#.....#.#.##\n#.#.#.#.###.#.##\n#...#...#.#.#.##\n#.###.###.#.#.##\n#.#...#.....#G##\n#.#.###.###.####\n#...#.#.#.....##\n#.###.#.#.#.#.##\n#...#.#.#...#.##\n###.#...#.#.#.##\n#.....#A..#...##\n################\n################", "question": "Can the agent reach the goal in at most 23 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "d5eae9dbe513e924be39806351b440104df4296c438690c834b19e17e63917b6"}1154{"id": "move_legal-001153", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[6,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"2,6\":\"wall\",\"2,7\":\"open\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"3,6\":\"wall\",\"3,7\":\"open\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"4,7\":\"open\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"wall\",\"5,7\":\"open\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"agent\",\"6,4\":\"wall\",\"6,5\":\"open\",\"6,6\":\"wall\",\"6,7\":\"open\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"wall\",\"7,3\":\"open\",\"7,4\":\"wall\",\"7,5\":\"open\",\"7,6\":\"wall\",\"7,7\":\"open\",\"8,-1\":\"outside\",\"8,0\":\"wall\",\"8,1\":\"open\",\"8,2\":\"wall\",\"8,3\":\"wall\",\"8,4\":\"wall\",\"8,5\":\"open\",\"8,6\":\"wall\",\"8,7\":\"wall\",\"9,-1\":\"outside\",\"9,0\":\"wall\",\"9,1\":\"open\",\"9,2\":\"open\",\"9,3\":\"open\",\"9,4\":\"open\",\"9,5\":\"open\",\"9,6\":\"open\",\"9,7\":\"open\",\"10,-1\":\"outside\",\"10,0\":\"wall\",\"10,1\":\"wall\",\"10,2\":\"wall\",\"10,3\":\"wall\",\"10,4\":\"wall\",\"10,5\":\"wall\",\"10,6\":\"wall\",\"10,7\":\"wall\"}}", "question": "The agent wants to move east. Is that move legal -- destination open and in bounds?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "68758479062bc7097cd348712446fce6809b01da836ec9dc528342a2c8b1a758"}1155{"id": "first_move-001154", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (13,7), goal at (3,1).\n################\n#.............##\n#.###########.##\n#G#.........#.##\n###.#######.#.##\n#.#.#.....#.#.##\n#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.##\n#.#.#.#.###.#.##\n#.#.#.#.#...#.##\n#.#.#.#.#.#.#.##\n#.#.#.#.#.#.#.##\n#.#.###.#.###.##\n#......A#.....##\n################\n################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "37ea7d78d1cfa7815bcc9bf63fb84a87584993a2f51a4514d83abddb660da24f"}1156{"id": "reachable_within-001155", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (10,1), goal at (9,5).\n################\n#.............##\n#.###########.##\n#.#.........#.##\n#.#########.#.##\n#...........#.##\n###########.#.##\n#.#.......#.#.##\n#.#.#####.#.#.##\n#.#.#G....#.#.##\n#A#.#######.#.##\n#.#.........#.##\n#.###########.##\n#.............##\n################\n################", "question": "Can the agent reach the goal in at most 80 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "fb2174312eb64d0d840c7ca33cd53650e7c621d3b8ec096cb7a2803162c77091"}1157{"id": "cell_kind-001156", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (8,5); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n....#..\n#######\n#...#..\n###A#.#\n..#.#.#\n###.#.#\n#...#..", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "7186295283fc1c537dc06c21dcf88dfe198231d8a2a51c3763d9b63e91dfdb45"}1158{"id": "plan_survives-001157", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (1,2); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXX\n#####\n#.A..\n#####\n#....", "question": "The agent will attempt this plan in order: east, east, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 0, "answer": "the plan completes; every step is legal", "answer_distribution": [1.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "bfe0278cc4695782ef185de6ea1df43e1dd13b6ae432d6b028f01d5493922b0c"}1159{"id": "snake_food-001158", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (6,8), travelling east, length 3.\n............\n............\n............\n............\n............\n............\n.......oH...\n.......o....\n............\n........*...\n............\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 1, "answer": "south", "answer_distribution": [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "33b9b015d961a96a1eeea6d6c141cb539cb658eb8c822519abe3efd5cf4790f4"}1160{"id": "move_two_step-001159", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 25, column 2. Rows increase to the south and columns to the east.\nRow 23: column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open\nRow 24: column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall\nRow 25: column 0 is wall, column 1 is open, column 2 is the agent, column 3 is open, column 4 is open\nRow 26: column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall\nRow 27: column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall", "question": "The agent plans to move east, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "8b69f983120731b0c348e5a4918d21a941b29f84c14ae328147f3900183d64d9"}1161{"id": "reachable_within-001160", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (1,5), goal at (9,17).\n########################\n#....A....#.#.........##\n#.#######.#.#.#######.##\n#.#.....#.#.#.#.....#.##\n#.#####.#.#.#.#.###.#.##\n#.#.....#.#.#.#...#.#.##\n#.#.###.#.#.#.###.#.#.##\n#.#.#.#.#.#.#...#.#.#.##\n#.#.#.#.#.#.###.#.#.#.##\n#.#.#.#.#.#...#.#G#.#.##\n#.#.#.#.#.#.###.###.#.##\n#.#.#.#.#.#.........#.##\n#.#.#.#.#############.##\n#.#.#.................##\n#.#.#################.##\n#.#...........#.....#.##\n#.#.#########.#.###.#.##\n#.#.#.......#.#.#.#.#.##\n#.#.#######.#.#.#.#.#.##\n#.#.........#.#.#...#.##\n#.###########.#.#####.##\n#.............#.......##\n########################\n########################", "question": "Can the agent reach the goal in at most 141 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "9d6428a864a9a045173581d7abfafff55f45e4892d269f225ade153c12976b49"}1162{"id": "onward-001161", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[4,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"open\",\"1,5\":\"open\",\"1,6\":\"open\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"open\",\"2,6\":\"wall\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\",\"3,4\":\"wall\",\"3,5\":\"open\",\"3,6\":\"wall\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"agent\",\"4,4\":\"wall\",\"4,5\":\"open\",\"4,6\":\"wall\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\",\"5,4\":\"wall\",\"5,5\":\"open\",\"5,6\":\"wall\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"open\",\"6,4\":\"wall\",\"6,5\":\"wall\",\"6,6\":\"wall\",\"7,0\":\"wall\",\"7,1\":\"open\",\"7,2\":\"wall\",\"7,3\":\"open\",\"7,4\":\"open\",\"7,5\":\"open\",\"7,6\":\"open\"}}", "question": "Suppose the agent moves one cell north. From that new cell, would a further step north be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "e73fbc9bbd151badf698b3516d69ff982364097c94704ca352a692b780903d57"}1163{"id": "reachable_within-001162", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (9,1), goal at (7,13).\n################\n#...........#.##\n#.#######.#.#.##\n#...#.....#...##\n#.#.#.#.#.###.##\n#...#.#...#...##\n#.#.#.#####.####\n#...#.....#..G##\n###.#####.######\n#A#.#...#.....##\n#.#.#.###.###.##\n#.#...#.....#.##\n#.#.#.###.#.#.##\n#.........#...##\n################\n################", "question": "Can the agent reach the goal in at most 39 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "75ffe642344b4964c9caefe7866c49b78199ebd889883a927bee01f334ba9144"}1164{"id": "snake_safe-001163", "family": "snake_safe", "environment": "snake", "seen_in_training": true, "state": "Snake game on a 10x10 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (4,5), travelling west, length 3.\n..........\n..........\n..........\n..........\n.....Ho...\n......o*..\n..........\n..........\n..........\n..........", "question": "If the snake moves north on the next tick, does it stay alive -- no wall and no collision with its own body?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "4037b36ac35d22b234be1a930e21f40cdf62a51d2c950bcf3d8369c1840f7f6f"}1165{"id": "goal_bearing-001164", "family": "goal_bearing", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,3), goal at (9,7).\n############\n#.....#...##\n#.#.#.#.####\n#.#A#.....##\n#.#.#.#.#.##\n#.#.#.#...##\n#.#.###.#.##\n#...#...#.##\n#.###.###.##\n#.....#G..##\n############\n############", "question": "Ignoring walls, in which compass direction does the goal lie from the agent? Use a diagonal name when the goal differs in both row and column.", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 6, "answer": "south-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "1561920f14ac5b024e0e0240ab186b0ec2c2964c0656897bf9d2b02bc98637cb"}1166{"id": "move_legal-001165", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 30, column 15. Rows increase to the south and columns to the east.\nRow 28: column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is wall\nRow 29: column 13 is open, column 14 is wall, column 15 is open, column 16 is open, column 17 is open\nRow 30: column 13 is open, column 14 is wall, column 15 is the agent, column 16 is wall, column 17 is wall\nRow 31: column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open\nRow 32: column 13 is wall, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is open", "question": "If the agent attempts one cell east (col+1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "9a7f0ae1cda5029b672f233c6fcda4212affac6050c521d15e7b70c4b68343b9"}1167{"id": "only_way-001166", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[0,2],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-3,-1\":\"outside\",\"-3,0\":\"outside\",\"-3,1\":\"outside\",\"-3,2\":\"outside\",\"-3,3\":\"outside\",\"-3,4\":\"outside\",\"-3,5\":\"outside\",\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-2,5\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"open\",\"0,2\":\"agent\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"open\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"wall\",\"1,5\":\"wall\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"open\",\"2,4\":\"wall\",\"2,5\":\"open\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\"}}", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 5, "answer": "more than one of them", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "727f0984615e7462f52687e4873db6b573423effd6a18702f38e0c74f4c01244"}1168{"id": "reachable_within-001167", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (3,7), goal at (9,3).\n############\n#...#.....##\n###.#.###.##\n#...#..A#.##\n#.#####.#.##\n#.......#.##\n#########.##\n#.....#...##\n#.###.#.####\n#..G#.....##\n############\n############", "question": "Can the agent reach the goal in at most 29 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "249d433511d1557cac09838d378cfad9057eec88c30ecda1dbe56d8663910607"}1169{"id": "distance_band-001168", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 16x16. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (11,7), goal at (1,1).\n################\n#G....#.......##\n#.##..#.#.#.####\n#.......#.#...##\n###.#.#.#.###.##\n#...#.....#...##\n###.###.#.#.####\n#.#...#...#...##\n#.###.#.#.###.##\n#.....#...#...##\n#.##..#.#.#.#.##\n#.#....A#.#...##\n#.#.###.###.#.##\n#...#.......#.##\n################\n################", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 2, "answer": "10 to 19 moves", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "249442a4cf84ecfd86a46a72a12dcd669b1cd7ae52c416be87cd75bba8b36b4d"}1170{"id": "cell_kind-001169", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (9,6); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n####.##\n...#...\n.#.###.\n...A.#.\n.###.#.\n.#.#.#.\n...#.##", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "6102716f627e7c1a0180d70d061ff46904feddc44b8c081359067510399c313b"}1171{"id": "plan_survives-001170", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (9,5); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#####.#\n....#.#\n###.#.#\n..#A#..\n#.#####\n#.....#\n#####.#", "question": "The agent will attempt this plan in order: east, south, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "68f30906e54128eb18a0d79dd878f9a1d46ee57f448c103ef1400257156b8fae"}1172{"id": "plan_survives-001171", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 13, column 1. Rows increase to the south and columns to the east.\nRow 11: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open\nRow 12: column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open\nRow 13: column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is open, column 3 is open\nRow 14: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall\nRow 15: column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall", "question": "The agent will attempt this plan in order: north, north, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "06bde41d5c439c67b03009fa27a2e2e1b7e42d9701b4c423489a2265bd2bbc8d"}1173{"id": "cell_kind-001172", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (2,0); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nXXXXXXXXX\nXXXXXXXXX\nXXXX....#\nXXXX.#...\nXXXXA....\nXXXX....#\nXXXX#....\nXXXX.....\nXXXX#.#..", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 2, "answer": "junction: three or more ways out", "answer_distribution": [0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "78be70fa6457b0616403c51f041be76daa255b9fabefa24dad2c7aac1bc5c27e"}1174{"id": "stochastic-001173", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[8,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"5,6\":\"wall\",\"5,7\":\"open\",\"5,8\":\"open\",\"5,9\":\"open\",\"5,10\":\"wall\",\"5,11\":\"wall\",\"5,12\":\"outside\",\"6,6\":\"wall\",\"6,7\":\"open\",\"6,8\":\"wall\",\"6,9\":\"wall\",\"6,10\":\"wall\",\"6,11\":\"wall\",\"6,12\":\"outside\",\"7,6\":\"open\",\"7,7\":\"open\",\"7,8\":\"wall\",\"7,9\":\"open\",\"7,10\":\"wall\",\"7,11\":\"wall\",\"7,12\":\"outside\",\"8,6\":\"wall\",\"8,7\":\"wall\",\"8,8\":\"wall\",\"8,9\":\"agent\",\"8,10\":\"wall\",\"8,11\":\"wall\",\"8,12\":\"outside\",\"9,6\":\"open\",\"9,7\":\"open\",\"9,8\":\"open\",\"9,9\":\"open\",\"9,10\":\"wall\",\"9,11\":\"wall\",\"9,12\":\"outside\",\"10,6\":\"wall\",\"10,7\":\"wall\",\"10,8\":\"wall\",\"10,9\":\"wall\",\"10,10\":\"wall\",\"10,11\":\"wall\",\"10,12\":\"outside\",\"11,6\":\"wall\",\"11,7\":\"wall\",\"11,8\":\"wall\",\"11,9\":\"wall\",\"11,10\":\"wall\",\"11,11\":\"wall\",\"11,12\":\"outside\"}}", "question": "The actuator executes the intended move with probability 0.80; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move west. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.8666666666666667, 0.1333333333333333], "deterministic": false, "fingerprint": "dcb422c3214fb9b3554dc5e68e9ae56049f8336c7f4f5d83a0912be25f30394a"}1175{"id": "move_two_step-001174", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[19,2],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"16,-1\":\"outside\",\"16,0\":\"wall\",\"16,1\":\"open\",\"16,2\":\"wall\",\"16,3\":\"wall\",\"16,4\":\"wall\",\"16,5\":\"wall\",\"17,-1\":\"outside\",\"17,0\":\"wall\",\"17,1\":\"open\",\"17,2\":\"wall\",\"17,3\":\"open\",\"17,4\":\"open\",\"17,5\":\"open\",\"18,-1\":\"outside\",\"18,0\":\"wall\",\"18,1\":\"wall\",\"18,2\":\"wall\",\"18,3\":\"open\",\"18,4\":\"wall\",\"18,5\":\"wall\",\"19,-1\":\"outside\",\"19,0\":\"wall\",\"19,1\":\"open\",\"19,2\":\"agent\",\"19,3\":\"open\",\"19,4\":\"wall\",\"19,5\":\"open\",\"20,-1\":\"outside\",\"20,0\":\"wall\",\"20,1\":\"wall\",\"20,2\":\"wall\",\"20,3\":\"wall\",\"20,4\":\"wall\",\"20,5\":\"open\",\"21,-1\":\"outside\",\"21,0\":\"wall\",\"21,1\":\"open\",\"21,2\":\"wall\",\"21,3\":\"open\",\"21,4\":\"open\",\"21,5\":\"open\",\"22,-1\":\"outside\",\"22,0\":\"wall\",\"22,1\":\"open\",\"22,2\":\"wall\",\"22,3\":\"open\",\"22,4\":\"wall\",\"22,5\":\"wall\"}}", "question": "The agent plans to move north, then east. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "d120dc004915823c9c21f313812fd89a2f60e184f5a6933e34487f42f6d480f4"}1176{"id": "reachable_within-001175", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 12x12. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (6,5), goal at (11,9).\n......#....#\n..#......###\n..#..#...#.#\n#..##.......\n#.......#..#\n......#.###.\n....#A...#..\n#.#....#....\n....#...##..\n........#..#\n...##.#.#..#\n#..######G.#", "question": "Can the agent reach the goal in at most 12 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "7fd93ce70fab938f5011433ba5155157ec86f1b5b9466edac28117373b96ab8c"}1177{"id": "blocked_pair-001176", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[6,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"2,-3\":\"outside\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"wall\",\"2,2\":\"open\",\"2,3\":\"open\",\"2,4\":\"wall\",\"2,5\":\"wall\",\"3,-3\":\"outside\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"open\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"4,-3\":\"outside\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"open\",\"4,1\":\"open\",\"4,2\":\"open\",\"4,3\":\"wall\",\"4,4\":\"open\",\"4,5\":\"open\",\"5,-3\":\"outside\",\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"open\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"wall\",\"5,4\":\"wall\",\"5,5\":\"open\",\"6,-3\":\"outside\",\"6,-2\":\"outside\",\"6,-1\":\"outside\",\"6,0\":\"open\",\"6,1\":\"agent\",\"6,2\":\"open\",\"6,3\":\"wall\",\"6,4\":\"open\",\"6,5\":\"open\",\"7,-3\":\"outside\",\"7,-2\":\"outside\",\"7,-1\":\"outside\",\"7,0\":\"open\",\"7,1\":\"open\",\"7,2\":\"open\",\"7,3\":\"open\",\"7,4\":\"open\",\"7,5\":\"open\",\"8,-3\":\"outside\",\"8,-2\":\"outside\",\"8,-1\":\"outside\",\"8,0\":\"open\",\"8,1\":\"open\",\"8,2\":\"wall\",\"8,3\":\"open\",\"8,4\":\"open\",\"8,5\":\"open\",\"9,-3\":\"outside\",\"9,-2\":\"outside\",\"9,-1\":\"outside\",\"9,0\":\"outside\",\"9,1\":\"outside\",\"9,2\":\"outside\",\"9,3\":\"outside\",\"9,4\":\"outside\",\"9,5\":\"outside\",\"10,-3\":\"outside\",\"10,-2\":\"outside\",\"10,-1\":\"outside\",\"10,0\":\"outside\",\"10,1\":\"outside\",\"10,2\":\"outside\",\"10,3\":\"outside\",\"10,4\":\"outside\",\"10,5\":\"outside\"}}", "question": "Are BOTH the cell one step east and the cell one step south blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "1f6bccdecb28f6c5164c1d68b6cb66b5baa39616ff54796dee7c7a12d3e6a0d7"}1178{"id": "plan_survives-001177", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 6, column 1. Rows increase to the south and columns to the east.\nRow 2: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 3: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is wall, column 5 is open\nRow 4: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is open, column 4 is wall, column 5 is open\nRow 5: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is wall, column 3 is open, column 4 is open, column 5 is open\nRow 6: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is the agent, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 7: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is open, column 2 is open, column 3 is open, column 4 is open, column 5 is open\nRow 8: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is wall, column 1 is wall, column 2 is wall, column 3 is wall, column 4 is wall, column 5 is wall\nRow 9: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze\nRow 10: column -3 is outside the maze, column -2 is outside the maze, column -1 is outside the maze, column 0 is outside the maze, column 1 is outside the maze, column 2 is outside the maze, column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze", "question": "The agent will attempt this plan in order: north, north, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "69cf8aff71419ccec9ccda6270631d532a0ed7cee194f385d5052481a6e8c375"}1179{"id": "cell_kind-001178", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 4, column 5. Rows increase to the south and columns to the east.\nRow 1: column 2 is open, column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall\nRow 2: column 2 is wall, column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall\nRow 3: column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 4: column 2 is wall, column 3 is open, column 4 is wall, column 5 is the agent, column 6 is wall, column 7 is open, column 8 is wall\nRow 5: column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 6: column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall\nRow 7: column 2 is wall, column 3 is open, column 4 is wall, column 5 is open, column 6 is open, column 7 is open, column 8 is wall", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "874a740b216723417ebd1374d7e14405f7f212af9bd86ee5d0577e7e2008c1da"}1180{"id": "stochastic-001179", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[4,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"open\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"wall\",\"3,3\":\"open\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"agent\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"open\",\"5,3\":\"open\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"wall\"}}", "question": "The actuator executes the intended move with probability 0.70; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move south. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.20000000000000007, 0.7999999999999999], "deterministic": false, "fingerprint": "09228906d0b865484c00af29a9af59eb5104f6d2be28e4c1c8df15b743ffb9c4"}1181{"id": "first_move-001180", "family": "first_move", "environment": "maze", "seen_in_training": false, "state": "Maze 28x28. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (23,20), goal at (13,13).\n############################\n#...#.........#...........##\n#.###.#.#######.#.#######.##\n#.....#.........#.....#...##\n#.###################.#.####\n#.#.........#.#.......#.#.##\n#.#.#######.#.#.#######.#.##\n#...#.....#.#.#.#.#.......##\n#####.#####.#.#.#.#.########\n#...#.....#.#.#.#...#.....##\n#.#.#.###.#.#.#.#####.###.##\n#.#.#.#.#.....#.#...#.#...##\n#.###.#.#######.#.#.#.#.####\n#...#.#.....#G#...#...#...##\n#.#.#.###.#.#.###.#######.##\n#.#.......#.#...#...#.#...##\n#.#############.###.#.#.#.##\n#.#.#...........#.....#.#.##\n#.#.#.#################.####\n#.#.#.......#...#.....#...##\n#.#.#######.#.#.#.###.#.#.##\n#.#.......#.#.#...#...#.#.##\n#.#.#####.#.#.#####.#####.##\n#.#.#.....#...#...#.A.#...##\n#.###.###.#####.#.###.#.#.##\n#.....#.........#...#...#.##\n############################\n############################", "question": "Which single first move starts a shortest path from the agent to the goal?", "options": ["north", "east", "south", "west"], "answer_index": 3, "answer": "west", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "f6cd2a166cfd190ce999b33f097e42982a07c72b723c6c32209a993712f55c73"}1182{"id": "open_count-001181", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (11,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX#...\nX#.#.\nX#A#.\nX#.##\nX#...", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 2, "answer": "2", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "01e8d181224d0daa35ce33fae353df008bb0562ac047aed4e61b041a4e673c04"}1183{"id": "stochastic-001182", "family": "stochastic", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 9, column 15. Rows increase to the south and columns to the east.\nRow 6: column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open, column 18 is wall\nRow 7: column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open, column 18 is wall\nRow 8: column 12 is wall, column 13 is open, column 14 is wall, column 15 is open, column 16 is wall, column 17 is open, column 18 is wall\nRow 9: column 12 is wall, column 13 is open, column 14 is wall, column 15 is the agent, column 16 is wall, column 17 is open, column 18 is wall\nRow 10: column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is open, column 18 is wall\nRow 11: column 12 is wall, column 13 is open, column 14 is open, column 15 is open, column 16 is open, column 17 is open, column 18 is wall\nRow 12: column 12 is wall, column 13 is open, column 14 is wall, column 15 is wall, column 16 is wall, column 17 is open, column 18 is wall", "question": "The actuator executes the intended move with probability 0.90; otherwise it executes one of the other three directions, chosen uniformly. The agent intends to move east. Will the move that is actually executed land on an open cell inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [0.9666666666666667, 0.033333333333333326], "deterministic": false, "fingerprint": "28b2b928bf6b9d48ce98a46bb87260b8ca8dff157ac35ba099cc117fc05dacea"}1184{"id": "move_legal-001183", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "{\"agent\":[27,7],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"24,4\":\"wall\",\"24,5\":\"open\",\"24,6\":\"wall\",\"24,7\":\"wall\",\"24,8\":\"wall\",\"24,9\":\"wall\",\"24,10\":\"wall\",\"25,4\":\"wall\",\"25,5\":\"open\",\"25,6\":\"wall\",\"25,7\":\"open\",\"25,8\":\"open\",\"25,9\":\"open\",\"25,10\":\"wall\",\"26,4\":\"wall\",\"26,5\":\"open\",\"26,6\":\"wall\",\"26,7\":\"open\",\"26,8\":\"wall\",\"26,9\":\"open\",\"26,10\":\"wall\",\"27,4\":\"wall\",\"27,5\":\"open\",\"27,6\":\"wall\",\"27,7\":\"agent\",\"27,8\":\"wall\",\"27,9\":\"open\",\"27,10\":\"wall\",\"28,4\":\"wall\",\"28,5\":\"open\",\"28,6\":\"wall\",\"28,7\":\"wall\",\"28,8\":\"wall\",\"28,9\":\"open\",\"28,10\":\"wall\",\"29,4\":\"wall\",\"29,5\":\"open\",\"29,6\":\"open\",\"29,7\":\"open\",\"29,8\":\"open\",\"29,9\":\"open\",\"29,10\":\"wall\",\"30,4\":\"wall\",\"30,5\":\"wall\",\"30,6\":\"wall\",\"30,7\":\"wall\",\"30,8\":\"wall\",\"30,9\":\"wall\",\"30,10\":\"wall\"}}", "question": "Is the cell one step west of the agent traversable and inside the maze?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "66eef908f5e218b15923a8a17fb5e8ba8c53bd26dc8e231b3c79a3ab88c61ead"}1185{"id": "plan_survives-001184", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 17, column 19. Rows increase to the south and columns to the east.\nRow 14: column 16 is open, column 17 is open, column 18 is open, column 19 is wall, column 20 is open, column 21 is outside the maze, column 22 is outside the maze\nRow 15: column 16 is open, column 17 is wall, column 18 is open, column 19 is open, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze\nRow 16: column 16 is open, column 17 is wall, column 18 is open, column 19 is wall, column 20 is open, column 21 is outside the maze, column 22 is outside the maze\nRow 17: column 16 is open, column 17 is open, column 18 is wall, column 19 is the agent, column 20 is open, column 21 is outside the maze, column 22 is outside the maze\nRow 18: column 16 is open, column 17 is open, column 18 is wall, column 19 is open, column 20 is wall, column 21 is outside the maze, column 22 is outside the maze\nRow 19: column 16 is wall, column 17 is open, column 18 is open, column 19 is open, column 20 is open, column 21 is outside the maze, column 22 is outside the maze\nRow 20: column 16 is open, column 17 is open, column 18 is open, column 19 is open, column 20 is open, column 21 is outside the maze, column 22 is outside the maze", "question": "The agent will attempt this plan in order: east, north, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 3, "answer": "it fails at step 3", "answer_distribution": [0.0, 0.0, 0.0, 1.0], "deterministic": true, "fingerprint": "c40144d84a96acb3303b4816c9abc2b14da135de92aa0720809a40b75c4bca2a"}1186{"id": "plan_survives-001185", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (2,1); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\nX###.\nX..##\nX.A..\nX..#.\nX####", "question": "The agent will attempt this plan in order: south, south, north. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 2, "answer": "it fails at step 2", "answer_distribution": [0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "453b15be6e67c9b1beddb37b1e6fbf705663cc9b20fd23744736994884205c7e"}1187{"id": "distance_band-001186", "family": "distance_band", "environment": "maze", "seen_in_training": true, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (2,2), goal at (1,0).\n#....#....#..#....#.#...\nG...#...#.#.#...........\n..A..####...............\n.........#..##..........\n#.......##.#####...#....\n............##..........\n##....#.#....###........\n#.........#.....#...#...\n#....#....#.#.#.##.....#\n..#...#..#....#.......#.\n#..#..#.................\n#....#.#..#.##.......#..\n###........##..##..##...\n##..#..........##.....#.\n......#..#.#.##......#..\n.#.#...#...........#....\n...#.#....#..#..........\n.....##.......#...#.##..\n.#.#.....#..........##..\n...........#.#.......#..\n.#.....#..#...#.##...#..\n....#.##.....#.........#\n..#.....#...#........#..\n...#.#...#.........##...", "question": "How many moves does the shortest path from the agent to the goal take?", "options": ["0 to 4 moves", "5 to 9 moves", "10 to 19 moves", "20 to 39 moves", "40 moves or more"], "answer_index": 0, "answer": "0 to 4 moves", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "c787c074d0e8c9cc3259a9af68c8736f5b880908539b39af44334e084b858e99"}1188{"id": "cell_kind-001187", "family": "cell_kind", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[13,14],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"11,12\":\"wall\",\"11,13\":\"open\",\"11,14\":\"open\",\"11,15\":\"open\",\"11,16\":\"open\",\"12,12\":\"wall\",\"12,13\":\"wall\",\"12,14\":\"wall\",\"12,15\":\"wall\",\"12,16\":\"wall\",\"13,12\":\"open\",\"13,13\":\"open\",\"13,14\":\"agent\",\"13,15\":\"open\",\"13,16\":\"open\",\"14,12\":\"wall\",\"14,13\":\"wall\",\"14,14\":\"wall\",\"14,15\":\"wall\",\"14,16\":\"wall\",\"15,12\":\"wall\",\"15,13\":\"open\",\"15,14\":\"wall\",\"15,15\":\"open\",\"15,16\":\"open\"}}", "question": "Considering only the four orthogonal neighbours, what kind of cell is the agent standing on?", "options": ["dead end: at most one way out", "corridor: exactly two ways out", "junction: three or more ways out"], "answer_index": 1, "answer": "corridor: exactly two ways out", "answer_distribution": [0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "493958c2c8251174764e8cb84990f16d2a0dd2467acebb33220a900a937a7553"}1189{"id": "onward-001188", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[11,3],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"8,0\":\"wall\",\"8,1\":\"open\",\"8,2\":\"wall\",\"8,3\":\"open\",\"8,4\":\"wall\",\"8,5\":\"wall\",\"8,6\":\"wall\",\"9,0\":\"wall\",\"9,1\":\"open\",\"9,2\":\"wall\",\"9,3\":\"open\",\"9,4\":\"wall\",\"9,5\":\"open\",\"9,6\":\"wall\",\"10,0\":\"wall\",\"10,1\":\"open\",\"10,2\":\"wall\",\"10,3\":\"open\",\"10,4\":\"wall\",\"10,5\":\"open\",\"10,6\":\"wall\",\"11,0\":\"wall\",\"11,1\":\"open\",\"11,2\":\"wall\",\"11,3\":\"agent\",\"11,4\":\"wall\",\"11,5\":\"open\",\"11,6\":\"open\",\"12,0\":\"wall\",\"12,1\":\"open\",\"12,2\":\"wall\",\"12,3\":\"wall\",\"12,4\":\"wall\",\"12,5\":\"open\",\"12,6\":\"wall\",\"13,0\":\"wall\",\"13,1\":\"open\",\"13,2\":\"open\",\"13,3\":\"open\",\"13,4\":\"open\",\"13,5\":\"open\",\"13,6\":\"wall\",\"14,0\":\"wall\",\"14,1\":\"open\",\"14,2\":\"wall\",\"14,3\":\"wall\",\"14,4\":\"wall\",\"14,5\":\"wall\",\"14,6\":\"wall\"}}", "question": "Suppose the agent moves one cell north. From that new cell, would a further step north be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "c4df54f3e90c80495b516a05307490c1749f9974002ac94b260549e3db711932"}1190{"id": "plan_survives-001189", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[9,9],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"7,7\":\"open\",\"7,8\":\"open\",\"7,9\":\"open\",\"7,10\":\"open\",\"7,11\":\"open\",\"8,7\":\"open\",\"8,8\":\"wall\",\"8,9\":\"wall\",\"8,10\":\"wall\",\"8,11\":\"open\",\"9,7\":\"open\",\"9,8\":\"wall\",\"9,9\":\"agent\",\"9,10\":\"wall\",\"9,11\":\"open\",\"10,7\":\"open\",\"10,8\":\"wall\",\"10,9\":\"open\",\"10,10\":\"wall\",\"10,11\":\"open\",\"11,7\":\"open\",\"11,8\":\"wall\",\"11,9\":\"open\",\"11,10\":\"wall\",\"11,11\":\"open\"}}", "question": "The agent will attempt this plan in order: north, south, east. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "54f3165af0ad165e3f5834e402e4df2964178c55d02a080ee4462b417b1b058e"}1191{"id": "open_count-001190", "family": "open_count", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (26,5); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n#######\n#######\n###.#..\n###A...\n#####..\n#####..\n######.", "question": "How many of the four cells orthogonally adjacent to the agent are traversable and inside the maze?", "options": ["0", "1", "2", "3", "4"], "answer_index": 2, "answer": "2", "answer_distribution": [0.0, 0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "283c1231978b2b6c52504ac51aec559ec5f66bd758214daaa5eb12feddec31ce"}1192{"id": "plan_survives-001191", "family": "plan_survives", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (9,13); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n###.#.#\n....#..\n###.###\n#..A..#\n#######\n#.....#\n#.###.#", "question": "The agent will attempt this plan in order: south, south, west. A step fails if its destination is a wall or outside the maze, and the agent then stops. What happens?", "options": ["the plan completes; every step is legal", "it fails at step 1", "it fails at step 2", "it fails at step 3"], "answer_index": 1, "answer": "it fails at step 1", "answer_distribution": [0.0, 1.0, 0.0, 0.0], "deterministic": true, "fingerprint": "8c239bf0333d070ded53d85b7c7138ed5c82405c04fc971b499bf6c4e0310243"}1193{"id": "reachable_within-001192", "family": "reachable_within", "environment": "maze", "seen_in_training": false, "state": "Maze 24x24. '#': wall; '.': open; 'A': agent; 'G': goal. Rows increase south; columns increase east. Agent at (15,3), goal at (19,21).\n########################\n#...#.............#...##\n#.#...#.#########.#.#.##\n#.#...#.#...........#.##\n#.###.#.#.#.#.#.#.###.##\n#.....#.#.#...#.#.....##\n#.#.###.#.##.##.#.#.#.##\n#.#...#.......#.....#.##\n#...#.#######.##.##.#.##\n#.#.#.....#...#.....#.##\n#.#.###.###.#.#.#.######\n#.#...#...#.#...#.#...##\n#.#####.#...#.#.#.#.#.##\n#.......#.#...#.....#.##\n#####.#.#.#####.#####.##\n#..A#.........#.#.....##\n#.###.#.###.#.###.###.##\n#...#.#...#.#...#.#...##\n#.#.#.#.###.###...#.####\n#.#.#...#...#.....#.#G##\n#...###.#.###.###.#.#.##\n#.#.......#.......#...##\n########################\n########################", "question": "Can the agent reach the goal in at most 49 moves, travelling only through open cells?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "1e1853fff69dbd5d0266c77e4f2682ca064b7b5448f0ee55044957df3aa428bb"}1194{"id": "snake_food-001193", "family": "snake_food", "environment": "snake", "seen_in_training": false, "state": "Snake game on a 12x12 board. 'H': head; 'o': body; '*': food; '.': empty. Rows increase south; columns increase east. The snake dies if it leaves the board or enters a cell its body occupies. Head at (3,6), travelling north, length 3.\n............\n............\n............\n......H.....\n......o...*.\n......o.....\n............\n............\n............\n............\n............\n............", "question": "Ignoring the snake's own body, in which compass direction does the food lie from the head?", "options": ["north", "south", "east", "west", "north-east", "north-west", "south-east", "south-west"], "answer_index": 6, "answer": "south-east", "answer_distribution": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], "deterministic": true, "fingerprint": "a433730e74d83f15ac9de23714d4b302eb5f1a9675ccd247bbf858fc21ae82f8"}1195{"id": "move_two_step-001194", "family": "move_two_step", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[1,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"-3,-3\":\"outside\",\"-3,-2\":\"outside\",\"-3,-1\":\"outside\",\"-3,0\":\"outside\",\"-3,1\":\"outside\",\"-3,2\":\"outside\",\"-3,3\":\"outside\",\"-3,4\":\"outside\",\"-3,5\":\"outside\",\"-2,-3\":\"outside\",\"-2,-2\":\"outside\",\"-2,-1\":\"outside\",\"-2,0\":\"outside\",\"-2,1\":\"outside\",\"-2,2\":\"outside\",\"-2,3\":\"outside\",\"-2,4\":\"outside\",\"-2,5\":\"outside\",\"-1,-3\":\"outside\",\"-1,-2\":\"outside\",\"-1,-1\":\"outside\",\"-1,0\":\"outside\",\"-1,1\":\"outside\",\"-1,2\":\"outside\",\"-1,3\":\"outside\",\"-1,4\":\"outside\",\"-1,5\":\"outside\",\"0,-3\":\"outside\",\"0,-2\":\"outside\",\"0,-1\":\"outside\",\"0,0\":\"wall\",\"0,1\":\"wall\",\"0,2\":\"wall\",\"0,3\":\"wall\",\"0,4\":\"wall\",\"0,5\":\"wall\",\"1,-3\":\"outside\",\"1,-2\":\"outside\",\"1,-1\":\"outside\",\"1,0\":\"wall\",\"1,1\":\"agent\",\"1,2\":\"open\",\"1,3\":\"open\",\"1,4\":\"wall\",\"1,5\":\"open\",\"2,-3\":\"outside\",\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"wall\",\"2,1\":\"open\",\"2,2\":\"wall\",\"2,3\":\"wall\",\"2,4\":\"wall\",\"2,5\":\"open\",\"3,-3\":\"outside\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"wall\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"open\",\"3,5\":\"open\",\"4,-3\":\"outside\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"wall\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"open\",\"4,4\":\"wall\",\"4,5\":\"open\",\"5,-3\":\"outside\",\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"open\",\"5,2\":\"wall\",\"5,3\":\"open\",\"5,4\":\"open\",\"5,5\":\"open\"}}", "question": "The agent plans to move east, then west. Are both steps legal -- each destination inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "9b3ab75bb9e9b7a7fa685b78246ce57d768f9968c0f3a3f7c5211f0b965843a9"}1196{"id": "blocked_pair-001195", "family": "blocked_pair", "environment": "maze", "seen_in_training": false, "state": "The agent stands at row 5, column 7. Rows increase to the south and columns to the east.\nRow 1: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 2: column 3 is open, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 3: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 4: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 5: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is the agent, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 6: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 7: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 8: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze\nRow 9: column 3 is outside the maze, column 4 is outside the maze, column 5 is outside the maze, column 6 is outside the maze, column 7 is outside the maze, column 8 is outside the maze, column 9 is outside the maze, column 10 is outside the maze, column 11 is outside the maze", "question": "Are BOTH the cell one step east and the cell one step west blocked -- each either a wall or outside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "8961d8ff75a341867992859d2f9ddb4d2cf8f821a06bb48a39da9ab277382a2d"}1197{"id": "onward-001196", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "{\"agent\":[5,1],\"axes\":\"rows increase south, columns increase east\",\"window\":{\"2,-2\":\"outside\",\"2,-1\":\"outside\",\"2,0\":\"open\",\"2,1\":\"open\",\"2,2\":\"open\",\"2,3\":\"open\",\"2,4\":\"open\",\"3,-2\":\"outside\",\"3,-1\":\"outside\",\"3,0\":\"open\",\"3,1\":\"open\",\"3,2\":\"open\",\"3,3\":\"open\",\"3,4\":\"open\",\"4,-2\":\"outside\",\"4,-1\":\"outside\",\"4,0\":\"open\",\"4,1\":\"open\",\"4,2\":\"wall\",\"4,3\":\"wall\",\"4,4\":\"wall\",\"5,-2\":\"outside\",\"5,-1\":\"outside\",\"5,0\":\"wall\",\"5,1\":\"agent\",\"5,2\":\"open\",\"5,3\":\"wall\",\"5,4\":\"open\",\"6,-2\":\"outside\",\"6,-1\":\"outside\",\"6,0\":\"wall\",\"6,1\":\"open\",\"6,2\":\"wall\",\"6,3\":\"wall\",\"6,4\":\"open\",\"7,-2\":\"outside\",\"7,-1\":\"outside\",\"7,0\":\"wall\",\"7,1\":\"wall\",\"7,2\":\"open\",\"7,3\":\"open\",\"7,4\":\"open\",\"8,-2\":\"outside\",\"8,-1\":\"outside\",\"8,0\":\"wall\",\"8,1\":\"open\",\"8,2\":\"open\",\"8,3\":\"open\",\"8,4\":\"open\"}}", "question": "Suppose the agent moves one cell north. From that new cell, would a further step north be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "bd74d5b693e002dfe1a1939ff98d769d453c17bfafcb0f45cb2987c3b90dd595"}1198{"id": "move_legal-001197", "family": "move_legal", "environment": "maze", "seen_in_training": true, "state": "The agent stands at row 6, column 7. Rows increase to the south and columns to the east.\nRow 2: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is open\nRow 3: column 3 is open, column 4 is open, column 5 is open, column 6 is open, column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is open\nRow 4: column 3 is wall, column 4 is wall, column 5 is wall, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open\nRow 5: column 3 is open, column 4 is open, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open\nRow 6: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is the agent, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open\nRow 7: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open\nRow 8: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is wall, column 9 is open, column 10 is wall, column 11 is open\nRow 9: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is open, column 8 is open, column 9 is open, column 10 is wall, column 11 is open\nRow 10: column 3 is open, column 4 is wall, column 5 is open, column 6 is wall, column 7 is wall, column 8 is wall, column 9 is wall, column 10 is wall, column 11 is open", "question": "If the agent attempts one cell west (col-1), will the destination be inside the maze and not a wall?", "options": ["no", "yes"], "answer_index": 0, "answer": "no", "answer_distribution": [1.0, 0.0], "deterministic": true, "fingerprint": "c122d2d988d90a473ead59404847768f94f999db11c9adc305c6654d777a8a37"}1199{"id": "only_way-001198", "family": "only_way", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (9,21); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#.#.#...\n.#.#.#.##\n.#.#.#.#.\n.#.#.#.#.\n.#.#A#.#.\n.#.###.##\n.#.#.....\n.#.#.####\n.#.#.#...", "question": "Which of the four orthogonal directions can the agent move in?", "options": ["north", "east", "south", "west", "none of them", "more than one of them"], "answer_index": 0, "answer": "north", "answer_distribution": [1.0, 0.0, 0.0, 0.0, 0.0, 0.0], "deterministic": true, "fingerprint": "5f74d71fa7e963380006c86473fa6760c2a79177180db1235a35398681961a6f"}1200{"id": "onward-001199", "family": "onward", "environment": "maze", "seen_in_training": false, "state": "Agent coordinate: (33,30); zero-based row and column. Rows increase south; columns increase east. The local window is centered on A. '#': wall; '.': open; 'A': agent; 'X': outside.\nLocal map:\n.#.#...\n.#...#.\n.#.###.\n.#.A...\n#######\n#######\nXXXXXXX", "question": "Suppose the agent moves one cell east. From that new cell, would a further step east be legal -- destination open and inside the maze?", "options": ["no", "yes"], "answer_index": 1, "answer": "yes", "answer_distribution": [0.0, 1.0], "deterministic": true, "fingerprint": "47cbef3d24644ad0b8ff6d1824ca60bc0924aa133880cea52307fcbfa237f0cf"}