Model: gemini-3-pro-preview

sample_run02

Select Configuration:

📊 Sample Metrics

🔍 Exploration

last_node_coverage: 1
last_edge_coverage: 0.439
n_exploration_steps: 11
action_counts:
move: 4
rotate: 8
return: 0
observe: 10
term: 1
forced_term: 0
query: 0
action_cost: 10
final_information_gain: 0.746
action_fail_ratio: 0.091
valid_action_ratio: 1
false_belief_steps: 5
false_belief_f1: 1
false_belief_f1_position: 1
false_belief_f1_facing: 1
false_belief_action_cost: 4
false_belief_action_cost_after_seen: 0

✅ Evaluation

overall:
n_total: 27
total_score: 21.230
avg_accuracy: 0.786
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
PovEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Action2ViewEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
View2ActionTextEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
AlloMappingEvaluationTask:
n_total: 3
task_score: 2.594
avg_accuracy: 0.865
RotEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Location2ViewEvaluationTask:
n_total: 3
task_score: 1
avg_accuracy: 0.333
View2LocationTextEvaluationTask:
n_total: 3
task_score: 1.635
avg_accuracy: 0.545

✅ Evaluation (prompt_cogmap)

overall:
n_total: 27
total_score: 21.773
avg_accuracy: 0.806
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
PovEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Action2ViewEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
View2ActionTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
AlloMappingEvaluationTask:
n_total: 3
task_score: 2.416
avg_accuracy: 0.805
RotEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Location2ViewEvaluationTask:
n_total: 3
task_score: 1
avg_accuracy: 0.333
View2LocationTextEvaluationTask:
n_total: 3
task_score: 1.357
avg_accuracy: 0.452

✅ Evaluation (use_gt_cogmap)

overall:
n_total: 27
total_score: 24.258
avg_accuracy: 0.898
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
PovEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Action2ViewEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
View2ActionTextEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
AlloMappingEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
RotEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Location2ViewEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
View2LocationTextEvaluationTask:
n_total: 3
task_score: 2.758
avg_accuracy: 0.919

✅ Evaluation (use_model_cogmap)

overall:
n_total: 27
total_score: 17.451
avg_accuracy: 0.646
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
PovEvaluationTask:
n_total: 3
task_score: 1.500
avg_accuracy: 0.500
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Action2ViewEvaluationTask:
n_total: 3
task_score: 0.500
avg_accuracy: 0.167
View2ActionTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
AlloMappingEvaluationTask:
n_total: 3
task_score: 2.309
avg_accuracy: 0.770
RotEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
Location2ViewEvaluationTask:
n_total: 3
task_score: 1
avg_accuracy: 0.333
View2LocationTextEvaluationTask:
n_total: 3
task_score: 1.142
avg_accuracy: 0.381

🧠 Cognitive Map

exploration:
error:
local_vs_gt_local_avg:
dir: 0.952
facing: 1
pos: 0.941
overall: 0.964
global_vs_gt_global_avg:
dir: 0.836
facing: 0.955
pos: 0.812
overall: 0.867
agent_vs_gt_agent_avg:
dir: 0.455
facing: 1
pos: 0.867
overall: 0.774
newly_observed_vs_gt_local_avg:
dir: 0.950
facing: 1
pos: 0.939
overall: 0.963
correctness:
last_global_vs_gt_full:
dir: 0.708
facing: 1
pos: 0.718
overall: 0.809
consistency:
local_vs_global_avg:
dir: 0.833
facing: 1
pos: 0.881
overall: 0.905
position_update_avg: 0.518
facing_update_avg: 0.926
position_stability_avg: 0.518
facing_stability_avg: 0.926
fog_probe:
f1_avg: 0.593
precision_avg: 0.641
recall_avg: 0.567
cogmap_fb:
metrics:
changed:
dir: None
pos: 0.812
facing: 1
overall: None
retention:
dir: None
pos: 0.421
facing: 0
overall: None
unchanged:
dir: 0.602
facing: 0.714
pos: 0.693
overall: 0.670
inertia: 0.185
inertia_list: [0.18450657484958408]
unchanged_retention:
dir: None
pos: 0.642
facing: 1
overall: None
unchanged_retention_minus_retention:
pos: 0.570
facing: 1
unchanged_exploration:
dir: 0.855
facing: 1
pos: 0.795
overall: 0.883

🧭 False Belief CogMap

inertia: 0.185
changed:
dir: None
pos: 0.812
facing: 1
overall: None
retention:
dir: None
pos: 0.421
facing: 0
overall: None
unchanged:
dir: 0.602
facing: 0.714
pos: 0.693
overall: 0.670
unchanged_retention:
dir: None
pos: 0.642
facing: 1
overall: None
unchanged_retention_minus_retention:
pos: 0.570
facing: 1
unchanged_exploration:
dir: 0.855
facing: 1
pos: 0.795
overall: 0.883

📊 Performance Charts

Cognitive Map (Update)
Global Update Metrics
Cognitive Map (Full)
Global Full Metrics
Cognitive Map (Self-Tracking)
Global Self-Tracking Metrics
FB CogMap (Unchanged)
False Belief CogMap Unchanged per Turn
Fog Probe F1
Fog Probe F1 per Turn
Fog Probe Precision
Fog Probe Precision per Turn
Fog Probe Recall
Fog Probe Recall per Turn
Position Update
Position Update per Turn
Facing Update
Facing Update per Turn
Position Stability
Position Stability per Turn
Facing Stability
Facing Stability per Turn

🌍 Exploration Phase

🔄 Turn 1

💬 Assistant Action
Actions: [Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "west"
  },
  "blue door": {
    "position": [
      -2,
      4
    ],
    "facing": "south"
  }
}
🎯 Ground Truth (Observed)
{
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Full)
{
  "laptop": {
    "position": [
      -4,
      -1
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "airplane": {
    "position": [
      -2,
      11
    ],
    "facing": "east"
  },
  "pan": {
    "position": [
      -2,
      1
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "lego man": {
    "position": [
      -6,
      9
    ],
    "facing": "north"
  },
  "teddybear": {
    "position": [
      -5,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -5,
      4
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "candle": {
    "position": [
      -6,
      11
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "fire extinguisher": {
    "position": [
      -2,
      7
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "green door": {
      "position": [
        1,
        3
      ]
    },
    "blue door": {
      "position": [
        -3,
        5
      ]
    }
  }
}
🎯 Ground Truth
{
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "green door": {
      "position": [
        1,
        3
      ]
    },
    "blue door": {
      "position": [
        -3,
        5
      ]
    }
  }
}
🎯 Ground Truth
{
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  }
}
🗺️ Symbolic Fog Map
#################
#......##########
#......##########
#......##########
#......##########
#......##########
#......##########
#####+####......#
###...F..#......#
###......+......#
###...C.D#......#
###.B....#......#
###....E*#......#
###A.....########
#################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: A, B, E
🎯 Ground Truth: B, A, E
🧠 Cognitive Map Metrics
Global:
dir: 0.833
facing: 1
pos: 0.899
overall: 0.911
Global (Full):
dir: 0.042
facing: 0.091
pos: 0.180
overall: 0.104
Local:
dir: 1
facing: 1
pos: 1
overall: 1
Local (Newly):
dir: 1
facing: 1
pos: 1
overall: 1
Fog Probe:
precision: 1
recall: 1
overall: 1
📈 Turn Metrics
node_coverage: 0.143
edge_coverage: 0.049
is_action_fail: False
step: 1
action_counts:
move: 0
rotate: 0
return: 0
observe: 1
term: 0
forced_term: 0
query: 0
information_gain: 0.113
reward: -0.100
is_done: False
is_valid_action: True
Current state
State at Turn 1

🔄 Turn 2

💬 Assistant Action
Actions: [JumpTo(green door), Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      3,
      1
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      3,
      1
    ]
  },
  "blue door": {
    "position": [
      -2,
      4
    ]
  },
  "bike": {
    "position": [
      7,
      2
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      5,
      0
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      5,
      2
    ],
    "facing": "west"
  },
  "table": {
    "position": [
      7,
      1
    ]
  }
}
🎯 Ground Truth (Observed)
{
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  }
}
🎯 Ground Truth (Full)
{
  "laptop": {
    "position": [
      -4,
      -1
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "airplane": {
    "position": [
      -2,
      11
    ],
    "facing": "east"
  },
  "pan": {
    "position": [
      -2,
      1
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "lego man": {
    "position": [
      -6,
      9
    ],
    "facing": "north"
  },
  "teddybear": {
    "position": [
      -5,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -5,
      4
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "candle": {
    "position": [
      -6,
      11
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "fire extinguisher": {
    "position": [
      -2,
      7
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "chair": {
      "position": [
        -2,
        3
      ],
      "facing": "-y"
    },
    "truck": {
      "position": [
        2,
        3
      ],
      "facing": "-x"
    },
    "table": {
      "position": [
        0,
        5
      ]
    },
    "bike": {
      "position": [
        -2,
        5
      ],
      "facing": "+x"
    }
  }
}
🎯 Ground Truth
{
  "truck": {
    "position": [
      2,
      3
    ],
    "facing": "north"
  },
  "table": {
    "position": [
      0,
      5
    ],
    "facing": "east"
  },
  "chair": {
    "position": [
      -2,
      3
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      -2,
      5
    ],
    "facing": "south"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "chair": {
      "position": [
        -2,
        3
      ],
      "facing": "-y"
    },
    "truck": {
      "position": [
        2,
        3
      ],
      "facing": "-x"
    },
    "table": {
      "position": [
        0,
        5
      ]
    },
    "bike": {
      "position": [
        -2,
        5
      ],
      "facing": "+x"
    }
  }
}
🎯 Ground Truth
{
  "truck": {
    "position": [
      2,
      3
    ],
    "facing": "north"
  },
  "table": {
    "position": [
      0,
      5
    ],
    "facing": "east"
  },
  "chair": {
    "position": [
      -2,
      3
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      -2,
      5
    ],
    "facing": "south"
  }
}
🗺️ Symbolic Fog Map
#################
#......##########
#......##########
#......##########
#......##########
#......##########
#......##########
#####+####D.....#
###.....G#......#
###......*......#
###.....F#...IC.#
###.E..K.#H..A..#
###....L.#J.....#
###.....B########
#################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: B, E, H, J, L
🎯 Ground Truth: L, E, B, H, J, D
🧠 Cognitive Map Metrics
Global:
dir: 0.821
facing: 1
pos: 0.718
overall: 0.847
Global (Full):
dir: 0.192
facing: 0.364
pos: 0.335
overall: 0.297
Local:
dir: 1
facing: 1
pos: 1
overall: 1
Local (Newly):
dir: 1
facing: 1
pos: 1
overall: 1
Fog Probe:
precision: 1
recall: 0.833
overall: 0.909
📈 Turn Metrics
node_coverage: 0.429
edge_coverage: 0.146
is_action_fail: False
step: 2
action_counts:
move: 1
rotate: 1
return: 0
observe: 2
term: 0
forced_term: 0
query: 0
information_gain: 0.384
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 2
Current state
State at Turn 2

🔄 Turn 3

💬 Assistant Action
Actions: [JumpTo(table), Rotate(-90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      6,
      3
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      1,
      3
    ]
  },
  "blue door": {
    "position": [
      -3,
      5
    ]
  },
  "table": {
    "position": [
      6,
      3
    ]
  },
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Observed)
{
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      6,
      3
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Full)
{
  "laptop": {
    "position": [
      -4,
      -1
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "airplane": {
    "position": [
      -2,
      11
    ],
    "facing": "east"
  },
  "pan": {
    "position": [
      -2,
      1
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "lego man": {
    "position": [
      -6,
      9
    ],
    "facing": "north"
  },
  "teddybear": {
    "position": [
      -5,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -5,
      4
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "candle": {
    "position": [
      -6,
      11
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "fire extinguisher": {
    "position": [
      -2,
      7
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      6,
      3
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "bike": {
      "position": [
        0,
        2
      ],
      "facing": "-y"
    },
    "chair": {
      "position": [
        -2,
        2
      ],
      "facing": "-x"
    }
  }
}
🎯 Ground Truth
{
  "chair": {
    "position": [
      -2,
      2
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      0,
      2
    ],
    "facing": "south"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "bike": {
      "position": [
        0,
        2
      ],
      "facing": "-y"
    },
    "chair": {
      "position": [
        -2,
        2
      ],
      "facing": "-x"
    }
  }
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
#################
#......##########
#......##########
#......##########
#......##########
#......##########
#......##########
#####+####......#
###C...J.#.....I#
###.....B+....*.#
###A....F#.H....#
###......#DG....#
###...E..#LK....#
###......########
#################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: A, D, F, G, H, K, L
🎯 Ground Truth: A, E, C, L, D, K
🧠 Cognitive Map Metrics
Global:
dir: 1
facing: 1
pos: 1
overall: 1
Global (Full):
dir: 0.233
facing: 0.364
pos: 0.467
overall: 0.355
Local:
dir: 1
facing: 1
pos: 1
overall: 1
Local (Newly):
(none)
Fog Probe:
precision: 0.571
recall: 0.667
overall: 0.615
📈 Turn Metrics
node_coverage: 0.429
edge_coverage: 0.195
is_action_fail: False
step: 3
action_counts:
move: 2
rotate: 2
return: 0
observe: 3
term: 0
forced_term: 0
query: 0
information_gain: 0.384
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 3
Current state
State at Turn 3

🔄 Turn 4

💬 Assistant Action
Actions: [Rotate(180), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      6,
      3
    ],
    "facing": "south"
  },
  "green door": {
    "position": [
      1,
      3
    ]
  },
  "blue door": {
    "position": [
      -2,
      5
    ]
  },
  "table": {
    "position": [
      6,
      3
    ]
  },
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Observed)
{
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      6,
      3
    ],
    "facing": "south"
  }
}
🎯 Ground Truth (Full)
{
  "laptop": {
    "position": [
      -4,
      -1
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "airplane": {
    "position": [
      -2,
      11
    ],
    "facing": "east"
  },
  "pan": {
    "position": [
      -2,
      1
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "lego man": {
    "position": [
      -6,
      9
    ],
    "facing": "north"
  },
  "teddybear": {
    "position": [
      -5,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -5,
      4
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "candle": {
    "position": [
      -6,
      11
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "fire extinguisher": {
    "position": [
      -2,
      7
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      6,
      3
    ],
    "facing": "south"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "truck": {
      "position": [
        2,
        2
      ],
      "facing": "-y"
    }
  }
}
🎯 Ground Truth
{
  "truck": {
    "position": [
      2,
      2
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "truck": {
      "position": [
        2,
        2
      ],
      "facing": "-y"
    }
  }
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
#################
#......##########
#......##########
#......##########
#......##########
#......##########
#......##########
#####+####E.....#
###.H....#.....C#
###.....J+....*.#
###......#......#
###...A.K#G....F#
###.IB...#D.L...#
###......########
#################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: B, D, G, I
🎯 Ground Truth: A, B, I, G, D, E
🧠 Cognitive Map Metrics
Global:
dir: 0.964
facing: 1
pos: 0.952
overall: 0.972
Global (Full):
dir: 0.225
facing: 0.364
pos: 0.444
overall: 0.344
Local:
dir: 1
facing: 1
pos: 1
overall: 1
Local (Newly):
(none)
Fog Probe:
precision: 1
recall: 0.667
overall: 0.800
📈 Turn Metrics
node_coverage: 0.429
edge_coverage: 0.220
is_action_fail: False
step: 4
action_counts:
move: 2
rotate: 3
return: 0
observe: 4
term: 0
forced_term: 0
query: 0
information_gain: 0.398
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 4
Current state
State at Turn 4

🔄 Turn 5

💬 Assistant Action
Actions: [Rotate(-90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      8,
      2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      3,
      2
    ]
  },
  "blue door": {
    "position": [
      -2,
      6
    ]
  },
  "table": {
    "position": [
      8,
      2
    ]
  },
  "bike": {
    "position": [
      8,
      4
    ],
    "facing": "south"
  },
  "chair": {
    "position": [
      6,
      4
    ],
    "facing": "west"
  },
  "truck": {
    "position": [
      6,
      0
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Observed)
{
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  }
}
🎯 Ground Truth (Full)
{
  "laptop": {
    "position": [
      -4,
      -1
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "airplane": {
    "position": [
      -2,
      11
    ],
    "facing": "east"
  },
  "pan": {
    "position": [
      -2,
      1
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "lego man": {
    "position": [
      -6,
      9
    ],
    "facing": "north"
  },
  "teddybear": {
    "position": [
      -5,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -5,
      4
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "candle": {
    "position": [
      -6,
      11
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "fire extinguisher": {
    "position": [
      -2,
      7
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  }
}
🗺️ Symbolic Fog Map
#################
#......##########
#......##########
#......##########
#......##########
#......##########
#......##########
#####+####C.....#
###....HG#....F.#
###A.....+.J..*.#
###...I..#..K...#
###L.....#B.....#
###......#E.....#
###....D.########
#################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: A, D, I
🎯 Ground Truth: L, D, A, B, C, E
🧠 Cognitive Map Metrics
Global:
dir: 0.857
facing: 1
pos: 0.757
overall: 0.871
Global (Full):
dir: 0.200
facing: 0.364
pos: 0.353
overall: 0.306
Local:
(none)
Local (Newly):
(none)
Fog Probe:
precision: 0.667
recall: 0.333
overall: 0.444
📈 Turn Metrics
node_coverage: 0.429
edge_coverage: 0.220
is_action_fail: False
step: 5
action_counts:
move: 2
rotate: 4
return: 0
observe: 5
term: 0
forced_term: 0
query: 0
information_gain: 0.398
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 5
Current state
State at Turn 5

🔄 Turn 6

💬 Assistant Action
Actions: [JumpTo(green door), Rotate(180), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      1,
      3
    ]
  },
  "blue door": {
    "position": [
      -3,
      5
    ]
  },
  "table": {
    "position": [
      6,
      3
    ]
  },
  "bike": {
    "position": [
      6,
      4
    ],
    "facing": "south"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Observed)
{
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  }
}
🎯 Ground Truth (Full)
{
  "laptop": {
    "position": [
      -4,
      -1
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "airplane": {
    "position": [
      -2,
      11
    ],
    "facing": "east"
  },
  "pan": {
    "position": [
      -2,
      1
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "lego man": {
    "position": [
      -6,
      9
    ],
    "facing": "north"
  },
  "teddybear": {
    "position": [
      -5,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -5,
      4
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "candle": {
    "position": [
      -6,
      11
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "fire extinguisher": {
    "position": [
      -2,
      7
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {}
}
🎯 Ground Truth
(no data)
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {}
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
#################
#......##########
#......##########
#......##########
#......##########
#......##########
#......##########
#####+####F.....#
###...B..#......#
###.....A+..I.*.#
###......#.G....#
###.J....#C.L...#
###D....E#H.....#
###....K.########
#################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: B, D, E, J, K
🎯 Ground Truth: K, J, D, F, C, H
🧠 Cognitive Map Metrics
Global:
dir: 0.964
facing: 1
pos: 0.952
overall: 0.972
Global (Full):
dir: 0.225
facing: 0.364
pos: 0.444
overall: 0.344
Local:
(none)
Local (Newly):
(none)
Fog Probe:
precision: 0.600
recall: 0.500
overall: 0.545
📈 Turn Metrics
node_coverage: 0.429
edge_coverage: 0.220
is_action_fail: True
step: 6
action_counts:
move: 2
rotate: 4
return: 0
observe: 6
term: 0
forced_term: 0
query: 0
information_gain: 0.398
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 6
Current state
State at Turn 6

🔄 Turn 7

💬 Assistant Action
Actions: [Rotate(180), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      7,
      2
    ],
    "facing": "west"
  },
  "green door": {
    "position": [
      2,
      2
    ]
  },
  "blue door": {
    "position": [
      -1,
      5
    ]
  },
  "bike": {
    "position": [
      7,
      4
    ],
    "facing": "south"
  },
  "chair": {
    "position": [
      5,
      4
    ],
    "facing": "west"
  },
  "truck": {
    "position": [
      5,
      0
    ],
    "facing": "north"
  },
  "table": {
    "position": [
      7,
      2
    ]
  }
}
🎯 Ground Truth (Observed)
{
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      6,
      3
    ],
    "facing": "west"
  }
}
🎯 Ground Truth (Full)
{
  "laptop": {
    "position": [
      -4,
      -1
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "airplane": {
    "position": [
      -2,
      11
    ],
    "facing": "east"
  },
  "pan": {
    "position": [
      -2,
      1
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "lego man": {
    "position": [
      -6,
      9
    ],
    "facing": "north"
  },
  "teddybear": {
    "position": [
      -5,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -5,
      4
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "candle": {
    "position": [
      -6,
      11
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "fire extinguisher": {
    "position": [
      -2,
      7
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      6,
      3
    ],
    "facing": "west"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "truck": {
      "position": [
        -3,
        3
      ],
      "facing": "+x"
    },
    "chair": {
      "position": [
        3,
        3
      ],
      "facing": "+y"
    },
    "green door": {
      "position": [
        0,
        6
      ],
      "facing": "-y"
    }
  }
}
🎯 Ground Truth
{
  "truck": {
    "position": [
      -2,
      2
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      0,
      5
    ],
    "facing": "east"
  },
  "chair": {
    "position": [
      2,
      2
    ],
    "facing": "west"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "truck": {
      "position": [
        -3,
        3
      ],
      "facing": "+x"
    },
    "chair": {
      "position": [
        3,
        3
      ],
      "facing": "+y"
    },
    "green door": {
      "position": [
        0,
        6
      ],
      "facing": "-y"
    }
  }
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
#################
#......##########
#......##########
#......##########
#......##########
#......##########
#......##########
#####+####......#
###...A..#......#
###......+....*.#
###....F.#......#
###...DC.#......#
###..BE..#......#
###......########
#################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: A
🎯 Ground Truth: E, D, B
🧠 Cognitive Map Metrics
Global:
dir: 0.857
facing: 1
pos: 0.821
overall: 0.893
Global (Full):
dir: 0.200
facing: 0.364
pos: 0.383
overall: 0.316
Local:
dir: 1
facing: 1
pos: 0.845
overall: 0.948
Local (Newly):
(none)
Fog Probe:
precision: 0
recall: 0
overall: 0
📈 Turn Metrics
node_coverage: 0.429
edge_coverage: 0.220
is_action_fail: False
step: 7
action_counts:
move: 2
rotate: 5
return: 0
observe: 7
term: 0
forced_term: 0
query: 0
information_gain: 0.398
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 7
Current state
State at Turn 7

🔄 Turn 8

💬 Assistant Action
Actions: [JumpTo(green door), Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      3,
      2
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      3,
      2
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -2,
      5
    ],
    "facing": "south"
  },
  "table": {
    "position": [
      7,
      2
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      7,
      4
    ],
    "facing": "south"
  },
  "chair": {
    "position": [
      5,
      4
    ],
    "facing": "east"
  },
  "truck": {
    "position": [
      6,
      0
    ],
    "facing": "west"
  }
}
🎯 Ground Truth (Observed)
{
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      1,
      3
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Full)
{
  "laptop": {
    "position": [
      -4,
      -1
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "airplane": {
    "position": [
      -2,
      11
    ],
    "facing": "east"
  },
  "pan": {
    "position": [
      -2,
      1
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "lego man": {
    "position": [
      -6,
      9
    ],
    "facing": "north"
  },
  "teddybear": {
    "position": [
      -5,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -5,
      4
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "candle": {
    "position": [
      -6,
      11
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "fire extinguisher": {
    "position": [
      -2,
      7
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      1,
      3
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {}
}
🎯 Ground Truth
(no data)
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {}
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
#################
#......##########
#......##########
#......##########
#......##########
#......##########
#......##########
#####+####......#
###.D....#......#
###......*......#
###F.EC..#......#
###..A..B#......#
###......#......#
###......########
#################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: A, E, F
🎯 Ground Truth: F, A, E
🧠 Cognitive Map Metrics
Global:
dir: 0.750
facing: 0.500
pos: 0.794
overall: 0.681
Global (Full):
dir: 0.175
facing: 0.182
pos: 0.370
overall: 0.242
Local:
(none)
Local (Newly):
(none)
Fog Probe:
precision: 1
recall: 1
overall: 1
📈 Turn Metrics
node_coverage: 0.429
edge_coverage: 0.220
is_action_fail: False
step: 8
action_counts:
move: 3
rotate: 6
return: 0
observe: 8
term: 0
forced_term: 0
query: 0
information_gain: 0.398
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 8
Current state
State at Turn 8

🔄 Turn 9

💬 Assistant Action
Actions: [Rotate(-90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      3,
      3
    ],
    "facing": "west"
  },
  "green door": {
    "position": [
      3,
      3
    ]
  },
  "blue door": {
    "position": [
      -2,
      6
    ]
  },
  "table": {
    "position": [
      8,
      3
    ]
  },
  "bike": {
    "position": [
      8,
      5
    ],
    "facing": "south"
  },
  "chair": {
    "position": [
      6,
      5
    ],
    "facing": "west"
  },
  "truck": {
    "position": [
      6,
      1
    ],
    "facing": "north"
  },
  "laptop": {
    "position": [
      0,
      0
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      0,
      1
    ],
    "facing": "south"
  },
  "teddybear": {
    "position": [
      -1,
      2
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      4
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Observed)
{
  "laptop": {
    "position": [
      -4,
      -1
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      -2,
      1
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "teddybear": {
    "position": [
      -5,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -5,
      4
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      1,
      3
    ],
    "facing": "west"
  }
}
🎯 Ground Truth (Full)
{
  "laptop": {
    "position": [
      -4,
      -1
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "airplane": {
    "position": [
      -2,
      11
    ],
    "facing": "east"
  },
  "pan": {
    "position": [
      -2,
      1
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "lego man": {
    "position": [
      -6,
      9
    ],
    "facing": "north"
  },
  "teddybear": {
    "position": [
      -5,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -5,
      4
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "candle": {
    "position": [
      -6,
      11
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "fire extinguisher": {
    "position": [
      -2,
      7
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      1,
      3
    ],
    "facing": "west"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "blue door": {
      "position": [
        3,
        5
      ],
      "facing": "-x"
    },
    "desklamp": {
      "position": [
        1,
        6
      ],
      "facing": "+x"
    },
    "teddybear": {
      "position": [
        -1,
        6
      ],
      "facing": "+x"
    },
    "pan": {
      "position": [
        -2,
        2
      ],
      "facing": "-x"
    },
    "laptop": {
      "position": [
        -4,
        4
      ],
      "facing": "-x"
    }
  }
}
🎯 Ground Truth
{
  "desklamp": {
    "position": [
      1,
      6
    ],
    "facing": "north"
  },
  "laptop": {
    "position": [
      -4,
      5
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      -2,
      3
    ],
    "facing": "south"
  },
  "blue door": {
    "position": [
      2,
      4
    ],
    "facing": "north"
  },
  "teddybear": {
    "position": [
      -2,
      6
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "blue door": {
      "position": [
        3,
        5
      ],
      "facing": "-x"
    },
    "desklamp": {
      "position": [
        1,
        6
      ],
      "facing": "+x"
    },
    "teddybear": {
      "position": [
        -1,
        6
      ],
      "facing": "+x"
    },
    "pan": {
      "position": [
        -2,
        2
      ],
      "facing": "-x"
    },
    "laptop": {
      "position": [
        -4,
        4
      ],
      "facing": "-x"
    }
  }
}
🎯 Ground Truth
{
  "desklamp": {
    "position": [
      1,
      6
    ],
    "facing": "north"
  },
  "laptop": {
    "position": [
      -4,
      5
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      -2,
      3
    ],
    "facing": "south"
  },
  "teddybear": {
    "position": [
      -2,
      6
    ],
    "facing": "north"
  }
}
🗺️ Symbolic Fog Map
#################
#......##########
#......##########
#......##########
#......##########
#......##########
#......##########
#####+####......#
###..A...#......#
###....C.*......#
###......#......#
###......#......#
###....D.#......#
###..FEB.########
#################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: A, C
🎯 Ground Truth: D, B, E
🧠 Cognitive Map Metrics
Global:
dir: 0.788
facing: 1
pos: 0.703
overall: 0.830
Global (Full):
dir: 0.433
facing: 0.727
pos: 0.516
overall: 0.559
Local:
dir: 0.867
facing: 1
pos: 0.878
overall: 0.915
Local (Newly):
dir: 1
facing: 1
pos: 0.893
overall: 0.964
Fog Probe:
precision: 0
recall: 0
overall: 0
📈 Turn Metrics
node_coverage: 0.714
edge_coverage: 0.341
is_action_fail: False
step: 9
action_counts:
move: 3
rotate: 7
return: 0
observe: 9
term: 0
forced_term: 0
query: 0
information_gain: 0.601
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 9
Current state
State at Turn 9

🔄 Turn 10

💬 Assistant Action
Actions: [JumpTo(blue door), Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "grid": "N x M",
  "agent": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      4,
      2
    ],
    "facing": "west"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "laptop": {
    "position": [
      1,
      0
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      2,
      0
    ],
    "facing": "south"
  },
  "teddybear": {
    "position": [
      1,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      1,
      3
    ],
    "facing": "north"
  },
  "table": {
    "position": [
      9,
      2
    ]
  },
  "bike": {
    "position": [
      9,
      4
    ],
    "facing": "south"
  },
  "chair": {
    "position": [
      7,
      4
    ],
    "facing": "west"
  },
  "truck": {
    "position": [
      9,
      -1
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      0,
      9
    ],
    "facing": "east"
  },
  "lego man": {
    "position": [
      -5,
      9
    ],
    "facing": "north"
  },
  "candle": {
    "position": [
      -5,
      8
    ]
  },
  "fire extinguisher": {
    "position": [
      -1,
      7
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Observed)
{
  "laptop": {
    "position": [
      -4,
      -1
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "airplane": {
    "position": [
      -2,
      11
    ],
    "facing": "east"
  },
  "pan": {
    "position": [
      -2,
      1
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "lego man": {
    "position": [
      -6,
      9
    ],
    "facing": "north"
  },
  "teddybear": {
    "position": [
      -5,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -5,
      4
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "candle": {
    "position": [
      -6,
      11
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "fire extinguisher": {
    "position": [
      -2,
      7
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Full)
{
  "laptop": {
    "position": [
      -4,
      -1
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "airplane": {
    "position": [
      -2,
      11
    ],
    "facing": "east"
  },
  "pan": {
    "position": [
      -2,
      1
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "lego man": {
    "position": [
      -6,
      9
    ],
    "facing": "north"
  },
  "teddybear": {
    "position": [
      -5,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -5,
      4
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "candle": {
    "position": [
      -6,
      11
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "fire extinguisher": {
    "position": [
      -2,
      7
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "airplane": {
      "position": [
        2,
        5
      ],
      "facing": "+x"
    },
    "lego man": {
      "position": [
        -3,
        5
      ],
      "facing": "+y"
    },
    "candle": {
      "position": [
        -3,
        6
      ]
    },
    "fire extinguisher": {
      "position": [
        2,
        3
      ],
      "facing": "+y"
    }
  }
}
🎯 Ground Truth
{
  "fire extinguisher": {
    "position": [
      1,
      2
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      1,
      6
    ],
    "facing": "east"
  },
  "lego man": {
    "position": [
      -3,
      4
    ],
    "facing": "north"
  },
  "candle": {
    "position": [
      -3,
      6
    ],
    "facing": "east"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "airplane": {
      "position": [
        2,
        5
      ],
      "facing": "+x"
    },
    "lego man": {
      "position": [
        -3,
        5
      ],
      "facing": "+y"
    },
    "candle": {
      "position": [
        -3,
        6
      ]
    },
    "fire extinguisher": {
      "position": [
        2,
        3
      ],
      "facing": "+y"
    }
  }
}
🎯 Ground Truth
{
  "fire extinguisher": {
    "position": [
      1,
      2
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      1,
      6
    ],
    "facing": "east"
  },
  "lego man": {
    "position": [
      -3,
      4
    ],
    "facing": "north"
  },
  "candle": {
    "position": [
      -3,
      6
    ],
    "facing": "east"
  }
}
🗺️ Symbolic Fog Map
#################
#.A...I##########
#..G...##########
#......##########
#......##########
#KC....##########
#.D....##########
#####*####......#
###......#......#
###......+......#
###......#......#
###...BH.#......#
###...EF.#......#
###....LJ########
#################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: A, D, E, F, I, J, L
🎯 Ground Truth: L, F, J, C, D, K
🧠 Cognitive Map Metrics
Global:
dir: 0.650
facing: 1
pos: 0.617
overall: 0.756
Global (Full):
dir: 0.650
facing: 1
pos: 0.617
overall: 0.756
Local:
dir: 0.800
facing: 1
pos: 0.864
overall: 0.888
Local (Newly):
dir: 0.800
facing: 1
pos: 0.864
overall: 0.888
Fog Probe:
precision: 0.571
recall: 0.667
overall: 0.615
📈 Turn Metrics
node_coverage: 1
edge_coverage: 0.439
is_action_fail: False
step: 10
action_counts:
move: 4
rotate: 8
return: 0
observe: 10
term: 0
forced_term: 0
query: 0
information_gain: 0.746
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 10
Current state
State at Turn 10

🔄 Turn 11

💬 Assistant Action
Actions: [Term()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      -2,
      7
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      1,
      3
    ]
  },
  "blue door": {
    "position": [
      -2,
      7
    ]
  },
  "laptop": {
    "position": [
      0,
      1
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      -1,
      2
    ],
    "facing": "south"
  },
  "teddybear": {
    "position": [
      -2,
      2
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      0,
      5
    ],
    "facing": "north"
  },
  "table": {
    "position": [
      6,
      3
    ]
  },
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "fire extinguisher": {
    "position": [
      0,
      9
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      2,
      12
    ],
    "facing": "east"
  },
  "lego man": {
    "position": [
      -5,
      10
    ],
    "facing": "north"
  },
  "candle": {
    "position": [
      -5,
      12
    ]
  }
}
🎯 Ground Truth (Observed)
{
  "laptop": {
    "position": [
      -4,
      -1
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "airplane": {
    "position": [
      -2,
      11
    ],
    "facing": "east"
  },
  "pan": {
    "position": [
      -2,
      1
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "lego man": {
    "position": [
      -6,
      9
    ],
    "facing": "north"
  },
  "teddybear": {
    "position": [
      -5,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -5,
      4
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "candle": {
    "position": [
      -6,
      11
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "fire extinguisher": {
    "position": [
      -2,
      7
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Full)
{
  "laptop": {
    "position": [
      -4,
      -1
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      6,
      5
    ],
    "facing": "south"
  },
  "airplane": {
    "position": [
      -2,
      11
    ],
    "facing": "east"
  },
  "pan": {
    "position": [
      -2,
      1
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      4,
      1
    ],
    "facing": "north"
  },
  "lego man": {
    "position": [
      -6,
      9
    ],
    "facing": "north"
  },
  "teddybear": {
    "position": [
      -5,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -5,
      4
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      4,
      5
    ],
    "facing": "west"
  },
  "candle": {
    "position": [
      -6,
      11
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      6,
      3
    ],
    "facing": "east"
  },
  "fire extinguisher": {
    "position": [
      -2,
      7
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      -3,
      5
    ],
    "facing": "north"
  }
}
🧠 Cognitive Map Metrics
Global:
dir: 0.708
facing: 1
pos: 0.718
overall: 0.809
Global (Full):
dir: 0.708
facing: 1
pos: 0.718
overall: 0.809
Local:
(none)
Local (Newly):
(none)
Fog Probe:
(none)
📈 Turn Metrics
node_coverage: 1
edge_coverage: 0.439
is_action_fail: False
step: 11
action_counts:
move: 4
rotate: 8
return: 0
observe: 10
term: 1
forced_term: 0
query: 0
information_gain: 0.746
reward: -0.100
is_done: True
is_valid_action: True
Previous state
State before Turn 11
Current state
State at Turn 11

🧭 False Belief Exploration

🔄 FB Turn 12

💬 Assistant Action
Actions: [Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Unchanged (all):
dir: 0.712
facing: 0.857
pos: 0.757
overall: 0.775
🧾 False Belief Object Sets
all_objects: ['airplane', 'bike', 'blue door', 'candle', 'chair', 'desklamp', 'fire extinguisher', 'green door', 'laptop', 'lego man', 'pan', 'table', 'teddybear', 'truck']
all_changed_objects: ['airplane', 'desklamp', 'fire extinguisher', 'teddybear']
newly_observed_changed (this turn): []
unchanged_objects: ['bike', 'blue door', 'candle', 'chair', 'green door', 'laptop', 'lego man', 'pan', 'table', 'truck']
pred_changed_keys: []
pred_unchanged_keys: ['airplane', 'bike', 'blue door', 'candle', 'chair', 'desklamp', 'fire extinguisher', 'green door', 'laptop', 'lego man', 'pan', 'table', 'teddybear', 'truck']
🧭 False Belief Info
step: 1
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'airplane', 'ori': True}, {'name': 'desklamp', 'ori': True}, {'name': 'teddybear', 'pos': True}, {'name': 'fire extinguisher', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: []
newly_observed_unchanged_objects: ['green door', 'blue door']
Current state
State at Turn 1

🔄 FB Turn 13

💬 Assistant Action
Actions: [Rotate(-90), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Changed: fire extinguisher:
dir: None
pos: 1
facing: None
overall: None
Retention: fire extinguisher:
dir: None
pos: 0.328
facing: None
overall: None
Changed: desklamp:
dir: None
pos: None
facing: 1
overall: None
Retention: desklamp:
dir: None
pos: None
facing: 0
overall: None
Unchanged (all):
dir: 0.682
facing: 0.714
pos: 0.761
overall: 0.719
🧾 False Belief Object Sets
all_objects: ['airplane', 'bike', 'blue door', 'candle', 'chair', 'desklamp', 'fire extinguisher', 'green door', 'laptop', 'lego man', 'pan', 'table', 'teddybear', 'truck']
all_changed_objects: ['airplane', 'desklamp', 'fire extinguisher', 'teddybear']
newly_observed_changed (this turn): ['desklamp', 'fire extinguisher']
unchanged_objects: ['bike', 'blue door', 'candle', 'chair', 'green door', 'laptop', 'lego man', 'pan', 'table', 'truck']
pred_changed_keys: ['desklamp', 'fire extinguisher']
pred_unchanged_keys: ['airplane', 'bike', 'blue door', 'candle', 'chair', 'desklamp', 'fire extinguisher', 'green door', 'laptop', 'lego man', 'pan', 'table', 'teddybear', 'truck']
🧭 False Belief Info
step: 2
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'airplane', 'ori': True}, {'name': 'desklamp', 'ori': True}, {'name': 'teddybear', 'pos': True}, {'name': 'fire extinguisher', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: ['fire extinguisher', 'desklamp']
newly_observed_unchanged_objects: ['laptop', 'pan']
Previous state
State before Turn 2
Current state
State at Turn 2

🔄 FB Turn 14

💬 Assistant Action
Actions: [JumpTo(blue door), Rotate(90), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Unchanged (all):
dir: 0.424
facing: 0.286
pos: 0.628
overall: 0.446
🧾 False Belief Object Sets
all_objects: ['airplane', 'bike', 'blue door', 'candle', 'chair', 'desklamp', 'fire extinguisher', 'green door', 'laptop', 'lego man', 'pan', 'table', 'teddybear', 'truck']
all_changed_objects: ['airplane', 'desklamp', 'fire extinguisher', 'teddybear']
newly_observed_changed (this turn): []
unchanged_objects: ['bike', 'blue door', 'candle', 'chair', 'green door', 'laptop', 'lego man', 'pan', 'table', 'truck']
pred_changed_keys: []
pred_unchanged_keys: ['airplane', 'bike', 'blue door', 'candle', 'chair', 'desklamp', 'fire extinguisher', 'green door', 'laptop', 'lego man', 'pan', 'table', 'teddybear', 'truck']
🧭 False Belief Info
step: 3
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'airplane', 'ori': True}, {'name': 'desklamp', 'ori': True}, {'name': 'teddybear', 'pos': True}, {'name': 'fire extinguisher', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: []
newly_observed_unchanged_objects: []
Previous state
State before Turn 3
Current state
State at Turn 3

🔄 FB Turn 15

💬 Assistant Action
Actions: [Rotate(90), JumpTo(blue door), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Changed: airplane:
dir: None
pos: None
facing: 1
overall: None
Retention: airplane:
dir: None
pos: None
facing: 0
overall: None
Changed: teddybear:
dir: None
pos: 0.625
facing: None
overall: None
Retention: teddybear:
dir: None
pos: 0.514
facing: None
overall: None
Unchanged (all):
dir: 0.591
facing: 1
pos: 0.626
overall: 0.739
🧾 False Belief Object Sets
all_objects: ['airplane', 'bike', 'blue door', 'candle', 'chair', 'desklamp', 'fire extinguisher', 'green door', 'laptop', 'lego man', 'pan', 'table', 'teddybear', 'truck']
all_changed_objects: ['airplane', 'desklamp', 'fire extinguisher', 'teddybear']
newly_observed_changed (this turn): ['airplane', 'teddybear']
unchanged_objects: ['bike', 'blue door', 'candle', 'chair', 'green door', 'laptop', 'lego man', 'pan', 'table', 'truck']
pred_changed_keys: ['airplane', 'teddybear']
pred_unchanged_keys: ['airplane', 'bike', 'blue door', 'candle', 'chair', 'desklamp', 'fire extinguisher', 'green door', 'laptop', 'lego man', 'pan', 'table', 'teddybear', 'truck']
🧭 False Belief Info
step: 4
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'airplane', 'ori': True}, {'name': 'desklamp', 'ori': True}, {'name': 'teddybear', 'pos': True}, {'name': 'fire extinguisher', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: ['airplane', 'teddybear']
newly_observed_unchanged_objects: ['lego man', 'candle']
Previous state
State before Turn 4
Current state
State at Turn 4

🔄 FB Turn 16

💬 Assistant Action
Actions: [Term(changes="fire extinguisher: position, desklamp: orientation, airplane: orientation, teddybear: position")]
📈 Turn Metrics
reward: 1
is_done: True
is_valid_action: True
success: 1
🧭 False Belief Info
step: 5
correctly_identified_changes: 1
f1_overall: 1
f1_position: 1
f1_facing: 1
action_cost: 4
action_cost_after_seen: 0
ground_truth_changes: [{'name': 'airplane', 'ori': True}, {'name': 'desklamp', 'ori': True}, {'name': 'teddybear', 'pos': True}, {'name': 'fire extinguisher', 'pos': True}]
reported_changes: [{'name': 'fire extinguisher', 'pos': True}, {'name': 'desklamp', 'ori': True}, {'name': 'airplane', 'ori': True}, {'name': 'teddybear', 'pos': True}]
newly_observed_changed_objects: []
newly_observed_unchanged_objects: []
Previous state
State before Turn 5
Current state
State at Turn 5

📊 Evaluation (default)

📊 Task: DirectionEvaluationTask

Question 1 (ID: ca87148432a7d552)

✅ Evaluation Results
id: ca87148432a7d552
question: You return to your starting position and face north. From a Top-Down map, describe where airplane is relative to laptop. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q1: DirectionEvaluationTask

Question 2 (ID: e031d760b96a5da9)

✅ Evaluation Results
id: e031d760b96a5da9
question: You return to your starting position and face north. From a Top-Down map, describe where chair is relative to pan. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north east, slightly far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q2: DirectionEvaluationTask

Question 3 (ID: f91d7ef70302b179)

✅ Evaluation Results
id: f91d7ef70302b179
question: You return to your starting position and face north. From a Top-Down map, describe where lego man is relative to pan. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north west, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q3: DirectionEvaluationTask

📊 Task: PovEvaluationTask

Question 1 (ID: b39725205f4d5931)

✅ Evaluation Results
id: b39725205f4d5931
question: Now you jump to teddybear's direction, facing its direction. Describe where desklamp is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q1: PovEvaluationTask

Question 2 (ID: ddaba1a0c4c8aaed)

✅ Evaluation Results
id: ddaba1a0c4c8aaed
question: Now you jump to pan's direction, facing its direction. Describe where laptop is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-right, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q2: PovEvaluationTask

Question 3 (ID: 583f1b341e5291a0)

✅ Evaluation Results
id: 583f1b341e5291a0
question: Now you jump to bike's direction, facing its direction. Describe where table is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front, near
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q3: PovEvaluationTask

📊 Task: BackwardPovTextEvaluationTask

Question 1 (ID: 38b08b9cd49edf9d)

✅ Evaluation Results
id: 38b08b9cd49edf9d
question: Now you jump to an object's position, facing its direction. You observe that desklamp is front, mid distance, facing forward; blue door is front-right, slightly far, on front wall. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: teddybear
final_pos: [3, 3]
final_ori: [0, 1]
final_observation: [{'name': 'desklamp', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'blue door', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
object_positions:
laptop: [4.0, 1.0]
bike: [14.0, 7.0]
airplane: [6.0, 13.0]
pan: [6.0, 3.0]
truck: [12.0, 3.0]
lego man: [2.0, 11.0]
teddybear: [3.0, 3.0]
desklamp: [3.0, 6.0]
chair: [12.0, 7.0]
candle: [2.0, 13.0]
table: [14.0, 5.0]
fire extinguisher: [6.0, 9.0]
green door: [9.0, 5.0]
blue door: [5.0, 7.0]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
room_id: 1
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: default
Evaluation state
Q1: BackwardPovTextEvaluationTask

Question 2 (ID: 593eaa3f0e5ee139)

✅ Evaluation Results
id: 593eaa3f0e5ee139
question: Now you jump to an object's position, facing its direction. You observe that bike is front-right, slightly far, facing backward; chair is front, mid distance, facing left; table is front-right, mid distance. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: truck
final_pos: [12, 3]
final_ori: [0, 1]
final_observation: [{'name': 'bike', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing backward'}, {'name': 'chair', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing left'}, {'name': 'table', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': None}]
object_positions:
laptop: [4.0, 1.0]
bike: [14.0, 7.0]
airplane: [6.0, 13.0]
pan: [6.0, 3.0]
truck: [12.0, 3.0]
lego man: [2.0, 11.0]
teddybear: [3.0, 3.0]
desklamp: [3.0, 6.0]
chair: [12.0, 7.0]
candle: [2.0, 13.0]
table: [14.0, 5.0]
fire extinguisher: [6.0, 9.0]
green door: [9.0, 5.0]
blue door: [5.0, 7.0]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
room_id: 2
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: default
Evaluation state
Q2: BackwardPovTextEvaluationTask

Question 3 (ID: 8478901165ea117a)

✅ Evaluation Results
id: 8478901165ea117a
question: Now you jump to an object's position, facing its direction. You observe that airplane is front, mid distance, facing right; candle is front-left, slightly far. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: fire extinguisher
final_pos: [6, 9]
final_ori: [0, 1]
final_observation: [{'name': 'airplane', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'candle', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': None}]
object_positions:
laptop: [4.0, 1.0]
bike: [14.0, 7.0]
airplane: [6.0, 13.0]
pan: [6.0, 3.0]
truck: [12.0, 3.0]
lego man: [2.0, 11.0]
teddybear: [3.0, 3.0]
desklamp: [3.0, 6.0]
chair: [12.0, 7.0]
candle: [2.0, 13.0]
table: [14.0, 5.0]
fire extinguisher: [6.0, 9.0]
green door: [9.0, 5.0]
blue door: [5.0, 7.0]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
room_id: 3
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: default
Evaluation state
Q3: BackwardPovTextEvaluationTask

📊 Task: Action2ViewEvaluationTask

Question 1 (ID: 21808bda9b228db0)

✅ Evaluation Results
id: 21808bda9b228db0
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(-90) 2. Jump to the object at front-right, mid distance. 3. Rotate(-180) 4. Jump to the object at front-left, mid distance. 5. Rotate(-180) After executing the actions, what is the ego relation of teddybear relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-slight-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Evaluation state
Q1: Action2ViewEvaluationTask

Question 2 (ID: ab43fa08726d1772)

✅ Evaluation Results
id: ab43fa08726d1772
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(-90) 2. Jump to the object at front-slight-right, slightly far. 3. Rotate(90) 4. Jump to the object at front-right, slightly far. After executing the actions, what is the ego relation of fire extinguisher relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-right, mid distance
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q2: Action2ViewEvaluationTask

Question 3 (ID: 604bedeabbf32688)

✅ Evaluation Results
id: 604bedeabbf32688
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(-90) 2. Jump to the object at front-right, slightly far. 3. Rotate(-90) 4. Jump to the object at front, mid distance. 5. Rotate(-90) After executing the actions, what is the ego relation of green door relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-slight-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Evaluation state
Q3: Action2ViewEvaluationTask

📊 Task: View2ActionTextEvaluationTask

Question 1 (ID: 1a5402f686fdbe8e)

✅ Evaluation Results
id: 1a5402f686fdbe8e
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: candle is at front, near What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [2, 11]
final_ori: [0, 1]
room_id: 3
init_pos: [8, 2]
init_ori: [0, 1]
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
minimal_plan: [['jumpto', 'blue door'], ['jumpto', 'lego man']]
final_observation: [{'name': 'candle', 'direction': 'front', 'distance': 'near', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [2.0, 11.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: default
Evaluation state
Q1: View2ActionTextEvaluationTask

Question 2 (ID: 0240108a1a776391)

✅ Evaluation Results
id: 0240108a1a776391
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: bike is at front, near, facing backward; chair is at front-left, mid distance, facing left What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [14, 5]
final_ori: [0, 1]
room_id: 2
init_pos: [8, 2]
init_ori: [0, 1]
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
minimal_plan: [['jumpto', 'green door'], ['rotate', 90], ['jumpto', 'table'], ['rotate', -90]]
final_observation: [{'name': 'bike', 'direction': 'front', 'distance': 'near', 'orientation': 'facing backward'}, {'name': 'chair', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing left'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [14.0, 5.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: default
Evaluation state
Q2: View2ActionTextEvaluationTask

Question 3 (ID: fe579174d12a0b9e)

✅ Evaluation Results
id: fe579174d12a0b9e
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: green door is at front-left, mid distance, on front wall What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [6, 3]
final_ori: [1, 0]
room_id: 1
init_pos: [8, 2]
init_ori: [0, 1]
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
minimal_plan: [['rotate', -90], ['jumpto', 'pan'], ['rotate', 180]]
final_observation: [{'name': 'green door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on front wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: False
final_pos: [12.0, 7.0]
final_ori: [-1, 0]
visible_match: False
score: 0
evaluation_mode: default
Evaluation state
Q3: View2ActionTextEvaluationTask

📊 Task: AlloMappingEvaluationTask

Question 1 (ID: 046899a176fbe951)

✅ Evaluation Results
id: 046899a176fbe951
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: airplane, laptop, candle, table, fire extinguisher. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-2, 11], [-4, -1], [-6, 11], [6, 3], [-2, 7]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.897
threshold: 0.900
score: 0.897
evaluation_mode: default
Evaluation state
Q1: AlloMappingEvaluationTask

Question 2 (ID: 64c2cfa4dfb804b2)

✅ Evaluation Results
id: 64c2cfa4dfb804b2
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: chair, pan, lego man. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[4, 5], [-2, 1], [-6, 9]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.948
threshold: 0.900
score: 0.948
evaluation_mode: default
Evaluation state
Q2: AlloMappingEvaluationTask

Question 3 (ID: bf689311469baa8a)

✅ Evaluation Results
id: bf689311469baa8a
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: lego man, pan, bike, fire extinguisher, airplane. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-6, 9], [-2, 1], [6, 5], [-2, 7], [-2, 11]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.750
threshold: 0.900
score: 0.750
evaluation_mode: default
Evaluation state
Q3: AlloMappingEvaluationTask

📊 Task: RotEvaluationTask

Question 1 (ID: 79135364fb4b164f)

✅ Evaluation Results
id: 79135364fb4b164f
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning counterclockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: airplane, desklamp, laptop. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['airplane', 'desklamp', 'laptop']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q1: RotEvaluationTask

Question 2 (ID: 75467a0fbc94e6b7)

✅ Evaluation Results
id: 75467a0fbc94e6b7
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: chair, laptop, truck. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['chair', 'truck', 'laptop']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q2: RotEvaluationTask

Question 3 (ID: 456b022cad9cba9a)

✅ Evaluation Results
id: 456b022cad9cba9a
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning counterclockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: bike, lego man, teddybear. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['lego man', 'teddybear', 'bike']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q3: RotEvaluationTask

📊 Task: RotDualEvaluationTask

Question 1 (ID: 63b9f1a78aaf3b24)

✅ Evaluation Results
id: 63b9f1a78aaf3b24
question: You return to your starting position and face north. You performed a complete 360° rotation in place. Assume all walls are removed (you can see through walls), so every object is visible. During the rotation, these objects appeared directly in front of you in this order: airplane, desklamp, laptop Based on this sequence, in which direction did you rotate? Answer format: clockwise or counterclockwise Example: clockwise
action: None
answer: counterclockwise
task_type: RotDualEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q1: RotDualEvaluationTask

Question 2 (ID: ea901f1db8ed2f4b)

✅ Evaluation Results
id: ea901f1db8ed2f4b
question: You return to your starting position and face north. You performed a complete 360° rotation in place. Assume all walls are removed (you can see through walls), so every object is visible. During the rotation, these objects appeared directly in front of you in this order: chair, truck, laptop Based on this sequence, in which direction did you rotate? Answer format: clockwise or counterclockwise Example: clockwise
action: None
answer: clockwise
task_type: RotDualEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q2: RotDualEvaluationTask

Question 3 (ID: 1546ca0f231b289d)

✅ Evaluation Results
id: 1546ca0f231b289d
question: You return to your starting position and face north. You performed a complete 360° rotation in place. Assume all walls are removed (you can see through walls), so every object is visible. During the rotation, these objects appeared directly in front of you in this order: lego man, teddybear, bike Based on this sequence, in which direction did you rotate? Answer format: clockwise or counterclockwise Example: clockwise
action: None
answer: counterclockwise
task_type: RotDualEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q3: RotDualEvaluationTask

📊 Task: Location2ViewEvaluationTask

Question 1 (ID: 77fb7413d8ff5f85)

✅ Evaluation Results
id: 77fb7413d8ff5f85
question: Still treat your initial position as origin (0, 0) You move to (-3, 2) and face west. What is the egocentric relation of teddybear? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-left, mid distance
task_type: Action2LocationEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: default
Evaluation state
Q1: Location2ViewEvaluationTask

Question 2 (ID: 3399c355c7d0785f)

✅ Evaluation Results
id: 3399c355c7d0785f
question: Still treat your initial position as origin (0, 0) You move to (-1, 0) and face west. What is the egocentric relation of pan? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-right, near
task_type: Action2LocationEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Evaluation state
Q2: Location2ViewEvaluationTask

Question 3 (ID: 1c9e744fd28c4e95)

✅ Evaluation Results
id: 1c9e744fd28c4e95
question: Treat the blue door as the new 'origin' (0, 0). You move to (0, 3) and face north. What is the egocentric relation of airplane? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-slight-right, mid distance
task_type: Action2LocationEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Evaluation state
Q3: Location2ViewEvaluationTask

📊 Task: View2LocationTextEvaluationTask

Question 1 (ID: 981cabb8e8f95e0e)

✅ Evaluation Results
id: 981cabb8e8f95e0e
question: You move to a new location and face west. You observe: teddybear is at front-left, mid distance, facing right; desklamp is at front-right, mid distance, facing right Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [-3, 2]
final_pos: [5, 4]
final_ori: [-1, 0]
room_id: 1
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'teddybear', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'desklamp', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing right'}]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: Location2ActionTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.676
raw_score: 0.676
score: 0.676
evaluation_mode: default
Evaluation state
Q1: View2LocationTextEvaluationTask

Question 2 (ID: 56bb180ac8571782)

✅ Evaluation Results
id: 56bb180ac8571782
question: You move to a new location and face west. You observe: pan is at front-right, near, facing left; teddybear is at front-slight-right, slightly far, facing right; laptop is at front-slight-left, mid distance, facing left Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [-1, 0]
final_pos: [7, 2]
final_ori: [-1, 0]
room_id: 1
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'pan', 'direction': 'front-right', 'distance': 'near', 'orientation': 'facing left'}, {'name': 'teddybear', 'direction': 'front-slight-right', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'laptop', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing left'}]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: Location2ActionTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.243
raw_score: 0.243
score: 0.243
evaluation_mode: default
Evaluation state
Q2: View2LocationTextEvaluationTask

Question 3 (ID: d06f40c11fae70c2)

✅ Evaluation Results
id: d06f40c11fae70c2
question: You move to a new location and face north. You observe: airplane is at front-slight-right, mid distance, facing right; candle is at front-left, slightly far Treat the blue door as the new 'origin' (0, 0). What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [0, 3]
final_pos: [5, 10]
final_ori: [0, 1]
room_id: 3
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'airplane', 'direction': 'front-slight-right', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'candle', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': None}]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: Location2ActionTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.717
raw_score: 0.717
score: 0.717
evaluation_mode: default
Evaluation state
Q3: View2LocationTextEvaluationTask

📊 Evaluation (prompt_cogmap)

📊 Task: DirectionEvaluationTask

Question 1 (ID: ca87148432a7d552)

✅ Evaluation Results
id: ca87148432a7d552
question: You return to your starting position and face north. From a Top-Down map, describe where airplane is relative to laptop. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q1: DirectionEvaluationTask

Question 2 (ID: e031d760b96a5da9)

✅ Evaluation Results
id: e031d760b96a5da9
question: You return to your starting position and face north. From a Top-Down map, describe where chair is relative to pan. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north east, slightly far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q2: DirectionEvaluationTask

Question 3 (ID: f91d7ef70302b179)

✅ Evaluation Results
id: f91d7ef70302b179
question: You return to your starting position and face north. From a Top-Down map, describe where lego man is relative to pan. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north west, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q3: DirectionEvaluationTask

📊 Task: PovEvaluationTask

Question 1 (ID: b39725205f4d5931)

✅ Evaluation Results
id: b39725205f4d5931
question: Now you jump to teddybear's direction, facing its direction. Describe where desklamp is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Evaluation state
Q1: PovEvaluationTask

Question 2 (ID: ddaba1a0c4c8aaed)

✅ Evaluation Results
id: ddaba1a0c4c8aaed
question: Now you jump to pan's direction, facing its direction. Describe where laptop is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-right, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Evaluation state
Q2: PovEvaluationTask

Question 3 (ID: 583f1b341e5291a0)

✅ Evaluation Results
id: 583f1b341e5291a0
question: Now you jump to bike's direction, facing its direction. Describe where table is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front, near
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q3: PovEvaluationTask

📊 Task: BackwardPovTextEvaluationTask

Question 1 (ID: 38b08b9cd49edf9d)

✅ Evaluation Results
id: 38b08b9cd49edf9d
question: Now you jump to an object's position, facing its direction. You observe that desklamp is front, mid distance, facing forward; blue door is front-right, slightly far, on front wall. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: teddybear
final_pos: [3, 3]
final_ori: [0, 1]
final_observation: [{'name': 'desklamp', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'blue door', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
object_positions:
laptop: [4.0, 1.0]
bike: [14.0, 7.0]
airplane: [6.0, 13.0]
pan: [6.0, 3.0]
truck: [12.0, 3.0]
lego man: [2.0, 11.0]
teddybear: [3.0, 3.0]
desklamp: [3.0, 6.0]
chair: [12.0, 7.0]
candle: [2.0, 13.0]
table: [14.0, 5.0]
fire extinguisher: [6.0, 9.0]
green door: [9.0, 5.0]
blue door: [5.0, 7.0]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
room_id: 1
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q1: BackwardPovTextEvaluationTask

Question 2 (ID: 593eaa3f0e5ee139)

✅ Evaluation Results
id: 593eaa3f0e5ee139
question: Now you jump to an object's position, facing its direction. You observe that bike is front-right, slightly far, facing backward; chair is front, mid distance, facing left; table is front-right, mid distance. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: truck
final_pos: [12, 3]
final_ori: [0, 1]
final_observation: [{'name': 'bike', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing backward'}, {'name': 'chair', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing left'}, {'name': 'table', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': None}]
object_positions:
laptop: [4.0, 1.0]
bike: [14.0, 7.0]
airplane: [6.0, 13.0]
pan: [6.0, 3.0]
truck: [12.0, 3.0]
lego man: [2.0, 11.0]
teddybear: [3.0, 3.0]
desklamp: [3.0, 6.0]
chair: [12.0, 7.0]
candle: [2.0, 13.0]
table: [14.0, 5.0]
fire extinguisher: [6.0, 9.0]
green door: [9.0, 5.0]
blue door: [5.0, 7.0]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
room_id: 2
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q2: BackwardPovTextEvaluationTask

Question 3 (ID: 8478901165ea117a)

✅ Evaluation Results
id: 8478901165ea117a
question: Now you jump to an object's position, facing its direction. You observe that airplane is front, mid distance, facing right; candle is front-left, slightly far. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: fire extinguisher
final_pos: [6, 9]
final_ori: [0, 1]
final_observation: [{'name': 'airplane', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'candle', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': None}]
object_positions:
laptop: [4.0, 1.0]
bike: [14.0, 7.0]
airplane: [6.0, 13.0]
pan: [6.0, 3.0]
truck: [12.0, 3.0]
lego man: [2.0, 11.0]
teddybear: [3.0, 3.0]
desklamp: [3.0, 6.0]
chair: [12.0, 7.0]
candle: [2.0, 13.0]
table: [14.0, 5.0]
fire extinguisher: [6.0, 9.0]
green door: [9.0, 5.0]
blue door: [5.0, 7.0]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
room_id: 3
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q3: BackwardPovTextEvaluationTask

📊 Task: Action2ViewEvaluationTask

Question 1 (ID: 21808bda9b228db0)

✅ Evaluation Results
id: 21808bda9b228db0
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(-90) 2. Jump to the object at front-right, mid distance. 3. Rotate(-180) 4. Jump to the object at front-left, mid distance. 5. Rotate(-180) After executing the actions, what is the ego relation of teddybear relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-slight-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q1: Action2ViewEvaluationTask

Question 2 (ID: 604bedeabbf32688)

✅ Evaluation Results
id: 604bedeabbf32688
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(-90) 2. Jump to the object at front-right, slightly far. 3. Rotate(-90) 4. Jump to the object at front, mid distance. 5. Rotate(-90) After executing the actions, what is the ego relation of green door relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-slight-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q2: Action2ViewEvaluationTask

Question 3 (ID: ab43fa08726d1772)

✅ Evaluation Results
id: ab43fa08726d1772
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(-90) 2. Jump to the object at front-slight-right, slightly far. 3. Rotate(90) 4. Jump to the object at front-right, slightly far. After executing the actions, what is the ego relation of fire extinguisher relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-right, mid distance
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q3: Action2ViewEvaluationTask

📊 Task: View2ActionTextEvaluationTask

Question 1 (ID: 1a5402f686fdbe8e)

✅ Evaluation Results
id: 1a5402f686fdbe8e
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: candle is at front, near What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [2, 11]
final_ori: [0, 1]
room_id: 3
init_pos: [8, 2]
init_ori: [0, 1]
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
minimal_plan: [['jumpto', 'blue door'], ['jumpto', 'lego man']]
final_observation: [{'name': 'candle', 'direction': 'front', 'distance': 'near', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [2.0, 11.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q1: View2ActionTextEvaluationTask

Question 2 (ID: 0240108a1a776391)

✅ Evaluation Results
id: 0240108a1a776391
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: bike is at front, near, facing backward; chair is at front-left, mid distance, facing left What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [14, 5]
final_ori: [0, 1]
room_id: 2
init_pos: [8, 2]
init_ori: [0, 1]
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
minimal_plan: [['jumpto', 'green door'], ['rotate', 90], ['jumpto', 'table'], ['rotate', -90]]
final_observation: [{'name': 'bike', 'direction': 'front', 'distance': 'near', 'orientation': 'facing backward'}, {'name': 'chair', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing left'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [14.0, 5.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q2: View2ActionTextEvaluationTask

Question 3 (ID: fe579174d12a0b9e)

✅ Evaluation Results
id: fe579174d12a0b9e
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: green door is at front-left, mid distance, on front wall What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [6, 3]
final_ori: [1, 0]
room_id: 1
init_pos: [8, 2]
init_ori: [0, 1]
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
minimal_plan: [['rotate', -90], ['jumpto', 'pan'], ['rotate', 180]]
final_observation: [{'name': 'green door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on front wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [6.0, 3.0]
final_ori: [1, 0]
visible_match: True
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q3: View2ActionTextEvaluationTask

📊 Task: AlloMappingEvaluationTask

Question 1 (ID: 046899a176fbe951)

✅ Evaluation Results
id: 046899a176fbe951
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: airplane, laptop, candle, table, fire extinguisher. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-2, 11], [-4, -1], [-6, 11], [6, 3], [-2, 7]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.757
threshold: 0.900
score: 0.757
evaluation_mode: prompt_cogmap
Evaluation state
Q1: AlloMappingEvaluationTask

Question 2 (ID: 64c2cfa4dfb804b2)

✅ Evaluation Results
id: 64c2cfa4dfb804b2
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: chair, pan, lego man. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[4, 5], [-2, 1], [-6, 9]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.769
threshold: 0.900
score: 0.769
evaluation_mode: prompt_cogmap
Evaluation state
Q2: AlloMappingEvaluationTask

Question 3 (ID: bf689311469baa8a)

✅ Evaluation Results
id: bf689311469baa8a
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: lego man, pan, bike, fire extinguisher, airplane. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-6, 9], [-2, 1], [6, 5], [-2, 7], [-2, 11]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.890
threshold: 0.900
score: 0.890
evaluation_mode: prompt_cogmap
Evaluation state
Q3: AlloMappingEvaluationTask

📊 Task: RotEvaluationTask

Question 1 (ID: 79135364fb4b164f)

✅ Evaluation Results
id: 79135364fb4b164f
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning counterclockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: airplane, desklamp, laptop. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['airplane', 'desklamp', 'laptop']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q1: RotEvaluationTask

Question 2 (ID: 75467a0fbc94e6b7)

✅ Evaluation Results
id: 75467a0fbc94e6b7
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: chair, laptop, truck. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['chair', 'truck', 'laptop']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q2: RotEvaluationTask

Question 3 (ID: 456b022cad9cba9a)

✅ Evaluation Results
id: 456b022cad9cba9a
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning counterclockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: bike, lego man, teddybear. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['lego man', 'teddybear', 'bike']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q3: RotEvaluationTask

📊 Task: Location2ViewEvaluationTask

Question 1 (ID: 77fb7413d8ff5f85)

✅ Evaluation Results
id: 77fb7413d8ff5f85
question: Still treat your initial position as origin (0, 0) You move to (-3, 2) and face west. What is the egocentric relation of teddybear? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-left, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: prompt_cogmap
Evaluation state
Q1: Location2ViewEvaluationTask

Question 2 (ID: 3399c355c7d0785f)

✅ Evaluation Results
id: 3399c355c7d0785f
question: Still treat your initial position as origin (0, 0) You move to (-1, 0) and face west. What is the egocentric relation of pan? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-right, near
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Evaluation state
Q2: Location2ViewEvaluationTask

Question 3 (ID: 1c9e744fd28c4e95)

✅ Evaluation Results
id: 1c9e744fd28c4e95
question: Treat the blue door as the new 'origin' (0, 0). You move to (0, 3) and face north. What is the egocentric relation of airplane? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-slight-right, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Evaluation state
Q3: Location2ViewEvaluationTask

📊 Task: View2LocationTextEvaluationTask

Question 1 (ID: 981cabb8e8f95e0e)

✅ Evaluation Results
id: 981cabb8e8f95e0e
question: You move to a new location and face west. You observe: teddybear is at front-left, mid distance, facing right; desklamp is at front-right, mid distance, facing right Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [-3, 2]
final_pos: [5, 4]
final_ori: [-1, 0]
room_id: 1
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'teddybear', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'desklamp', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing right'}]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.330
raw_score: 0.330
score: 0.330
evaluation_mode: prompt_cogmap
Evaluation state
Q1: View2LocationTextEvaluationTask

Question 2 (ID: 56bb180ac8571782)

✅ Evaluation Results
id: 56bb180ac8571782
question: You move to a new location and face west. You observe: pan is at front-right, near, facing left; teddybear is at front-slight-right, slightly far, facing right; laptop is at front-slight-left, mid distance, facing left Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [-1, 0]
final_pos: [7, 2]
final_ori: [-1, 0]
room_id: 1
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'pan', 'direction': 'front-right', 'distance': 'near', 'orientation': 'facing left'}, {'name': 'teddybear', 'direction': 'front-slight-right', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'laptop', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing left'}]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.027
raw_score: 0.027
score: 0.027
evaluation_mode: prompt_cogmap
Evaluation state
Q2: View2LocationTextEvaluationTask

Question 3 (ID: d06f40c11fae70c2)

✅ Evaluation Results
id: d06f40c11fae70c2
question: You move to a new location and face north. You observe: airplane is at front-slight-right, mid distance, facing right; candle is at front-left, slightly far Treat the blue door as the new 'origin' (0, 0). What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [0, 3]
final_pos: [5, 10]
final_ori: [0, 1]
room_id: 3
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'airplane', 'direction': 'front-slight-right', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'candle', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': None}]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q3: View2LocationTextEvaluationTask

📊 Evaluation (use_gt_cogmap)

📊 Task: DirectionEvaluationTask

Question 1 (ID: ca87148432a7d552)

✅ Evaluation Results
id: ca87148432a7d552
question: You return to your starting position and face north. From a Top-Down map, describe where airplane is relative to laptop. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: DirectionEvaluationTask

Question 2 (ID: e031d760b96a5da9)

✅ Evaluation Results
id: e031d760b96a5da9
question: You return to your starting position and face north. From a Top-Down map, describe where chair is relative to pan. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north east, slightly far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: DirectionEvaluationTask

Question 3 (ID: f91d7ef70302b179)

✅ Evaluation Results
id: f91d7ef70302b179
question: You return to your starting position and face north. From a Top-Down map, describe where lego man is relative to pan. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north west, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: DirectionEvaluationTask

📊 Task: PovEvaluationTask

Question 1 (ID: b39725205f4d5931)

✅ Evaluation Results
id: b39725205f4d5931
question: Now you jump to teddybear's direction, facing its direction. Describe where desklamp is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: PovEvaluationTask

Question 2 (ID: ddaba1a0c4c8aaed)

✅ Evaluation Results
id: ddaba1a0c4c8aaed
question: Now you jump to pan's direction, facing its direction. Describe where laptop is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-right, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: PovEvaluationTask

Question 3 (ID: 583f1b341e5291a0)

✅ Evaluation Results
id: 583f1b341e5291a0
question: Now you jump to bike's direction, facing its direction. Describe where table is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front, near
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: PovEvaluationTask

📊 Task: BackwardPovTextEvaluationTask

Question 1 (ID: 38b08b9cd49edf9d)

✅ Evaluation Results
id: 38b08b9cd49edf9d
question: Now you jump to an object's position, facing its direction. You observe that desklamp is front, mid distance, facing forward; blue door is front-right, slightly far, on front wall. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: teddybear
final_pos: [3, 3]
final_ori: [0, 1]
final_observation: [{'name': 'desklamp', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'blue door', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
object_positions:
laptop: [4.0, 1.0]
bike: [14.0, 7.0]
airplane: [6.0, 13.0]
pan: [6.0, 3.0]
truck: [12.0, 3.0]
lego man: [2.0, 11.0]
teddybear: [3.0, 3.0]
desklamp: [3.0, 6.0]
chair: [12.0, 7.0]
candle: [2.0, 13.0]
table: [14.0, 5.0]
fire extinguisher: [6.0, 9.0]
green door: [9.0, 5.0]
blue door: [5.0, 7.0]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
room_id: 1
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: BackwardPovTextEvaluationTask

Question 2 (ID: 593eaa3f0e5ee139)

✅ Evaluation Results
id: 593eaa3f0e5ee139
question: Now you jump to an object's position, facing its direction. You observe that bike is front-right, slightly far, facing backward; chair is front, mid distance, facing left; table is front-right, mid distance. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: truck
final_pos: [12, 3]
final_ori: [0, 1]
final_observation: [{'name': 'bike', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing backward'}, {'name': 'chair', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing left'}, {'name': 'table', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': None}]
object_positions:
laptop: [4.0, 1.0]
bike: [14.0, 7.0]
airplane: [6.0, 13.0]
pan: [6.0, 3.0]
truck: [12.0, 3.0]
lego man: [2.0, 11.0]
teddybear: [3.0, 3.0]
desklamp: [3.0, 6.0]
chair: [12.0, 7.0]
candle: [2.0, 13.0]
table: [14.0, 5.0]
fire extinguisher: [6.0, 9.0]
green door: [9.0, 5.0]
blue door: [5.0, 7.0]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
room_id: 2
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: BackwardPovTextEvaluationTask

Question 3 (ID: 8478901165ea117a)

✅ Evaluation Results
id: 8478901165ea117a
question: Now you jump to an object's position, facing its direction. You observe that airplane is front, mid distance, facing right; candle is front-left, slightly far. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: fire extinguisher
final_pos: [6, 9]
final_ori: [0, 1]
final_observation: [{'name': 'airplane', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'candle', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': None}]
object_positions:
laptop: [4.0, 1.0]
bike: [14.0, 7.0]
airplane: [6.0, 13.0]
pan: [6.0, 3.0]
truck: [12.0, 3.0]
lego man: [2.0, 11.0]
teddybear: [3.0, 3.0]
desklamp: [3.0, 6.0]
chair: [12.0, 7.0]
candle: [2.0, 13.0]
table: [14.0, 5.0]
fire extinguisher: [6.0, 9.0]
green door: [9.0, 5.0]
blue door: [5.0, 7.0]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
room_id: 3
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: BackwardPovTextEvaluationTask

📊 Task: Action2ViewEvaluationTask

Question 1 (ID: 21808bda9b228db0)

✅ Evaluation Results
id: 21808bda9b228db0
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(-90) 2. Jump to the object at front-right, mid distance. 3. Rotate(-180) 4. Jump to the object at front-left, mid distance. 5. Rotate(-180) After executing the actions, what is the ego relation of teddybear relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-slight-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: Action2ViewEvaluationTask

Question 2 (ID: 604bedeabbf32688)

✅ Evaluation Results
id: 604bedeabbf32688
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(-90) 2. Jump to the object at front-right, slightly far. 3. Rotate(-90) 4. Jump to the object at front, mid distance. 5. Rotate(-90) After executing the actions, what is the ego relation of green door relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-slight-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: Action2ViewEvaluationTask

Question 3 (ID: ab43fa08726d1772)

✅ Evaluation Results
id: ab43fa08726d1772
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(-90) 2. Jump to the object at front-slight-right, slightly far. 3. Rotate(90) 4. Jump to the object at front-right, slightly far. After executing the actions, what is the ego relation of fire extinguisher relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-right, mid distance
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: Action2ViewEvaluationTask

📊 Task: View2ActionTextEvaluationTask

Question 1 (ID: 1a5402f686fdbe8e)

✅ Evaluation Results
id: 1a5402f686fdbe8e
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: candle is at front, near What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [2, 11]
final_ori: [0, 1]
room_id: 3
init_pos: [8, 2]
init_ori: [0, 1]
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
minimal_plan: [['jumpto', 'blue door'], ['jumpto', 'lego man']]
final_observation: [{'name': 'candle', 'direction': 'front', 'distance': 'near', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [2.0, 11.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: View2ActionTextEvaluationTask

Question 2 (ID: 0240108a1a776391)

✅ Evaluation Results
id: 0240108a1a776391
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: bike is at front, near, facing backward; chair is at front-left, mid distance, facing left What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [14, 5]
final_ori: [0, 1]
room_id: 2
init_pos: [8, 2]
init_ori: [0, 1]
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
minimal_plan: [['jumpto', 'green door'], ['rotate', 90], ['jumpto', 'table'], ['rotate', -90]]
final_observation: [{'name': 'bike', 'direction': 'front', 'distance': 'near', 'orientation': 'facing backward'}, {'name': 'chair', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing left'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [14.0, 5.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: View2ActionTextEvaluationTask

Question 3 (ID: fe579174d12a0b9e)

✅ Evaluation Results
id: fe579174d12a0b9e
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: green door is at front-left, mid distance, on front wall What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [6, 3]
final_ori: [1, 0]
room_id: 1
init_pos: [8, 2]
init_ori: [0, 1]
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
minimal_plan: [['rotate', -90], ['jumpto', 'pan'], ['rotate', 180]]
final_observation: [{'name': 'green door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on front wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: False
final_pos: [12.0, 7.0]
final_ori: [-1, 0]
visible_match: False
score: 0
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: View2ActionTextEvaluationTask

📊 Task: AlloMappingEvaluationTask

Question 1 (ID: 046899a176fbe951)

✅ Evaluation Results
id: 046899a176fbe951
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: airplane, laptop, candle, table, fire extinguisher. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-2, 11], [-4, -1], [-6, 11], [6, 3], [-2, 7]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
threshold: 0.900
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: AlloMappingEvaluationTask

Question 2 (ID: 64c2cfa4dfb804b2)

✅ Evaluation Results
id: 64c2cfa4dfb804b2
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: chair, pan, lego man. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[4, 5], [-2, 1], [-6, 9]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
threshold: 0.900
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: AlloMappingEvaluationTask

Question 3 (ID: bf689311469baa8a)

✅ Evaluation Results
id: bf689311469baa8a
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: lego man, pan, bike, fire extinguisher, airplane. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-6, 9], [-2, 1], [6, 5], [-2, 7], [-2, 11]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
threshold: 0.900
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: AlloMappingEvaluationTask

📊 Task: RotEvaluationTask

Question 1 (ID: 79135364fb4b164f)

✅ Evaluation Results
id: 79135364fb4b164f
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning counterclockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: airplane, desklamp, laptop. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['airplane', 'desklamp', 'laptop']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: RotEvaluationTask

Question 2 (ID: 75467a0fbc94e6b7)

✅ Evaluation Results
id: 75467a0fbc94e6b7
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: chair, laptop, truck. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['chair', 'truck', 'laptop']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: RotEvaluationTask

Question 3 (ID: 456b022cad9cba9a)

✅ Evaluation Results
id: 456b022cad9cba9a
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning counterclockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: bike, lego man, teddybear. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['lego man', 'teddybear', 'bike']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: RotEvaluationTask

📊 Task: Location2ViewEvaluationTask

Question 1 (ID: 77fb7413d8ff5f85)

✅ Evaluation Results
id: 77fb7413d8ff5f85
question: Still treat your initial position as origin (0, 0) You move to (-3, 2) and face west. What is the egocentric relation of teddybear? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-left, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: Location2ViewEvaluationTask

Question 2 (ID: 3399c355c7d0785f)

✅ Evaluation Results
id: 3399c355c7d0785f
question: Still treat your initial position as origin (0, 0) You move to (-1, 0) and face west. What is the egocentric relation of pan? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-right, near
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: Location2ViewEvaluationTask

Question 3 (ID: 1c9e744fd28c4e95)

✅ Evaluation Results
id: 1c9e744fd28c4e95
question: Treat the blue door as the new 'origin' (0, 0). You move to (0, 3) and face north. What is the egocentric relation of airplane? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-slight-right, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: Location2ViewEvaluationTask

📊 Task: View2LocationTextEvaluationTask

Question 1 (ID: 981cabb8e8f95e0e)

✅ Evaluation Results
id: 981cabb8e8f95e0e
question: You move to a new location and face west. You observe: teddybear is at front-left, mid distance, facing right; desklamp is at front-right, mid distance, facing right Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [-3, 2]
final_pos: [5, 4]
final_ori: [-1, 0]
room_id: 1
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'teddybear', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'desklamp', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing right'}]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.758
raw_score: 0.758
score: 0.758
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: View2LocationTextEvaluationTask

Question 2 (ID: 56bb180ac8571782)

✅ Evaluation Results
id: 56bb180ac8571782
question: You move to a new location and face west. You observe: pan is at front-right, near, facing left; teddybear is at front-slight-right, slightly far, facing right; laptop is at front-slight-left, mid distance, facing left Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [-1, 0]
final_pos: [7, 2]
final_ori: [-1, 0]
room_id: 1
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'pan', 'direction': 'front-right', 'distance': 'near', 'orientation': 'facing left'}, {'name': 'teddybear', 'direction': 'front-slight-right', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'laptop', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing left'}]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: View2LocationTextEvaluationTask

Question 3 (ID: d06f40c11fae70c2)

✅ Evaluation Results
id: d06f40c11fae70c2
question: You move to a new location and face north. You observe: airplane is at front-slight-right, mid distance, facing right; candle is at front-left, slightly far Treat the blue door as the new 'origin' (0, 0). What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [0, 3]
final_pos: [5, 10]
final_ori: [0, 1]
room_id: 3
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'airplane', 'direction': 'front-slight-right', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'candle', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': None}]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: View2LocationTextEvaluationTask

📊 Evaluation (use_model_cogmap)

📊 Task: DirectionEvaluationTask

Question 1 (ID: ca87148432a7d552)

✅ Evaluation Results
id: ca87148432a7d552
question: You return to your starting position and face north. From a Top-Down map, describe where airplane is relative to laptop. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q1: DirectionEvaluationTask

Question 2 (ID: e031d760b96a5da9)

✅ Evaluation Results
id: e031d760b96a5da9
question: You return to your starting position and face north. From a Top-Down map, describe where chair is relative to pan. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north east, slightly far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q2: DirectionEvaluationTask

Question 3 (ID: f91d7ef70302b179)

✅ Evaluation Results
id: f91d7ef70302b179
question: You return to your starting position and face north. From a Top-Down map, describe where lego man is relative to pan. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north west, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q3: DirectionEvaluationTask

📊 Task: PovEvaluationTask

Question 1 (ID: b39725205f4d5931)

✅ Evaluation Results
id: b39725205f4d5931
question: Now you jump to teddybear's direction, facing its direction. Describe where desklamp is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Evaluation state
Q1: PovEvaluationTask

Question 2 (ID: ddaba1a0c4c8aaed)

✅ Evaluation Results
id: ddaba1a0c4c8aaed
question: Now you jump to pan's direction, facing its direction. Describe where laptop is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-right, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: use_model_cogmap
Evaluation state
Q2: PovEvaluationTask

Question 3 (ID: 583f1b341e5291a0)

✅ Evaluation Results
id: 583f1b341e5291a0
question: Now you jump to bike's direction, facing its direction. Describe where table is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front, near
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q3: PovEvaluationTask

📊 Task: BackwardPovTextEvaluationTask

Question 1 (ID: 38b08b9cd49edf9d)

✅ Evaluation Results
id: 38b08b9cd49edf9d
question: Now you jump to an object's position, facing its direction. You observe that desklamp is front, mid distance, facing forward; blue door is front-right, slightly far, on front wall. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: teddybear
final_pos: [3, 3]
final_ori: [0, 1]
final_observation: [{'name': 'desklamp', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'blue door', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
object_positions:
laptop: [4.0, 1.0]
bike: [14.0, 7.0]
airplane: [6.0, 13.0]
pan: [6.0, 3.0]
truck: [12.0, 3.0]
lego man: [2.0, 11.0]
teddybear: [3.0, 3.0]
desklamp: [3.0, 6.0]
chair: [12.0, 7.0]
candle: [2.0, 13.0]
table: [14.0, 5.0]
fire extinguisher: [6.0, 9.0]
green door: [9.0, 5.0]
blue door: [5.0, 7.0]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
room_id: 1
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q1: BackwardPovTextEvaluationTask

Question 2 (ID: 593eaa3f0e5ee139)

✅ Evaluation Results
id: 593eaa3f0e5ee139
question: Now you jump to an object's position, facing its direction. You observe that bike is front-right, slightly far, facing backward; chair is front, mid distance, facing left; table is front-right, mid distance. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: truck
final_pos: [12, 3]
final_ori: [0, 1]
final_observation: [{'name': 'bike', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing backward'}, {'name': 'chair', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing left'}, {'name': 'table', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': None}]
object_positions:
laptop: [4.0, 1.0]
bike: [14.0, 7.0]
airplane: [6.0, 13.0]
pan: [6.0, 3.0]
truck: [12.0, 3.0]
lego man: [2.0, 11.0]
teddybear: [3.0, 3.0]
desklamp: [3.0, 6.0]
chair: [12.0, 7.0]
candle: [2.0, 13.0]
table: [14.0, 5.0]
fire extinguisher: [6.0, 9.0]
green door: [9.0, 5.0]
blue door: [5.0, 7.0]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
room_id: 2
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q2: BackwardPovTextEvaluationTask

Question 3 (ID: 8478901165ea117a)

✅ Evaluation Results
id: 8478901165ea117a
question: Now you jump to an object's position, facing its direction. You observe that airplane is front, mid distance, facing right; candle is front-left, slightly far. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: fire extinguisher
final_pos: [6, 9]
final_ori: [0, 1]
final_observation: [{'name': 'airplane', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'candle', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': None}]
object_positions:
laptop: [4.0, 1.0]
bike: [14.0, 7.0]
airplane: [6.0, 13.0]
pan: [6.0, 3.0]
truck: [12.0, 3.0]
lego man: [2.0, 11.0]
teddybear: [3.0, 3.0]
desklamp: [3.0, 6.0]
chair: [12.0, 7.0]
candle: [2.0, 13.0]
table: [14.0, 5.0]
fire extinguisher: [6.0, 9.0]
green door: [9.0, 5.0]
blue door: [5.0, 7.0]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
room_id: 3
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q3: BackwardPovTextEvaluationTask

📊 Task: Action2ViewEvaluationTask

Question 1 (ID: 21808bda9b228db0)

✅ Evaluation Results
id: 21808bda9b228db0
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(-90) 2. Jump to the object at front-right, mid distance. 3. Rotate(-180) 4. Jump to the object at front-left, mid distance. 5. Rotate(-180) After executing the actions, what is the ego relation of teddybear relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-slight-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: use_model_cogmap
Evaluation state
Q1: Action2ViewEvaluationTask

Question 2 (ID: 604bedeabbf32688)

✅ Evaluation Results
id: 604bedeabbf32688
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(-90) 2. Jump to the object at front-right, slightly far. 3. Rotate(-90) 4. Jump to the object at front, mid distance. 5. Rotate(-90) After executing the actions, what is the ego relation of green door relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-slight-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: use_model_cogmap
Evaluation state
Q2: Action2ViewEvaluationTask

Question 3 (ID: ab43fa08726d1772)

✅ Evaluation Results
id: ab43fa08726d1772
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(-90) 2. Jump to the object at front-slight-right, slightly far. 3. Rotate(90) 4. Jump to the object at front-right, slightly far. After executing the actions, what is the ego relation of fire extinguisher relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-right, mid distance
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Evaluation state
Q3: Action2ViewEvaluationTask

📊 Task: View2ActionTextEvaluationTask

Question 1 (ID: 1a5402f686fdbe8e)

✅ Evaluation Results
id: 1a5402f686fdbe8e
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: candle is at front, near What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [2, 11]
final_ori: [0, 1]
room_id: 3
init_pos: [8, 2]
init_ori: [0, 1]
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
minimal_plan: [['jumpto', 'blue door'], ['jumpto', 'lego man']]
final_observation: [{'name': 'candle', 'direction': 'front', 'distance': 'near', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [2.0, 11.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q1: View2ActionTextEvaluationTask

Question 2 (ID: 0240108a1a776391)

✅ Evaluation Results
id: 0240108a1a776391
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: bike is at front, near, facing backward; chair is at front-left, mid distance, facing left What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [14, 5]
final_ori: [0, 1]
room_id: 2
init_pos: [8, 2]
init_ori: [0, 1]
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
minimal_plan: [['jumpto', 'green door'], ['rotate', 90], ['jumpto', 'table'], ['rotate', -90]]
final_observation: [{'name': 'bike', 'direction': 'front', 'distance': 'near', 'orientation': 'facing backward'}, {'name': 'chair', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing left'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [14.0, 5.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q2: View2ActionTextEvaluationTask

Question 3 (ID: fe579174d12a0b9e)

✅ Evaluation Results
id: fe579174d12a0b9e
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: green door is at front-left, mid distance, on front wall What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [6, 3]
final_ori: [1, 0]
room_id: 1
init_pos: [8, 2]
init_ori: [0, 1]
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
minimal_plan: [['rotate', -90], ['jumpto', 'pan'], ['rotate', 180]]
final_observation: [{'name': 'green door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on front wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [6.0, 3.0]
final_ori: [1, 0]
visible_match: True
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q3: View2ActionTextEvaluationTask

📊 Task: AlloMappingEvaluationTask

Question 1 (ID: 046899a176fbe951)

✅ Evaluation Results
id: 046899a176fbe951
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: airplane, laptop, candle, table, fire extinguisher. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-2, 11], [-4, -1], [-6, 11], [6, 3], [-2, 7]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.754
threshold: 0.900
score: 0.754
evaluation_mode: use_model_cogmap
Evaluation state
Q1: AlloMappingEvaluationTask

Question 2 (ID: 64c2cfa4dfb804b2)

✅ Evaluation Results
id: 64c2cfa4dfb804b2
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: chair, pan, lego man. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[4, 5], [-2, 1], [-6, 9]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.844
threshold: 0.900
score: 0.844
evaluation_mode: use_model_cogmap
Evaluation state
Q2: AlloMappingEvaluationTask

Question 3 (ID: bf689311469baa8a)

✅ Evaluation Results
id: bf689311469baa8a
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: lego man, pan, bike, fire extinguisher, airplane. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-6, 9], [-2, 1], [6, 5], [-2, 7], [-2, 11]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.710
threshold: 0.900
score: 0.710
evaluation_mode: use_model_cogmap
Evaluation state
Q3: AlloMappingEvaluationTask

📊 Task: RotEvaluationTask

Question 1 (ID: 79135364fb4b164f)

✅ Evaluation Results
id: 79135364fb4b164f
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning counterclockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: airplane, desklamp, laptop. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['airplane', 'desklamp', 'laptop']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q1: RotEvaluationTask

Question 2 (ID: 75467a0fbc94e6b7)

✅ Evaluation Results
id: 75467a0fbc94e6b7
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: chair, laptop, truck. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['chair', 'truck', 'laptop']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: use_model_cogmap
Evaluation state
Q2: RotEvaluationTask

Question 3 (ID: 456b022cad9cba9a)

✅ Evaluation Results
id: 456b022cad9cba9a
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning counterclockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: bike, lego man, teddybear. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['lego man', 'teddybear', 'bike']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q3: RotEvaluationTask

📊 Task: Location2ViewEvaluationTask

Question 1 (ID: 77fb7413d8ff5f85)

✅ Evaluation Results
id: 77fb7413d8ff5f85
question: Still treat your initial position as origin (0, 0) You move to (-3, 2) and face west. What is the egocentric relation of teddybear? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-left, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: use_model_cogmap
Evaluation state
Q1: Location2ViewEvaluationTask

Question 2 (ID: 3399c355c7d0785f)

✅ Evaluation Results
id: 3399c355c7d0785f
question: Still treat your initial position as origin (0, 0) You move to (-1, 0) and face west. What is the egocentric relation of pan? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-right, near
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Evaluation state
Q2: Location2ViewEvaluationTask

Question 3 (ID: 1c9e744fd28c4e95)

✅ Evaluation Results
id: 1c9e744fd28c4e95
question: Treat the blue door as the new 'origin' (0, 0). You move to (0, 3) and face north. What is the egocentric relation of airplane? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-slight-right, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Evaluation state
Q3: Location2ViewEvaluationTask

📊 Task: View2LocationTextEvaluationTask

Question 1 (ID: 981cabb8e8f95e0e)

✅ Evaluation Results
id: 981cabb8e8f95e0e
question: You move to a new location and face west. You observe: teddybear is at front-left, mid distance, facing right; desklamp is at front-right, mid distance, facing right Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [-3, 2]
final_pos: [5, 4]
final_ori: [-1, 0]
room_id: 1
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'teddybear', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'desklamp', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing right'}]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.319
raw_score: 0.319
score: 0.319
evaluation_mode: use_model_cogmap
Evaluation state
Q1: View2LocationTextEvaluationTask

Question 2 (ID: 56bb180ac8571782)

✅ Evaluation Results
id: 56bb180ac8571782
question: You move to a new location and face west. You observe: pan is at front-right, near, facing left; teddybear is at front-slight-right, slightly far, facing right; laptop is at front-slight-left, mid distance, facing left Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [-1, 0]
final_pos: [7, 2]
final_ori: [-1, 0]
room_id: 1
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'pan', 'direction': 'front-right', 'distance': 'near', 'orientation': 'facing left'}, {'name': 'teddybear', 'direction': 'front-slight-right', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'laptop', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing left'}]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.107
raw_score: 0.107
score: 0.107
evaluation_mode: use_model_cogmap
Evaluation state
Q2: View2LocationTextEvaluationTask

Question 3 (ID: d06f40c11fae70c2)

✅ Evaluation Results
id: d06f40c11fae70c2
question: You move to a new location and face north. You observe: airplane is at front-slight-right, mid distance, facing right; candle is at front-left, slightly far Treat the blue door as the new 'origin' (0, 0). What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [0, 3]
final_pos: [5, 10]
final_ori: [0, 1]
room_id: 3
object_positions:
laptop: [4, 1]
bike: [14, 7]
airplane: [6, 13]
pan: [6, 3]
truck: [12, 3]
lego man: [2, 11]
teddybear: [3, 3]
desklamp: [3, 6]
chair: [12, 7]
candle: [2, 13]
table: [14, 5]
fire extinguisher: [6, 9]
green door: [9, 5]
blue door: [5, 7]
object_orientations:
laptop: [0, -1]
bike: [0, -1]
airplane: [1, 0]
pan: [0, -1]
truck: [0, 1]
lego man: [0, 1]
teddybear: [0, 1]
desklamp: [0, 1]
chair: [-1, 0]
fire extinguisher: [0, 1]
green door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'airplane', 'direction': 'front-slight-right', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'candle', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': None}]
gate_info:
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
blue door:
room_ids: [1, 3]
ori_by_room:
1: [0, -1]
3: [0, 1]
object_rooms:
laptop: 1
bike: 2
airplane: 3
pan: 1
truck: 2
lego man: 3
teddybear: 1
desklamp: 1
chair: 2
candle: 3
table: 2
fire extinguisher: 3
green door: [1, 2]
blue door: [1, 3]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.717
raw_score: 0.717
score: 0.717
evaluation_mode: use_model_cogmap
Evaluation state
Q3: View2LocationTextEvaluationTask