sample_run06
Select Configuration:
📊 Sample Metrics
🔍 Exploration
last_node_coverage: 1
last_edge_coverage: 0.463
n_exploration_steps: 13
action_counts:
move: 3
rotate: 8
return: 0
observe: 12
term: 1
forced_term: 0
query: 0
action_cost: 12
final_information_gain: 0.851
action_fail_ratio: 0.077
valid_action_ratio: 1
false_belief_steps: 8
false_belief_f1: 1
false_belief_f1_position: 1
false_belief_f1_facing: 1
false_belief_action_cost: 7
false_belief_action_cost_after_seen: 3
✅ Evaluation
overall:
n_total: 27
total_score: 20.109
avg_accuracy: 0.745
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
PovEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
Action2ViewEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
View2ActionTextEvaluationTask:
n_total: 3
task_score: 0
avg_accuracy: 0
AlloMappingEvaluationTask:
n_total: 3
task_score: 2.588
avg_accuracy: 0.863
RotEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Location2ViewEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
View2LocationTextEvaluationTask:
n_total: 3
task_score: 2.521
avg_accuracy: 0.840
✅ Evaluation (prompt_cogmap)
overall:
n_total: 27
total_score: 17.371
avg_accuracy: 0.643
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 1
avg_accuracy: 0.333
PovEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
Action2ViewEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
View2ActionTextEvaluationTask:
n_total: 3
task_score: 0
avg_accuracy: 0
AlloMappingEvaluationTask:
n_total: 3
task_score: 2.371
avg_accuracy: 0.790
RotEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Location2ViewEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
View2LocationTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
✅ Evaluation (use_gt_cogmap)
overall:
n_total: 27
total_score: 26
avg_accuracy: 0.963
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
PovEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Action2ViewEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
View2ActionTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
AlloMappingEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
RotEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Location2ViewEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
View2LocationTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
✅ Evaluation (use_model_cogmap)
overall:
n_total: 27
total_score: 21.517
avg_accuracy: 0.797
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
PovEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Action2ViewEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
View2ActionTextEvaluationTask:
n_total: 3
task_score: 1
avg_accuracy: 0.333
AlloMappingEvaluationTask:
n_total: 3
task_score: 2.698
avg_accuracy: 0.899
RotEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Location2ViewEvaluationTask:
n_total: 3
task_score: 1.500
avg_accuracy: 0.500
View2LocationTextEvaluationTask:
n_total: 3
task_score: 2.819
avg_accuracy: 0.940
🧠 Cognitive Map
exploration:
error:
local_vs_gt_local_avg:
dir: 0.756
facing: 1
pos: 0.808
overall: 0.854
global_vs_gt_global_avg:
dir: 0.824
facing: 1
pos: 0.798
overall: 0.874
agent_vs_gt_agent_avg:
dir: 1
facing: 1
pos: 0.948
overall: 0.983
newly_observed_vs_gt_local_avg:
dir: 0.917
facing: 1
pos: 0.850
overall: 0.922
correctness:
last_global_vs_gt_full:
dir: 0.825
facing: 1
pos: 0.858
overall: 0.894
consistency:
local_vs_global_avg:
dir: 0.693
facing: 1
pos: 0.844
overall: 0.846
position_update_avg: 0.632
facing_update_avg: 1
position_stability_avg: 0.632
facing_stability_avg: 1
fog_probe:
f1_avg: 0.672
precision_avg: 0.548
recall_avg: 0.903
cogmap_fb:
metrics:
changed:
dir: None
pos: 0.568
facing: 0.500
overall: None
retention:
dir: None
pos: 0.266
facing: 0
overall: None
unchanged:
dir: 0.721
facing: 0.743
pos: 0.700
overall: 0.721
inertia: 0.000
inertia_list: [-4.372867232561502e-11, 0.00010889100378069601]
unchanged_retention:
dir: None
pos: 0.852
facing: 0.778
overall: None
unchanged_retention_minus_retention:
pos: 0.902
facing: 1
unchanged_exploration:
dir: 0.745
facing: 1
pos: 0.833
overall: 0.860
🧭 False Belief CogMap
inertia: 0.000
changed:
dir: None
pos: 0.568
facing: 0.500
overall: None
retention:
dir: None
pos: 0.266
facing: 0
overall: None
unchanged:
dir: 0.721
facing: 0.743
pos: 0.700
overall: 0.721
unchanged_retention:
dir: None
pos: 0.852
facing: 0.778
overall: None
unchanged_retention_minus_retention:
pos: 0.902
facing: 1
unchanged_exploration:
dir: 0.745
facing: 1
pos: 0.833
overall: 0.860
📊 Performance Charts
Cognitive Map (Update)
Cognitive Map (Full)
Cognitive Map (Self-Tracking)
FB CogMap (Unchanged)
Fog Probe F1
Fog Probe Precision
Fog Probe Recall
Position Update
Facing Update
Position Stability
Facing Stability
🌍 Exploration Phase
🔄 Turn 1
💬 Assistant Action
Actions: [Observe()]
Actions: [Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
0,
0
],
"facing": "north"
},
"lego man": {
"position": [
-4,
5
],
"facing": "south"
}
}
🎯 Ground Truth (Observed)
{
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"agent": {
"position": [
0,
0
],
"facing": "north"
}
}
🎯 Ground Truth (Full)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"airplane": {
"position": [
3,
-3
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
0,
0
],
"facing": "north"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"lego man": {
"position": [
-3,
5
],
"facing": "-y"
}
}
}
🎯 Ground Truth
{
"lego man": {
"position": [
-3,
3
],
"facing": "south"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"lego man": {
"position": [
-3,
5
],
"facing": "-y"
}
}
}
🎯 Ground Truth
{
"lego man": {
"position": [
-3,
3
],
"facing": "south"
}
}
🗺️ Symbolic Fog Map
###################### #......############### #......############### #......#...B.E######## #......#A....D######## #......#......#......# #......+....*.+......# ########.....C#......# ########F.....#......# ###############......# ###############......# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: A, C, F
🎯 Ground Truth: A, F, C
🧠 Cognitive Map Metrics
Global:
dir: 1
facing: 1
pos: 0.790
overall: 0.930
Global (Full):
dir: 0.025
facing: 0.222
pos: 0.105
overall: 0.118
Local:
dir: 1
facing: 1
pos: 0.743
overall: 0.914
Local (Newly):
dir: 1
facing: 1
pos: 0.743
overall: 0.914
Fog Probe:
precision: 1
recall: 1
overall: 1
📈 Turn Metrics
node_coverage: 0.071
edge_coverage: 0.024
is_action_fail: False
step: 1
action_counts:
move: 0
rotate: 0
return: 0
observe: 1
term: 0
forced_term: 0
query: 0
information_gain: 0.044
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 2
💬 Assistant Action
Actions: [Rotate(90), Observe()]
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
0,
0
],
"facing": "east"
},
"lego man": {
"position": [
-4,
5
],
"facing": "south"
},
"green door": {
"position": [
1,
0
]
}
}
🎯 Ground Truth (Observed)
{
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
0,
0
],
"facing": "east"
}
}
🎯 Ground Truth (Full)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"airplane": {
"position": [
3,
-3
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
0,
0
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"green door": {
"position": [
0,
1
]
}
}
}
🎯 Ground Truth
{
"green door": {
"position": [
0,
2
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"green door": {
"position": [
0,
1
]
}
}
}
🎯 Ground Truth
{
"green door": {
"position": [
0,
2
],
"facing": "east"
}
}
🗺️ Symbolic Fog Map
###################### #......############### #......############### #......#......######## #......#.A..F.######## #......#...E..#......# #......+D...*C+......# ########....B.#......# ########......#......# ###############......# ###############......# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: A, B, D, E, F
🎯 Ground Truth: D, B, A
🧠 Cognitive Map Metrics
Global:
dir: 1
facing: 1
pos: 0.810
overall: 0.937
Global (Full):
dir: 0.050
facing: 0.222
pos: 0.162
overall: 0.145
Local:
dir: 1
facing: 1
pos: 0.862
overall: 0.954
Local (Newly):
dir: 1
facing: 1
pos: 0.862
overall: 0.954
Fog Probe:
precision: 0.600
recall: 1
overall: 0.750
📈 Turn Metrics
node_coverage: 0.143
edge_coverage: 0.049
is_action_fail: False
step: 2
action_counts:
move: 0
rotate: 1
return: 0
observe: 2
term: 0
forced_term: 0
query: 0
information_gain: 0.107
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 3
💬 Assistant Action
Actions: [JumpTo(green door), Observe()]
Actions: [JumpTo(green door), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
2,
0
],
"facing": "east"
},
"lego man": {
"position": [
-3,
4
],
"facing": "south"
},
"green door": {
"position": [
2,
0
]
},
"toilet": {
"position": [
6,
-3
],
"facing": "east"
},
"fridge": {
"position": [
6,
-4
]
},
"basket": {
"position": [
5,
1
]
}
}
🎯 Ground Truth (Observed)
{
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
2,
0
],
"facing": "east"
}
}
🎯 Ground Truth (Full)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"airplane": {
"position": [
3,
-3
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
2,
0
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"toilet": {
"position": [
1,
5
],
"facing": "+y"
},
"fridge": {
"position": [
3,
5
]
},
"basket": {
"position": [
-1,
3
]
}
}
}
🎯 Ground Truth
{
"toilet": {
"position": [
1,
5
],
"facing": "east"
},
"fridge": {
"position": [
3,
3
],
"facing": "east"
},
"basket": {
"position": [
-1,
3
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"toilet": {
"position": [
1,
5
],
"facing": "+y"
},
"fridge": {
"position": [
3,
5
]
},
"basket": {
"position": [
-1,
3
]
}
}
}
🎯 Ground Truth
{
"toilet": {
"position": [
1,
5
],
"facing": "east"
},
"fridge": {
"position": [
3,
3
],
"facing": "east"
},
"basket": {
"position": [
-1,
3
],
"facing": "east"
}
}
🗺️ Symbolic Fog Map
###################### #......############### #......############### #......#....A.######## #......#...J.L######## #......#......#..K...# #......+..E...*......# ########......#..G...# ########I..H..#B.....# ###############.C....# ###############.F..D.# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: A, B, C, D, F, G, J, K, L
🎯 Ground Truth: H, I, E, B, C, F
🧠 Cognitive Map Metrics
Global:
dir: 0.714
facing: 1
pos: 0.842
overall: 0.852
Global (Full):
dir: 0.125
facing: 0.333
pos: 0.337
overall: 0.265
Local:
dir: 0.667
facing: 1
pos: 0.842
overall: 0.836
Local (Newly):
dir: 0.667
facing: 1
pos: 0.842
overall: 0.836
Fog Probe:
precision: 0.333
recall: 0.500
overall: 0.400
📈 Turn Metrics
node_coverage: 0.357
edge_coverage: 0.122
is_action_fail: False
step: 3
action_counts:
move: 1
rotate: 1
return: 0
observe: 3
term: 0
forced_term: 0
query: 0
information_gain: 0.257
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 4
💬 Assistant Action
Actions: [Rotate(-90), Observe()]
Actions: [Rotate(-90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
1,
0
],
"facing": "north"
},
"lego man": {
"position": [
-3,
5
],
"facing": "south"
},
"green door": {
"position": [
1,
0
]
},
"toilet": {
"position": [
7,
-2
],
"facing": "east"
},
"fridge": {
"position": [
7,
-4
]
},
"basket": {
"position": [
4,
1
]
}
}
🎯 Ground Truth (Observed)
{
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
2,
0
],
"facing": "north"
}
}
🎯 Ground Truth (Full)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"airplane": {
"position": [
3,
-3
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
2,
0
],
"facing": "north"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {}
}
🎯 Ground Truth
(no data)
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {}
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
###################### #......############### #......############### #......#.JF...######## #......#..H...######## #......#......#....D.# #......+......*...E..# ########.I....#......# ########....GB#.L....# ###############CA....# ###############..K...# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: A, B, C, D, E, F, G, H, I, J, K, L
🎯 Ground Truth: B, I, G, A, C, K
🧠 Cognitive Map Metrics
Global:
dir: 0.810
facing: 1
pos: 0.803
overall: 0.871
Global (Full):
dir: 0.142
facing: 0.333
pos: 0.321
overall: 0.265
Local:
(none)
Local (Newly):
(none)
Fog Probe:
precision: 0.500
recall: 1
overall: 0.667
📈 Turn Metrics
node_coverage: 0.357
edge_coverage: 0.122
is_action_fail: False
step: 4
action_counts:
move: 1
rotate: 2
return: 0
observe: 4
term: 0
forced_term: 0
query: 0
information_gain: 0.257
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 5
💬 Assistant Action
Actions: [Rotate(-90), Observe()]
Actions: [Rotate(-90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
2,
0
],
"facing": "west"
},
"green door": {
"position": [
2,
0
]
},
"red door": {
"position": [
-4,
0
]
},
"lego man": {
"position": [
-3,
4
],
"facing": "south"
},
"bike": {
"position": [
-3,
2
],
"facing": "south"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"basket": {
"position": [
5,
-1
]
},
"toilet": {
"position": [
7,
2
],
"facing": "east"
},
"fridge": {
"position": [
7,
4
]
}
}
🎯 Ground Truth (Observed)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
2,
0
],
"facing": "west"
}
}
🎯 Ground Truth (Full)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"airplane": {
"position": [
3,
-3
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
2,
0
],
"facing": "west"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"bike": {
"position": [
1,
5
],
"facing": "-x"
},
"lego man": {
"position": [
3,
5
],
"facing": "-x"
},
"backpack": {
"position": [
-2,
3
],
"facing": "-x"
},
"red door": {
"position": [
0,
5
]
}
}
}
🎯 Ground Truth
{
"lego man": {
"position": [
3,
5
],
"facing": "south"
},
"bike": {
"position": [
1,
5
],
"facing": "south"
},
"red door": {
"position": [
0,
7
],
"facing": "east"
},
"backpack": {
"position": [
-2,
3
],
"facing": "south"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"bike": {
"position": [
1,
5
],
"facing": "-x"
},
"lego man": {
"position": [
3,
5
],
"facing": "-x"
},
"backpack": {
"position": [
-2,
3
],
"facing": "-x"
},
"red door": {
"position": [
0,
5
]
}
}
}
🎯 Ground Truth
{
"bike": {
"position": [
1,
5
],
"facing": "south"
},
"red door": {
"position": [
0,
7
],
"facing": "east"
},
"backpack": {
"position": [
-2,
3
],
"facing": "south"
}
}
🗺️ Symbolic Fog Map
###################### #......############### #......############### #......#......######## #......#......######## #......#....C.#......# #......+......*G.....# ########......#.A....# ########.....H#......# ###############B.....# ###############.FE.D.# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H
🤖 Predicted: A, B, C, D, E, F, H
🎯 Ground Truth: H, F, B, E
🧠 Cognitive Map Metrics
Global:
dir: 0.689
facing: 1
pos: 0.662
overall: 0.784
Global (Full):
dir: 0.258
facing: 0.556
pos: 0.397
overall: 0.404
Local:
dir: 0.800
facing: 1
pos: 0.862
overall: 0.887
Local (Newly):
dir: 0.833
facing: 1
pos: 0.842
overall: 0.892
Fog Probe:
precision: 0.571
recall: 1
overall: 0.727
📈 Turn Metrics
node_coverage: 0.571
edge_coverage: 0.220
is_action_fail: False
step: 5
action_counts:
move: 1
rotate: 3
return: 0
observe: 5
term: 0
forced_term: 0
query: 0
information_gain: 0.436
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 6
💬 Assistant Action
Actions: [JumpTo(red door), Observe()]
Actions: [JumpTo(red door), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
-4,
0
],
"facing": "west"
},
"green door": {
"position": [
1,
0
]
},
"red door": {
"position": [
-4,
0
]
},
"lego man": {
"position": [
-2,
2
],
"facing": "south"
},
"bike": {
"position": [
-2,
1
],
"facing": "south"
},
"backpack": {
"position": [
-2,
-2
],
"facing": "south"
},
"toilet": {
"position": [
6,
-1
],
"facing": "east"
},
"fridge": {
"position": [
5,
-3
]
},
"basket": {
"position": [
4,
2
]
},
"rubberduck": {
"position": [
-8,
2
],
"facing": "north"
},
"truck": {
"position": [
-6,
2
],
"facing": "east"
},
"lamp": {
"position": [
-9,
4
]
}
}
🎯 Ground Truth (Observed)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
-5,
0
],
"facing": "west"
}
}
🎯 Ground Truth (Full)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"airplane": {
"position": [
3,
-3
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
-5,
0
],
"facing": "west"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"rubberduck": {
"position": [
1,
5
],
"facing": "+x"
},
"truck": {
"position": [
2,
3
],
"facing": "-y"
},
"lamp": {
"position": [
3,
5
]
}
}
}
🎯 Ground Truth
{
"rubberduck": {
"position": [
1,
6
],
"facing": "north"
},
"truck": {
"position": [
1,
2
],
"facing": "east"
},
"lamp": {
"position": [
5,
6
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"rubberduck": {
"position": [
1,
5
],
"facing": "+x"
},
"truck": {
"position": [
2,
3
],
"facing": "-y"
},
"lamp": {
"position": [
3,
5
]
}
}
}
🎯 Ground Truth
{
"rubberduck": {
"position": [
1,
6
],
"facing": "north"
},
"truck": {
"position": [
1,
2
],
"facing": "east"
},
"lamp": {
"position": [
5,
6
],
"facing": "east"
}
}
🗺️ Symbolic Fog Map
###################### #....BM############### #......############### #....N.#......######## #....H.#...I..######## #.....E#......#.....L# #.....K*......+......# ########......#.A....# ########.....D#G.....# ###############....J.# ###############F.C...# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L, M, N
🤖 Predicted: A, C, F, G, I, J, L
🎯 Ground Truth: D, F, G, C, B, N, M
🧠 Cognitive Map Metrics
Global:
dir: 0.872
facing: 1
pos: 0.800
overall: 0.891
Global (Full):
dir: 0.567
facing: 0.778
pos: 0.640
overall: 0.661
Local:
dir: 0.833
facing: 1
pos: 0.784
overall: 0.873
Local (Newly):
dir: 0.833
facing: 1
pos: 0.784
overall: 0.873
Fog Probe:
precision: 0.429
recall: 0.429
overall: 0.429
📈 Turn Metrics
node_coverage: 0.786
edge_coverage: 0.293
is_action_fail: False
step: 6
action_counts:
move: 2
rotate: 3
return: 0
observe: 6
term: 0
forced_term: 0
query: 0
information_gain: 0.553
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 7
💬 Assistant Action
Actions: [Rotate(90), Observe()]
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
-3,
0
],
"facing": "north"
},
"green door": {
"position": [
2,
0
]
},
"red door": {
"position": [
-3,
0
]
},
"lego man": {
"position": [
-1,
3
],
"facing": "south"
},
"bike": {
"position": [
-1,
2
],
"facing": "south"
},
"backpack": {
"position": [
-2,
-1
],
"facing": "south"
},
"basket": {
"position": [
5,
1
]
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"fridge": {
"position": [
7,
-3
]
},
"rubberduck": {
"position": [
-8,
1
],
"facing": "north"
},
"truck": {
"position": [
-6,
2
],
"facing": "east"
},
"lamp": {
"position": [
-8,
3
]
},
"soda can": {
"position": [
-4,
3
]
}
}
🎯 Ground Truth (Observed)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
-5,
0
],
"facing": "north"
}
}
🎯 Ground Truth (Full)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"airplane": {
"position": [
3,
-3
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
-5,
0
],
"facing": "north"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"lego man": {
"position": [
2,
2
],
"facing": "-y"
},
"soda can": {
"position": [
-2,
2
]
}
}
}
🎯 Ground Truth
{
"lego man": {
"position": [
2,
3
],
"facing": "south"
},
"soda can": {
"position": [
-2,
3
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"lego man": {
"position": [
2,
2
],
"facing": "-y"
},
"soda can": {
"position": [
-2,
2
]
}
}
}
🎯 Ground Truth
{
"soda can": {
"position": [
-2,
3
],
"facing": "east"
}
}
🗺️ Symbolic Fog Map
###################### #......############### #......############### #......#......######## #......#......######## #......#......#.BG...# #......*......+.....D# ########......#......# ########...E.A#......# ###############......# ###############CFH...# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H
🤖 Predicted: A, B, C, D, E, F, G, H
🎯 Ground Truth: A, F, C, H
🧠 Cognitive Map Metrics
Global:
dir: 0.780
facing: 1
pos: 0.734
overall: 0.838
Global (Full):
dir: 0.592
facing: 0.778
pos: 0.636
overall: 0.669
Local:
dir: 1
facing: 1
pos: 0.862
overall: 0.954
Local (Newly):
dir: 1
facing: 1
pos: 0.862
overall: 0.954
Fog Probe:
precision: 0.500
recall: 1
overall: 0.667
📈 Turn Metrics
node_coverage: 0.857
edge_coverage: 0.341
is_action_fail: False
step: 7
action_counts:
move: 2
rotate: 4
return: 0
observe: 7
term: 0
forced_term: 0
query: 0
information_gain: 0.682
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 8
💬 Assistant Action
Actions: [Rotate(90), Observe()]
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
]
},
"red door": {
"position": [
-5,
0
]
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"bike": {
"position": [
-3,
2
],
"facing": "south"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"soda can": {
"position": [
-7,
3
]
},
"toilet": {
"position": [
8,
-2
],
"facing": "east"
},
"fridge": {
"position": [
7,
-3
]
},
"basket": {
"position": [
5,
1
]
},
"rubberduck": {
"position": [
-11,
2
],
"facing": "north"
},
"truck": {
"position": [
-8,
2
],
"facing": "east"
},
"lamp": {
"position": [
-10,
3
]
}
}
🎯 Ground Truth (Observed)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
-5,
0
],
"facing": "east"
}
}
🎯 Ground Truth (Full)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"airplane": {
"position": [
3,
-3
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
-5,
0
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"bike": {
"position": [
-1,
3
],
"facing": "+x"
},
"backpack": {
"position": [
1,
6
],
"facing": "+x"
},
"shoppingcart": {
"position": [
1,
6
],
"facing": "-y"
},
"green door": {
"position": [
0,
6
]
}
}
}
🎯 Ground Truth
{
"shoppingcart": {
"position": [
2,
6
],
"facing": "west"
},
"bike": {
"position": [
-1,
2
],
"facing": "south"
},
"backpack": {
"position": [
2,
4
],
"facing": "south"
},
"green door": {
"position": [
0,
7
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"bike": {
"position": [
-1,
3
],
"facing": "+x"
},
"backpack": {
"position": [
1,
6
],
"facing": "+x"
},
"shoppingcart": {
"position": [
1,
6
],
"facing": "-y"
},
"green door": {
"position": [
0,
6
]
}
}
}
🎯 Ground Truth
{
"shoppingcart": {
"position": [
2,
6
],
"facing": "west"
}
}
🗺️ Symbolic Fog Map
###################### #......############### #......############### #......#......######## #......#......######## #......#......#......# #......*......+...D..# ########......#......# ########......#......# ###############CA..FE# ###############.B....# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: A, B, C, D, E, F
🎯 Ground Truth: A, C, B
🧠 Cognitive Map Metrics
Global:
dir: 0.876
facing: 1
pos: 0.857
overall: 0.911
Global (Full):
dir: 0.767
facing: 0.889
pos: 0.800
overall: 0.819
Local:
dir: 0.500
facing: 1
pos: 0.810
overall: 0.770
Local (Newly):
dir: 1
facing: 1
pos: 0.862
overall: 0.954
Fog Probe:
precision: 0.500
recall: 1
overall: 0.667
📈 Turn Metrics
node_coverage: 0.929
edge_coverage: 0.415
is_action_fail: False
step: 8
action_counts:
move: 2
rotate: 5
return: 0
observe: 8
term: 0
forced_term: 0
query: 0
information_gain: 0.764
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 9
💬 Assistant Action
Actions: [Rotate(90), Observe()]
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
-4,
0
],
"facing": "south"
},
"green door": {
"position": [
2,
0
]
},
"red door": {
"position": [
-4,
0
]
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"bike": {
"position": [
-3,
2
],
"facing": "south"
},
"backpack": {
"position": [
0,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-3
],
"facing": "west"
},
"soda can": {
"position": [
-6,
3
]
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"fridge": {
"position": [
6,
-4
]
},
"basket": {
"position": [
5,
1
]
},
"rubberduck": {
"position": [
-8,
2
],
"facing": "north"
},
"truck": {
"position": [
-7,
2
],
"facing": "east"
},
"lamp": {
"position": [
-9,
3
]
}
}
🎯 Ground Truth (Observed)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
-5,
0
],
"facing": "south"
}
}
🎯 Ground Truth (Full)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"airplane": {
"position": [
3,
-3
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
-5,
0
],
"facing": "south"
}
}
🗺️ Symbolic Fog Map
###################### #......############### #......############### #......#......######## #......#......######## #......#......#......# #......*......+..D...# ########......#....B.# ########......#C...F.# ###############E.....# ###############.A....# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: A, B, C, D, E, F
🎯 Ground Truth: A, C, E
🧠 Cognitive Map Metrics
Global:
dir: 0.762
facing: 1
pos: 0.813
overall: 0.858
Global (Full):
dir: 0.667
facing: 0.889
pos: 0.759
overall: 0.772
Local:
(none)
Local (Newly):
(none)
Fog Probe:
precision: 0.500
recall: 1
overall: 0.667
📈 Turn Metrics
node_coverage: 0.929
edge_coverage: 0.415
is_action_fail: False
step: 9
action_counts:
move: 2
rotate: 6
return: 0
observe: 9
term: 0
forced_term: 0
query: 0
information_gain: 0.764
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 10
💬 Assistant Action
Actions: [JumpTo(green door), Observe()]
Actions: [JumpTo(green door), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
-5,
0
],
"facing": "south"
},
"green door": {
"position": [
1,
0
]
},
"red door": {
"position": [
-5,
0
]
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"basket": {
"position": [
4,
1
]
},
"toilet": {
"position": [
6,
-1
],
"facing": "east"
},
"fridge": {
"position": [
6,
-3
]
},
"rubberduck": {
"position": [
-10,
1
],
"facing": "north"
},
"truck": {
"position": [
-8,
2
],
"facing": "east"
},
"lamp": {
"position": [
-10,
3
]
},
"soda can": {
"position": [
-7,
3
]
},
"shoppingcart": {
"position": [
0,
-2
],
"facing": "west"
}
}
🎯 Ground Truth (Observed)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
-5,
0
],
"facing": "south"
}
}
🎯 Ground Truth (Full)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"airplane": {
"position": [
3,
-3
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
-5,
0
],
"facing": "south"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {}
}
🎯 Ground Truth
(no data)
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {}
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
###################### #......############### #......############### #......#......######## #......#......######## #......#......#.....D# #......*......+......# ########......#..E...# ########......#F.....# ###############......# ###############AB.C..# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: A, B, C, D, E, F
🎯 Ground Truth: B, A, F
🧠 Cognitive Map Metrics
Global:
dir: 0.886
facing: 1
pos: 0.866
overall: 0.917
Global (Full):
dir: 0.775
facing: 0.889
pos: 0.809
overall: 0.824
Local:
(none)
Local (Newly):
(none)
Fog Probe:
precision: 0.500
recall: 1
overall: 0.667
📈 Turn Metrics
node_coverage: 0.929
edge_coverage: 0.415
is_action_fail: True
step: 10
action_counts:
move: 2
rotate: 6
return: 0
observe: 10
term: 0
forced_term: 0
query: 0
information_gain: 0.764
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 11
💬 Assistant Action
Actions: [Rotate(-90), JumpTo(green door), Observe()]
Actions: [Rotate(-90), JumpTo(green door), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
2,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
]
},
"red door": {
"position": [
-5,
0
]
},
"lego man": {
"position": [
-3,
4
],
"facing": "south"
},
"bike": {
"position": [
-4,
4
],
"facing": "south"
},
"backpack": {
"position": [
0,
-2
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"fridge": {
"position": [
6,
-3
]
},
"basket": {
"position": [
5,
1
]
},
"rubberduck": {
"position": [
-9,
3
],
"facing": "north"
},
"truck": {
"position": [
-7,
2
],
"facing": "east"
},
"lamp": {
"position": [
-8,
4
]
},
"soda can": {
"position": [
-7,
3
]
},
"shoppingcart": {
"position": [
0,
-1
],
"facing": "west"
}
}
🎯 Ground Truth (Observed)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
2,
0
],
"facing": "east"
}
}
🎯 Ground Truth (Full)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"airplane": {
"position": [
3,
-3
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
2,
0
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"toilet": {
"position": [
1,
6
],
"facing": "+y"
},
"fridge": {
"position": [
3,
6
]
},
"basket": {
"position": [
-1,
3
]
}
}
}
🎯 Ground Truth
{
"toilet": {
"position": [
1,
5
],
"facing": "east"
},
"fridge": {
"position": [
3,
3
],
"facing": "east"
},
"basket": {
"position": [
-1,
3
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"toilet": {
"position": [
1,
6
],
"facing": "+y"
},
"fridge": {
"position": [
3,
6
]
},
"basket": {
"position": [
-1,
3
]
}
}
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
###################### #......############### #......############### #......#......######## #......#......######## #......#......#.....A# #......+......*....D.# ########......#......# ########......#..B...# ###############FE....# ###############..C...# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: A, B, C, E, F
🎯 Ground Truth: E, F, C
🧠 Cognitive Map Metrics
Global:
dir: 0.781
facing: 1
pos: 0.793
overall: 0.858
Global (Full):
dir: 0.683
facing: 0.889
pos: 0.740
overall: 0.771
Local:
dir: 0.667
facing: 1
pos: 0.762
overall: 0.810
Local (Newly):
(none)
Fog Probe:
precision: 0.600
recall: 1
overall: 0.750
📈 Turn Metrics
node_coverage: 0.929
edge_coverage: 0.415
is_action_fail: False
step: 11
action_counts:
move: 3
rotate: 7
return: 0
observe: 11
term: 0
forced_term: 0
query: 0
information_gain: 0.764
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 12
💬 Assistant Action
Actions: [Rotate(90), Observe()]
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
2,
0
],
"facing": "south"
},
"green door": {
"position": [
2,
0
]
},
"red door": {
"position": [
-3,
0
]
},
"lego man": {
"position": [
-3,
5
],
"facing": "south"
},
"bike": {
"position": [
-3,
3
],
"facing": "south"
},
"backpack": {
"position": [
-2,
-3
],
"facing": "south"
},
"toilet": {
"position": [
8,
-2
],
"facing": "east"
},
"fridge": {
"position": [
5,
-3
]
},
"basket": {
"position": [
5,
1
]
},
"soda can": {
"position": [
-4,
2
]
},
"shoppingcart": {
"position": [
0,
-3
],
"facing": "west"
},
"airplane": {
"position": [
3,
-3
],
"facing": "south"
},
"rubberduck": {
"position": [
-7,
1
],
"facing": "north"
},
"truck": {
"position": [
-6,
2
],
"facing": "east"
},
"lamp": {
"position": [
-8,
3
]
}
}
🎯 Ground Truth (Observed)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"airplane": {
"position": [
3,
-3
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
2,
0
],
"facing": "south"
}
}
🎯 Ground Truth (Full)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"airplane": {
"position": [
3,
-3
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
2,
0
],
"facing": "south"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"airplane": {
"position": [
-1,
3
],
"facing": "+y"
},
"shoppingcart": {
"position": [
2,
3
],
"facing": "+x"
},
"fridge": {
"position": [
-2,
6
]
}
}
}
🎯 Ground Truth
{
"shoppingcart": {
"position": [
1,
2
],
"facing": "west"
},
"airplane": {
"position": [
-1,
3
],
"facing": "south"
},
"fridge": {
"position": [
-3,
3
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"airplane": {
"position": [
-1,
3
],
"facing": "+y"
},
"shoppingcart": {
"position": [
2,
3
],
"facing": "+x"
},
"fridge": {
"position": [
-2,
6
]
}
}
}
🎯 Ground Truth
{
"airplane": {
"position": [
-1,
3
],
"facing": "south"
}
}
🧠 Cognitive Map Metrics
Global:
dir: 0.717
facing: 1
pos: 0.744
overall: 0.820
Global (Full):
dir: 0.717
facing: 1
pos: 0.744
overall: 0.820
Local:
dir: 0.333
facing: 1
pos: 0.743
overall: 0.692
Local (Newly):
dir: 1
facing: 1
pos: 1
overall: 1
Fog Probe:
(none)
📈 Turn Metrics
node_coverage: 1
edge_coverage: 0.463
is_action_fail: False
step: 12
action_counts:
move: 3
rotate: 8
return: 0
observe: 12
term: 0
forced_term: 0
query: 0
information_gain: 0.851
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 13
💬 Assistant Action
Actions: [Term()]
Actions: [Term()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
2,
0
],
"facing": "south"
},
"green door": {
"position": [
2,
0
]
},
"red door": {
"position": [
-5,
0
]
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"backpack": {
"position": [
0,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"soda can": {
"position": [
-7,
2
]
},
"rubberduck": {
"position": [
-10,
1
],
"facing": "north"
},
"truck": {
"position": [
-8,
2
],
"facing": "east"
},
"lamp": {
"position": [
-10,
2
]
},
"basket": {
"position": [
5,
1
]
},
"toilet": {
"position": [
6,
-1
],
"facing": "east"
},
"fridge": {
"position": [
5,
-3
]
},
"airplane": {
"position": [
3,
-3
],
"facing": "south"
}
}
🎯 Ground Truth (Observed)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"airplane": {
"position": [
3,
-3
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
2,
0
],
"facing": "south"
}
}
🎯 Ground Truth (Full)
{
"bike": {
"position": [
-3,
1
],
"facing": "south"
},
"airplane": {
"position": [
3,
-3
],
"facing": "south"
},
"rubberduck": {
"position": [
-11,
1
],
"facing": "north"
},
"lego man": {
"position": [
-3,
3
],
"facing": "south"
},
"toilet": {
"position": [
7,
-1
],
"facing": "east"
},
"truck": {
"position": [
-7,
1
],
"facing": "east"
},
"backpack": {
"position": [
-1,
-2
],
"facing": "south"
},
"shoppingcart": {
"position": [
1,
-2
],
"facing": "west"
},
"fridge": {
"position": [
5,
-3
],
"facing": "east"
},
"soda can": {
"position": [
-7,
3
],
"facing": "east"
},
"basket": {
"position": [
5,
1
],
"facing": "east"
},
"lamp": {
"position": [
-11,
5
],
"facing": "east"
},
"red door": {
"position": [
-5,
0
],
"facing": "east"
},
"green door": {
"position": [
2,
0
],
"facing": "east"
},
"agent": {
"position": [
2,
0
],
"facing": "south"
}
}
🧠 Cognitive Map Metrics
Global:
dir: 0.825
facing: 1
pos: 0.858
overall: 0.894
Global (Full):
dir: 0.825
facing: 1
pos: 0.858
overall: 0.894
Local:
(none)
Local (Newly):
(none)
Fog Probe:
(none)
📈 Turn Metrics
node_coverage: 1
edge_coverage: 0.463
is_action_fail: False
step: 13
action_counts:
move: 3
rotate: 8
return: 0
observe: 12
term: 1
forced_term: 0
query: 0
information_gain: 0.851
reward: -0.100
is_done: True
is_valid_action: True
🧭 False Belief Exploration
🔄 FB Turn 14
💬 Assistant Action
Actions: [Observe()]
Actions: [Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Changed: lego man:
dir: None
pos: None
facing: 1
overall: None
Retention: lego man:
dir: None
pos: None
facing: 0
overall: None
Unchanged (all):
dir: 0.864
facing: 1
pos: 0.789
overall: 0.884
🧾 False Belief Object Sets
all_objects: ['airplane', 'backpack', 'basket', 'bike', 'fridge', 'green door', 'lamp', 'lego man', 'red door', 'rubberduck', 'shoppingcart', 'soda can', 'toilet', 'truck']
all_changed_objects: ['bike', 'lego man', 'shoppingcart', 'toilet']
newly_observed_changed (this turn): ['lego man']
unchanged_objects: ['airplane', 'backpack', 'basket', 'fridge', 'green door', 'lamp', 'red door', 'rubberduck', 'soda can', 'truck']
pred_changed_keys: ['lego man']
pred_unchanged_keys: ['airplane', 'backpack', 'basket', 'bike', 'fridge', 'green door', 'lamp', 'lego man', 'red door', 'rubberduck', 'shoppingcart', 'soda can', 'toilet', 'truck']
🧭 False Belief Info
step: 1
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'lego man', 'ori': True}, {'name': 'toilet', 'ori': True}, {'name': 'shoppingcart', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: ['lego man']
newly_observed_unchanged_objects: []
🔄 FB Turn 15
💬 Assistant Action
Actions: [Rotate(90), JumpTo(green door), Observe()]
Actions: [Rotate(90), JumpTo(green door), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Changed: toilet:
dir: None
pos: None
facing: 0
overall: None
Retention: toilet:
dir: None
pos: None
facing: 0
overall: None
Changed: bike:
dir: None
pos: 0.717
facing: None
overall: None
Retention: bike:
dir: None
pos: 0.183
facing: None
overall: None
Unchanged (all):
dir: 0.697
facing: 0.200
pos: 0.753
overall: 0.550
🧾 False Belief Object Sets
all_objects: ['airplane', 'backpack', 'basket', 'bike', 'fridge', 'green door', 'lamp', 'lego man', 'red door', 'rubberduck', 'shoppingcart', 'soda can', 'toilet', 'truck']
all_changed_objects: ['bike', 'lego man', 'shoppingcart', 'toilet']
newly_observed_changed (this turn): ['bike', 'toilet']
unchanged_objects: ['airplane', 'backpack', 'basket', 'fridge', 'green door', 'lamp', 'red door', 'rubberduck', 'soda can', 'truck']
pred_changed_keys: ['bike', 'toilet']
pred_unchanged_keys: ['airplane', 'backpack', 'basket', 'bike', 'fridge', 'green door', 'lamp', 'lego man', 'red door', 'rubberduck', 'shoppingcart', 'soda can', 'toilet', 'truck']
🧭 False Belief Info
step: 2
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'lego man', 'ori': True}, {'name': 'toilet', 'ori': True}, {'name': 'shoppingcart', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: ['toilet', 'bike']
newly_observed_unchanged_objects: ['fridge', 'basket']
🔄 FB Turn 16
💬 Assistant Action
Actions: [Rotate(180), Observe()]
Actions: [Rotate(180), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Unchanged (all):
dir: 0.727
facing: 1
pos: 0.745
overall: 0.824
🧾 False Belief Object Sets
all_objects: ['airplane', 'backpack', 'basket', 'bike', 'fridge', 'green door', 'lamp', 'lego man', 'red door', 'rubberduck', 'shoppingcart', 'soda can', 'toilet', 'truck']
all_changed_objects: ['bike', 'lego man', 'shoppingcart', 'toilet']
newly_observed_changed (this turn): []
unchanged_objects: ['airplane', 'backpack', 'basket', 'fridge', 'green door', 'lamp', 'red door', 'rubberduck', 'soda can', 'truck']
pred_changed_keys: []
pred_unchanged_keys: ['airplane', 'backpack', 'basket', 'bike', 'fridge', 'green door', 'lamp', 'lego man', 'red door', 'rubberduck', 'shoppingcart', 'soda can', 'toilet', 'truck']
🧭 False Belief Info
step: 3
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'lego man', 'ori': True}, {'name': 'toilet', 'ori': True}, {'name': 'shoppingcart', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: []
newly_observed_unchanged_objects: ['red door', 'backpack']
🔄 FB Turn 17
💬 Assistant Action
Actions: [JumpTo(red door), Observe()]
Actions: [JumpTo(red door), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Changed: shoppingcart:
dir: None
pos: 0.420
facing: None
overall: None
Retention: shoppingcart:
dir: None
pos: 0.349
facing: None
overall: None
Unchanged (all):
dir: 0.667
facing: 0.200
pos: 0.671
overall: 0.513
🧾 False Belief Object Sets
all_objects: ['airplane', 'backpack', 'basket', 'bike', 'fridge', 'green door', 'lamp', 'lego man', 'red door', 'rubberduck', 'shoppingcart', 'soda can', 'toilet', 'truck']
all_changed_objects: ['bike', 'lego man', 'shoppingcart', 'toilet']
newly_observed_changed (this turn): ['shoppingcart']
unchanged_objects: ['airplane', 'backpack', 'basket', 'fridge', 'green door', 'lamp', 'red door', 'rubberduck', 'soda can', 'truck']
pred_changed_keys: ['shoppingcart']
pred_unchanged_keys: ['airplane', 'backpack', 'basket', 'bike', 'fridge', 'green door', 'lamp', 'lego man', 'red door', 'rubberduck', 'shoppingcart', 'soda can', 'toilet', 'truck']
🧭 False Belief Info
step: 4
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'lego man', 'ori': True}, {'name': 'toilet', 'ori': True}, {'name': 'shoppingcart', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: ['shoppingcart']
newly_observed_unchanged_objects: ['lamp', 'rubberduck', 'truck']
🔄 FB Turn 18
💬 Assistant Action
Actions: [Rotate(180), Observe()]
Actions: [Rotate(180), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Unchanged (all):
dir: 0.500
facing: 0.800
pos: 0.508
overall: 0.603
🧾 False Belief Object Sets
all_objects: ['airplane', 'backpack', 'basket', 'bike', 'fridge', 'green door', 'lamp', 'lego man', 'red door', 'rubberduck', 'shoppingcart', 'soda can', 'toilet', 'truck']
all_changed_objects: ['bike', 'lego man', 'shoppingcart', 'toilet']
newly_observed_changed (this turn): []
unchanged_objects: ['airplane', 'backpack', 'basket', 'fridge', 'green door', 'lamp', 'red door', 'rubberduck', 'soda can', 'truck']
pred_changed_keys: []
pred_unchanged_keys: ['airplane', 'backpack', 'basket', 'bike', 'fridge', 'green door', 'lamp', 'lego man', 'red door', 'rubberduck', 'shoppingcart', 'soda can', 'toilet', 'truck']
🧭 False Belief Info
step: 5
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'lego man', 'ori': True}, {'name': 'toilet', 'ori': True}, {'name': 'shoppingcart', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: []
newly_observed_unchanged_objects: ['green door']
🔄 FB Turn 19
💬 Assistant Action
Actions: [Rotate(-90), Observe()]
Actions: [Rotate(-90), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Unchanged (all):
dir: 0.879
facing: 1
pos: 0.776
overall: 0.885
🧾 False Belief Object Sets
all_objects: ['airplane', 'backpack', 'basket', 'bike', 'fridge', 'green door', 'lamp', 'lego man', 'red door', 'rubberduck', 'shoppingcart', 'soda can', 'toilet', 'truck']
all_changed_objects: ['bike', 'lego man', 'shoppingcart', 'toilet']
newly_observed_changed (this turn): []
unchanged_objects: ['airplane', 'backpack', 'basket', 'fridge', 'green door', 'lamp', 'red door', 'rubberduck', 'soda can', 'truck']
pred_changed_keys: []
pred_unchanged_keys: ['airplane', 'backpack', 'basket', 'bike', 'fridge', 'green door', 'lamp', 'lego man', 'red door', 'rubberduck', 'shoppingcart', 'soda can', 'toilet', 'truck']
🧭 False Belief Info
step: 6
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'lego man', 'ori': True}, {'name': 'toilet', 'ori': True}, {'name': 'shoppingcart', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: []
newly_observed_unchanged_objects: ['soda can']
🔄 FB Turn 20
💬 Assistant Action
Actions: [Rotate(90), Observe()]
Actions: [Rotate(90), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Unchanged (all):
dir: 0.712
facing: 1
pos: 0.660
overall: 0.791
🧾 False Belief Object Sets
all_objects: ['airplane', 'backpack', 'basket', 'bike', 'fridge', 'green door', 'lamp', 'lego man', 'red door', 'rubberduck', 'shoppingcart', 'soda can', 'toilet', 'truck']
all_changed_objects: ['bike', 'lego man', 'shoppingcart', 'toilet']
newly_observed_changed (this turn): []
unchanged_objects: ['airplane', 'backpack', 'basket', 'fridge', 'green door', 'lamp', 'red door', 'rubberduck', 'soda can', 'truck']
pred_changed_keys: []
pred_unchanged_keys: ['airplane', 'backpack', 'basket', 'bike', 'fridge', 'green door', 'lamp', 'lego man', 'red door', 'rubberduck', 'shoppingcart', 'soda can', 'toilet', 'truck']
🧭 False Belief Info
step: 7
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'lego man', 'ori': True}, {'name': 'toilet', 'ori': True}, {'name': 'shoppingcart', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: []
newly_observed_unchanged_objects: []
🔄 FB Turn 21
💬 Assistant Action
Actions: [Term(changes="lego man: orientation, toilet: orientation, bike: position, shoppingcart: position")]
Actions: [Term(changes="lego man: orientation, toilet: orientation, bike: position, shoppingcart: position")]
📈 Turn Metrics
reward: 1
is_done: True
is_valid_action: True
success: 1
🧭 False Belief Info
step: 8
correctly_identified_changes: 1
f1_overall: 1
f1_position: 1
f1_facing: 1
action_cost: 7
action_cost_after_seen: 3
ground_truth_changes: [{'name': 'lego man', 'ori': True}, {'name': 'toilet', 'ori': True}, {'name': 'shoppingcart', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: [{'name': 'lego man', 'ori': True}, {'name': 'toilet', 'ori': True}, {'name': 'bike', 'pos': True}, {'name': 'shoppingcart', 'pos': True}]
newly_observed_changed_objects: []
newly_observed_unchanged_objects: []
📊 Evaluation (default)
📊 Task: DirectionEvaluationTask
Question 1 (ID: 9d13d2040c8f0e75)
✅ Evaluation Results
id: 9d13d2040c8f0e75
question: You return to your starting position and face north.
From a Top-Down map, describe where basket is relative to toilet.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: north west, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 2 (ID: e2300d26c5895d2d)
✅ Evaluation Results
id: e2300d26c5895d2d
question: You return to your starting position and face north.
From a Top-Down map, describe where rubberduck is relative to lego man.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: west, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 3 (ID: c2a21d8364e4dca4)
✅ Evaluation Results
id: c2a21d8364e4dca4
question: You return to your starting position and face north.
From a Top-Down map, describe where shoppingcart is relative to airplane.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: north west, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
📊 Task: PovEvaluationTask
Question 1 (ID: 063960e28e84e1b8)
✅ Evaluation Results
id: 063960e28e84e1b8
question: Now you jump to rubberduck's direction, facing its direction.
Describe where lamp is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Question 2 (ID: dbbb7b93fba9d779)
✅ Evaluation Results
id: dbbb7b93fba9d779
question: Now you jump to lego man's direction, facing its direction.
Describe where shoppingcart is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front-left, slightly far
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 3 (ID: b2e43f1e4b97a13e)
✅ Evaluation Results
id: b2e43f1e4b97a13e
question: Now you jump to shoppingcart's direction, facing its direction.
Describe where bike is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front-right, slightly far
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
📊 Task: BackwardPovTextEvaluationTask
Question 1 (ID: 4022a79578213eb8)
✅ Evaluation Results
id: 4022a79578213eb8
question: Now you jump to an object's position, facing its direction.
You observe that shoppingcart is front-left, slightly far, facing right; red door is front-right, mid distance, on right wall; backpack is front-slight-left, slightly far, facing forward.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: lego man
final_pos: [9, 8]
final_ori: [0, -1]
final_observation: [{'name': 'shoppingcart', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'red door', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'on right wall'}, {'name': 'backpack', 'direction': 'front-slight-left', 'distance': 'slightly far', 'orientation': 'facing forward'}]
object_positions:
bike: [9.0, 6.0]
airplane: [15.0, 2.0]
rubberduck: [1.0, 6.0]
lego man: [9.0, 8.0]
toilet: [19.0, 4.0]
truck: [5.0, 6.0]
backpack: [11.0, 3.0]
shoppingcart: [13.0, 3.0]
fridge: [17.0, 2.0]
soda can: [5.0, 8.0]
basket: [17.0, 6.0]
lamp: [1.0, 10.0]
red door: [7.0, 5.0]
green door: [14.0, 5.0]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
room_id: 1
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: default
Question 2 (ID: d217d8691521707e)
✅ Evaluation Results
id: d217d8691521707e
question: Now you jump to an object's position, facing its direction.
You observe that red door is front-right, mid distance, on front wall.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: truck
final_pos: [5, 6]
final_ori: [1, 0]
final_observation: [{'name': 'red door', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'on front wall'}]
object_positions:
bike: [9.0, 6.0]
airplane: [15.0, 2.0]
rubberduck: [1.0, 6.0]
lego man: [9.0, 8.0]
toilet: [19.0, 4.0]
truck: [5.0, 6.0]
backpack: [11.0, 3.0]
shoppingcart: [13.0, 3.0]
fridge: [17.0, 2.0]
soda can: [5.0, 8.0]
basket: [17.0, 6.0]
lamp: [1.0, 10.0]
red door: [7.0, 5.0]
green door: [14.0, 5.0]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
room_id: 3
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: False
view_match: False
score: 0
evaluation_mode: default
Question 3 (ID: 7599c12dc388ac1a)
✅ Evaluation Results
id: 7599c12dc388ac1a
question: Now you jump to an object's position, facing its direction.
You observe that lamp is front, mid distance.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: rubberduck
final_pos: [1, 6]
final_ori: [0, 1]
final_observation: [{'name': 'lamp', 'direction': 'front', 'distance': 'mid distance', 'orientation': None}]
object_positions:
bike: [9.0, 6.0]
airplane: [15.0, 2.0]
rubberduck: [1.0, 6.0]
lego man: [9.0, 8.0]
toilet: [19.0, 4.0]
truck: [5.0, 6.0]
backpack: [11.0, 3.0]
shoppingcart: [13.0, 3.0]
fridge: [17.0, 2.0]
soda can: [5.0, 8.0]
basket: [17.0, 6.0]
lamp: [1.0, 10.0]
red door: [7.0, 5.0]
green door: [14.0, 5.0]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
room_id: 3
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: default
📊 Task: Action2ViewEvaluationTask
Question 1 (ID: e5146a0f564c765f)
✅ Evaluation Results
id: e5146a0f564c765f
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Rotate(90)
2. Jump to the object at front, near.
3. Jump to the object at front-right, slightly far.
After executing the actions, what is the ego relation of toilet relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-left, mid distance
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Question 2 (ID: 90f3cb1a03f64a70)
✅ Evaluation Results
id: 90f3cb1a03f64a70
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Rotate(90)
2. Jump to the object at front, near.
3. Jump to the object at front-right, slightly far.
4. Rotate(-180)
After executing the actions, what is the ego relation of green door relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-right, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 3 (ID: 7b132df57af38164)
✅ Evaluation Results
id: 7b132df57af38164
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Jump to the object at front-left, slightly far.
2. Rotate(90)
3. Jump to the object at front-right, slightly far.
4. Rotate(90)
After executing the actions, what is the ego relation of fridge relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
📊 Task: View2ActionTextEvaluationTask
Question 1 (ID: a1b55f183063b5a2)
✅ Evaluation Results
id: a1b55f183063b5a2
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
truck is at front, near, facing left; red door is at front-left, mid distance, on left wall
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [5, 8]
final_ori: [0, -1]
room_id: 3
init_pos: [12, 5]
init_ori: [0, 1]
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', -90], ['jumpto', 'red door'], ['rotate', 90], ['jumpto', 'soda can'], ['rotate', 180]]
final_observation: [{'name': 'truck', 'direction': 'front', 'distance': 'near', 'orientation': 'facing left'}, {'name': 'red door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on left wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: True
final_pos: [1.0, 10.0]
final_ori: [0, -1]
visible_match: False
score: 0
evaluation_mode: default
Question 2 (ID: 514e1fc99b9dfec8)
✅ Evaluation Results
id: 514e1fc99b9dfec8
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
basket is at front-right, slightly far; green door is at front-slight-left, mid distance, on left wall
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [15, 2]
final_ori: [0, 1]
room_id: 2
init_pos: [12, 5]
init_ori: [0, 1]
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', 90], ['jumpto', 'green door'], ['rotate', 90], ['jumpto', 'airplane'], ['rotate', 180]]
final_observation: [{'name': 'basket', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': None}, {'name': 'green door', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'on left wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: True
final_pos: [17.0, 2.0]
final_ori: [0, 1]
visible_match: False
score: 0
evaluation_mode: default
Question 3 (ID: bf8d3e056f97abbf)
✅ Evaluation Results
id: bf8d3e056f97abbf
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
fridge is at front-right, mid distance
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [19, 4]
final_ori: [0, -1]
room_id: 2
init_pos: [12, 5]
init_ori: [0, 1]
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', 90], ['jumpto', 'green door'], ['jumpto', 'toilet'], ['rotate', 90]]
final_observation: [{'name': 'fridge', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: False
final_pos: [17.0, 6.0]
final_ori: [1, 0]
visible_match: False
score: 0
evaluation_mode: default
📊 Task: AlloMappingEvaluationTask
Question 1 (ID: 11808d88af254359)
✅ Evaluation Results
id: 11808d88af254359
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: basket, toilet, fridge.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[5, 1], [7, -1], [5, -3]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.854
threshold: 0.900
score: 0.854
evaluation_mode: default
Question 2 (ID: 2b6ee8e5881229c1)
✅ Evaluation Results
id: 2b6ee8e5881229c1
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: rubberduck, lego man, basket, bike, airplane.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-11, 1], [-3, 3], [5, 1], [-3, 1], [3, -3]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.893
threshold: 0.900
score: 0.893
evaluation_mode: default
Question 3 (ID: 6a481cb096f39335)
✅ Evaluation Results
id: 6a481cb096f39335
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: backpack, basket, lamp, airplane, rubberduck.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-1, -2], [5, 1], [-11, 5], [3, -3], [-11, 1]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.842
threshold: 0.900
score: 0.842
evaluation_mode: default
📊 Task: RotEvaluationTask
Question 1 (ID: 37b10f92ed55a226)
✅ Evaluation Results
id: 37b10f92ed55a226
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning clockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: basket, fridge, shoppingcart.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['basket', 'fridge', 'shoppingcart']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 2 (ID: 4b2dc57e77686a8b)
✅ Evaluation Results
id: 4b2dc57e77686a8b
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning counterclockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: backpack, lego man, rubberduck.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['lego man', 'rubberduck', 'backpack']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 3 (ID: 3af660d5e0016740)
✅ Evaluation Results
id: 3af660d5e0016740
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning clockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: airplane, backpack, lamp, toilet.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['toilet', 'airplane', 'backpack', 'lamp']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
📊 Task: RotDualEvaluationTask
Question 1 (ID: 03400bfc656221c4)
✅ Evaluation Results
id: 03400bfc656221c4
question: You return to your starting position and face north.
You performed a complete 360° rotation in place.
Assume all walls are removed (you can see through walls), so every object is visible.
During the rotation, these objects appeared directly in front of you in this order:
basket, fridge, shoppingcart
Based on this sequence, in which direction did you rotate?
Answer format: clockwise or counterclockwise
Example: clockwise
action: None
answer: clockwise
task_type: RotDualEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 2 (ID: 6d8e9eae3f521541)
✅ Evaluation Results
id: 6d8e9eae3f521541
question: You return to your starting position and face north.
You performed a complete 360° rotation in place.
Assume all walls are removed (you can see through walls), so every object is visible.
During the rotation, these objects appeared directly in front of you in this order:
lego man, rubberduck, backpack
Based on this sequence, in which direction did you rotate?
Answer format: clockwise or counterclockwise
Example: clockwise
action: None
answer: counterclockwise
task_type: RotDualEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 3 (ID: a0d83aabc0d2bbd6)
✅ Evaluation Results
id: a0d83aabc0d2bbd6
question: You return to your starting position and face north.
You performed a complete 360° rotation in place.
Assume all walls are removed (you can see through walls), so every object is visible.
During the rotation, these objects appeared directly in front of you in this order:
toilet, airplane, backpack, lamp
Based on this sequence, in which direction did you rotate?
Answer format: clockwise or counterclockwise
Example: clockwise
action: None
answer: clockwise
task_type: RotDualEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
📊 Task: Location2ViewEvaluationTask
Question 1 (ID: 3af268195886a8df)
✅ Evaluation Results
id: 3af268195886a8df
question: Treat the green door as the new 'origin' (0, 0).
You move to (2, -2) and face east.
What is the egocentric relation of toilet?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front-slight-left, mid distance
task_type: Action2LocationEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 2 (ID: 56728a7c1f765ce3)
✅ Evaluation Results
id: 56728a7c1f765ce3
question: Treat the green door as the new 'origin' (0, 0).
You move to (2, -2) and face north.
What is the egocentric relation of basket?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front-slight-right, mid distance
task_type: Action2LocationEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 3 (ID: 843e8e1b199cd022)
✅ Evaluation Results
id: 843e8e1b199cd022
question: Treat the red door as the new 'origin' (0, 0).
You move to (-3, 4) and face south.
What is the egocentric relation of truck?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front-slight-left, mid distance
task_type: Action2LocationEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
📊 Task: View2LocationTextEvaluationTask
Question 1 (ID: 0627604bb51f9403)
✅ Evaluation Results
id: 0627604bb51f9403
question: You move to a new location and face east.
You observe: toilet is at front-slight-left, mid distance, facing forward; fridge is at front-right, near
Treat the green door as the new 'origin' (0, 0).
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [2, -2]
final_pos: [16, 3]
final_ori: [1, 0]
room_id: 2
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'toilet', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'fridge', 'direction': 'front-right', 'distance': 'near', 'orientation': None}]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: Location2ActionTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.702
raw_score: 0.702
score: 0.702
evaluation_mode: default
Question 2 (ID: 9d51c9ea1c03280c)
✅ Evaluation Results
id: 9d51c9ea1c03280c
question: You move to a new location and face north.
You observe: basket is at front-slight-right, mid distance; green door is at front-left, mid distance, on left wall
Treat the green door as the new 'origin' (0, 0).
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [2, -2]
final_pos: [16, 3]
final_ori: [0, 1]
room_id: 2
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'basket', 'direction': 'front-slight-right', 'distance': 'mid distance', 'orientation': None}, {'name': 'green door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on left wall'}]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: Location2ActionTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: default
Question 3 (ID: 231594f7927a5d40)
✅ Evaluation Results
id: 231594f7927a5d40
question: You move to a new location and face south.
You observe: truck is at front-slight-left, mid distance, facing left; rubberduck is at front-right, slightly far, facing backward; red door is at front-left, slightly far, on left wall
Treat the red door as the new 'origin' (0, 0).
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [-3, 4]
final_pos: [4, 9]
final_ori: [0, -1]
room_id: 3
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'truck', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing left'}, {'name': 'rubberduck', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing backward'}, {'name': 'red door', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'on left wall'}]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: Location2ActionTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.819
raw_score: 0.819
score: 0.819
evaluation_mode: default
📊 Evaluation (prompt_cogmap)
📊 Task: DirectionEvaluationTask
Question 1 (ID: 9d13d2040c8f0e75)
✅ Evaluation Results
id: 9d13d2040c8f0e75
question: You return to your starting position and face north.
From a Top-Down map, describe where basket is relative to toilet.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: north west, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Question 2 (ID: e2300d26c5895d2d)
✅ Evaluation Results
id: e2300d26c5895d2d
question: You return to your starting position and face north.
From a Top-Down map, describe where rubberduck is relative to lego man.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: west, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: prompt_cogmap
Question 3 (ID: c2a21d8364e4dca4)
✅ Evaluation Results
id: c2a21d8364e4dca4
question: You return to your starting position and face north.
From a Top-Down map, describe where shoppingcart is relative to airplane.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: north west, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
📊 Task: PovEvaluationTask
Question 1 (ID: 063960e28e84e1b8)
✅ Evaluation Results
id: 063960e28e84e1b8
question: Now you jump to rubberduck's direction, facing its direction.
Describe where lamp is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: prompt_cogmap
Question 2 (ID: dbbb7b93fba9d779)
✅ Evaluation Results
id: dbbb7b93fba9d779
question: Now you jump to lego man's direction, facing its direction.
Describe where shoppingcart is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front-left, slightly far
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Question 3 (ID: b2e43f1e4b97a13e)
✅ Evaluation Results
id: b2e43f1e4b97a13e
question: Now you jump to shoppingcart's direction, facing its direction.
Describe where bike is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front-right, slightly far
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
📊 Task: BackwardPovTextEvaluationTask
Question 1 (ID: 4022a79578213eb8)
✅ Evaluation Results
id: 4022a79578213eb8
question: Now you jump to an object's position, facing its direction.
You observe that shoppingcart is front-left, slightly far, facing right; red door is front-right, mid distance, on right wall; backpack is front-slight-left, slightly far, facing forward.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: lego man
final_pos: [9, 8]
final_ori: [0, -1]
final_observation: [{'name': 'shoppingcart', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'red door', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'on right wall'}, {'name': 'backpack', 'direction': 'front-slight-left', 'distance': 'slightly far', 'orientation': 'facing forward'}]
object_positions:
bike: [9.0, 6.0]
airplane: [15.0, 2.0]
rubberduck: [1.0, 6.0]
lego man: [9.0, 8.0]
toilet: [19.0, 4.0]
truck: [5.0, 6.0]
backpack: [11.0, 3.0]
shoppingcart: [13.0, 3.0]
fridge: [17.0, 2.0]
soda can: [5.0, 8.0]
basket: [17.0, 6.0]
lamp: [1.0, 10.0]
red door: [7.0, 5.0]
green door: [14.0, 5.0]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
room_id: 1
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: prompt_cogmap
Question 2 (ID: d217d8691521707e)
✅ Evaluation Results
id: d217d8691521707e
question: Now you jump to an object's position, facing its direction.
You observe that red door is front-right, mid distance, on front wall.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: truck
final_pos: [5, 6]
final_ori: [1, 0]
final_observation: [{'name': 'red door', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'on front wall'}]
object_positions:
bike: [9.0, 6.0]
airplane: [15.0, 2.0]
rubberduck: [1.0, 6.0]
lego man: [9.0, 8.0]
toilet: [19.0, 4.0]
truck: [5.0, 6.0]
backpack: [11.0, 3.0]
shoppingcart: [13.0, 3.0]
fridge: [17.0, 2.0]
soda can: [5.0, 8.0]
basket: [17.0, 6.0]
lamp: [1.0, 10.0]
red door: [7.0, 5.0]
green door: [14.0, 5.0]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
room_id: 3
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: False
view_match: False
score: 0
evaluation_mode: prompt_cogmap
Question 3 (ID: 7599c12dc388ac1a)
✅ Evaluation Results
id: 7599c12dc388ac1a
question: Now you jump to an object's position, facing its direction.
You observe that lamp is front, mid distance.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: rubberduck
final_pos: [1, 6]
final_ori: [0, 1]
final_observation: [{'name': 'lamp', 'direction': 'front', 'distance': 'mid distance', 'orientation': None}]
object_positions:
bike: [9.0, 6.0]
airplane: [15.0, 2.0]
rubberduck: [1.0, 6.0]
lego man: [9.0, 8.0]
toilet: [19.0, 4.0]
truck: [5.0, 6.0]
backpack: [11.0, 3.0]
shoppingcart: [13.0, 3.0]
fridge: [17.0, 2.0]
soda can: [5.0, 8.0]
basket: [17.0, 6.0]
lamp: [1.0, 10.0]
red door: [7.0, 5.0]
green door: [14.0, 5.0]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
room_id: 3
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: prompt_cogmap
📊 Task: Action2ViewEvaluationTask
Question 1 (ID: e5146a0f564c765f)
✅ Evaluation Results
id: e5146a0f564c765f
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Rotate(90)
2. Jump to the object at front, near.
3. Jump to the object at front-right, slightly far.
After executing the actions, what is the ego relation of toilet relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-left, mid distance
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: prompt_cogmap
Question 2 (ID: 90f3cb1a03f64a70)
✅ Evaluation Results
id: 90f3cb1a03f64a70
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Rotate(90)
2. Jump to the object at front, near.
3. Jump to the object at front-right, slightly far.
4. Rotate(-180)
After executing the actions, what is the ego relation of green door relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-right, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Question 3 (ID: 7b132df57af38164)
✅ Evaluation Results
id: 7b132df57af38164
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Jump to the object at front-left, slightly far.
2. Rotate(90)
3. Jump to the object at front-right, slightly far.
4. Rotate(90)
After executing the actions, what is the ego relation of fridge relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
📊 Task: View2ActionTextEvaluationTask
Question 1 (ID: a1b55f183063b5a2)
✅ Evaluation Results
id: a1b55f183063b5a2
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
truck is at front, near, facing left; red door is at front-left, mid distance, on left wall
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [5, 8]
final_ori: [0, -1]
room_id: 3
init_pos: [12, 5]
init_ori: [0, 1]
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', -90], ['jumpto', 'red door'], ['rotate', 90], ['jumpto', 'soda can'], ['rotate', 180]]
final_observation: [{'name': 'truck', 'direction': 'front', 'distance': 'near', 'orientation': 'facing left'}, {'name': 'red door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on left wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: True
final_pos: [1.0, 10.0]
final_ori: [0, -1]
visible_match: False
score: 0
evaluation_mode: prompt_cogmap
Question 2 (ID: 514e1fc99b9dfec8)
✅ Evaluation Results
id: 514e1fc99b9dfec8
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
basket is at front-right, slightly far; green door is at front-slight-left, mid distance, on left wall
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [15, 2]
final_ori: [0, 1]
room_id: 2
init_pos: [12, 5]
init_ori: [0, 1]
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', 90], ['jumpto', 'green door'], ['rotate', 90], ['jumpto', 'airplane'], ['rotate', 180]]
final_observation: [{'name': 'basket', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': None}, {'name': 'green door', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'on left wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: True
final_pos: [19.0, 4.0]
final_ori: [0, 1]
visible_match: False
score: 0
evaluation_mode: prompt_cogmap
Question 3 (ID: bf8d3e056f97abbf)
✅ Evaluation Results
id: bf8d3e056f97abbf
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
fridge is at front-right, mid distance
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [19, 4]
final_ori: [0, -1]
room_id: 2
init_pos: [12, 5]
init_ori: [0, 1]
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', 90], ['jumpto', 'green door'], ['jumpto', 'toilet'], ['rotate', 90]]
final_observation: [{'name': 'fridge', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: False
final_pos: [14.0, 5.0]
final_ori: [1, 0]
visible_match: False
score: 0
evaluation_mode: prompt_cogmap
📊 Task: AlloMappingEvaluationTask
Question 1 (ID: 11808d88af254359)
✅ Evaluation Results
id: 11808d88af254359
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: basket, toilet, fridge.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[5, 1], [7, -1], [5, -3]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.841
threshold: 0.900
score: 0.841
evaluation_mode: prompt_cogmap
Question 2 (ID: 2b6ee8e5881229c1)
✅ Evaluation Results
id: 2b6ee8e5881229c1
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: rubberduck, lego man, basket, bike, airplane.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-11, 1], [-3, 3], [5, 1], [-3, 1], [3, -3]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.745
threshold: 0.900
score: 0.745
evaluation_mode: prompt_cogmap
Question 3 (ID: 6a481cb096f39335)
✅ Evaluation Results
id: 6a481cb096f39335
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: backpack, basket, lamp, airplane, rubberduck.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-1, -2], [5, 1], [-11, 5], [3, -3], [-11, 1]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.785
threshold: 0.900
score: 0.785
evaluation_mode: prompt_cogmap
📊 Task: RotEvaluationTask
Question 1 (ID: 37b10f92ed55a226)
✅ Evaluation Results
id: 37b10f92ed55a226
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning clockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: basket, fridge, shoppingcart.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['basket', 'fridge', 'shoppingcart']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Question 2 (ID: 4b2dc57e77686a8b)
✅ Evaluation Results
id: 4b2dc57e77686a8b
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning counterclockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: backpack, lego man, rubberduck.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['lego man', 'rubberduck', 'backpack']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Question 3 (ID: 3af660d5e0016740)
✅ Evaluation Results
id: 3af660d5e0016740
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning clockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: airplane, backpack, lamp, toilet.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['toilet', 'airplane', 'backpack', 'lamp']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
📊 Task: Location2ViewEvaluationTask
Question 1 (ID: 3af268195886a8df)
✅ Evaluation Results
id: 3af268195886a8df
question: Treat the green door as the new 'origin' (0, 0).
You move to (2, -2) and face east.
What is the egocentric relation of toilet?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front-slight-left, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Question 2 (ID: 56728a7c1f765ce3)
✅ Evaluation Results
id: 56728a7c1f765ce3
question: Treat the green door as the new 'origin' (0, 0).
You move to (2, -2) and face north.
What is the egocentric relation of basket?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front-slight-right, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Question 3 (ID: 843e8e1b199cd022)
✅ Evaluation Results
id: 843e8e1b199cd022
question: Treat the red door as the new 'origin' (0, 0).
You move to (-3, 4) and face south.
What is the egocentric relation of truck?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front-slight-left, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: prompt_cogmap
📊 Task: View2LocationTextEvaluationTask
Question 1 (ID: 0627604bb51f9403)
✅ Evaluation Results
id: 0627604bb51f9403
question: You move to a new location and face east.
You observe: toilet is at front-slight-left, mid distance, facing forward; fridge is at front-right, near
Treat the green door as the new 'origin' (0, 0).
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [2, -2]
final_pos: [16, 3]
final_ori: [1, 0]
room_id: 2
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'toilet', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'fridge', 'direction': 'front-right', 'distance': 'near', 'orientation': None}]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: prompt_cogmap
Question 2 (ID: 9d51c9ea1c03280c)
✅ Evaluation Results
id: 9d51c9ea1c03280c
question: You move to a new location and face north.
You observe: basket is at front-slight-right, mid distance; green door is at front-left, mid distance, on left wall
Treat the green door as the new 'origin' (0, 0).
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [2, -2]
final_pos: [16, 3]
final_ori: [0, 1]
room_id: 2
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'basket', 'direction': 'front-slight-right', 'distance': 'mid distance', 'orientation': None}, {'name': 'green door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on left wall'}]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: prompt_cogmap
Question 3 (ID: 231594f7927a5d40)
✅ Evaluation Results
id: 231594f7927a5d40
question: You move to a new location and face south.
You observe: truck is at front-slight-left, mid distance, facing left; rubberduck is at front-right, slightly far, facing backward; red door is at front-left, slightly far, on left wall
Treat the red door as the new 'origin' (0, 0).
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [-3, 4]
final_pos: [4, 9]
final_ori: [0, -1]
room_id: 3
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'truck', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing left'}, {'name': 'rubberduck', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing backward'}, {'name': 'red door', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'on left wall'}]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: prompt_cogmap
📊 Evaluation (use_gt_cogmap)
📊 Task: DirectionEvaluationTask
Question 1 (ID: 9d13d2040c8f0e75)
✅ Evaluation Results
id: 9d13d2040c8f0e75
question: You return to your starting position and face north.
From a Top-Down map, describe where basket is relative to toilet.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: north west, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: e2300d26c5895d2d)
✅ Evaluation Results
id: e2300d26c5895d2d
question: You return to your starting position and face north.
From a Top-Down map, describe where rubberduck is relative to lego man.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: west, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 3 (ID: c2a21d8364e4dca4)
✅ Evaluation Results
id: c2a21d8364e4dca4
question: You return to your starting position and face north.
From a Top-Down map, describe where shoppingcart is relative to airplane.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: north west, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
📊 Task: PovEvaluationTask
Question 1 (ID: 063960e28e84e1b8)
✅ Evaluation Results
id: 063960e28e84e1b8
question: Now you jump to rubberduck's direction, facing its direction.
Describe where lamp is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: dbbb7b93fba9d779)
✅ Evaluation Results
id: dbbb7b93fba9d779
question: Now you jump to lego man's direction, facing its direction.
Describe where shoppingcart is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front-left, slightly far
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 3 (ID: b2e43f1e4b97a13e)
✅ Evaluation Results
id: b2e43f1e4b97a13e
question: Now you jump to shoppingcart's direction, facing its direction.
Describe where bike is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front-right, slightly far
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
📊 Task: BackwardPovTextEvaluationTask
Question 1 (ID: 4022a79578213eb8)
✅ Evaluation Results
id: 4022a79578213eb8
question: Now you jump to an object's position, facing its direction.
You observe that shoppingcart is front-left, slightly far, facing right; red door is front-right, mid distance, on right wall; backpack is front-slight-left, slightly far, facing forward.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: lego man
final_pos: [9, 8]
final_ori: [0, -1]
final_observation: [{'name': 'shoppingcart', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'red door', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'on right wall'}, {'name': 'backpack', 'direction': 'front-slight-left', 'distance': 'slightly far', 'orientation': 'facing forward'}]
object_positions:
bike: [9.0, 6.0]
airplane: [15.0, 2.0]
rubberduck: [1.0, 6.0]
lego man: [9.0, 8.0]
toilet: [19.0, 4.0]
truck: [5.0, 6.0]
backpack: [11.0, 3.0]
shoppingcart: [13.0, 3.0]
fridge: [17.0, 2.0]
soda can: [5.0, 8.0]
basket: [17.0, 6.0]
lamp: [1.0, 10.0]
red door: [7.0, 5.0]
green door: [14.0, 5.0]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
room_id: 1
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: d217d8691521707e)
✅ Evaluation Results
id: d217d8691521707e
question: Now you jump to an object's position, facing its direction.
You observe that red door is front-right, mid distance, on front wall.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: truck
final_pos: [5, 6]
final_ori: [1, 0]
final_observation: [{'name': 'red door', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'on front wall'}]
object_positions:
bike: [9.0, 6.0]
airplane: [15.0, 2.0]
rubberduck: [1.0, 6.0]
lego man: [9.0, 8.0]
toilet: [19.0, 4.0]
truck: [5.0, 6.0]
backpack: [11.0, 3.0]
shoppingcart: [13.0, 3.0]
fridge: [17.0, 2.0]
soda can: [5.0, 8.0]
basket: [17.0, 6.0]
lamp: [1.0, 10.0]
red door: [7.0, 5.0]
green door: [14.0, 5.0]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
room_id: 3
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_gt_cogmap
Question 3 (ID: 7599c12dc388ac1a)
✅ Evaluation Results
id: 7599c12dc388ac1a
question: Now you jump to an object's position, facing its direction.
You observe that lamp is front, mid distance.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: rubberduck
final_pos: [1, 6]
final_ori: [0, 1]
final_observation: [{'name': 'lamp', 'direction': 'front', 'distance': 'mid distance', 'orientation': None}]
object_positions:
bike: [9.0, 6.0]
airplane: [15.0, 2.0]
rubberduck: [1.0, 6.0]
lego man: [9.0, 8.0]
toilet: [19.0, 4.0]
truck: [5.0, 6.0]
backpack: [11.0, 3.0]
shoppingcart: [13.0, 3.0]
fridge: [17.0, 2.0]
soda can: [5.0, 8.0]
basket: [17.0, 6.0]
lamp: [1.0, 10.0]
red door: [7.0, 5.0]
green door: [14.0, 5.0]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
room_id: 3
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_gt_cogmap
📊 Task: Action2ViewEvaluationTask
Question 1 (ID: e5146a0f564c765f)
✅ Evaluation Results
id: e5146a0f564c765f
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Rotate(90)
2. Jump to the object at front, near.
3. Jump to the object at front-right, slightly far.
After executing the actions, what is the ego relation of toilet relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-left, mid distance
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: 90f3cb1a03f64a70)
✅ Evaluation Results
id: 90f3cb1a03f64a70
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Rotate(90)
2. Jump to the object at front, near.
3. Jump to the object at front-right, slightly far.
4. Rotate(-180)
After executing the actions, what is the ego relation of green door relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-right, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 3 (ID: 7b132df57af38164)
✅ Evaluation Results
id: 7b132df57af38164
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Jump to the object at front-left, slightly far.
2. Rotate(90)
3. Jump to the object at front-right, slightly far.
4. Rotate(90)
After executing the actions, what is the ego relation of fridge relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
📊 Task: View2ActionTextEvaluationTask
Question 1 (ID: a1b55f183063b5a2)
✅ Evaluation Results
id: a1b55f183063b5a2
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
truck is at front, near, facing left; red door is at front-left, mid distance, on left wall
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [5, 8]
final_ori: [0, -1]
room_id: 3
init_pos: [12, 5]
init_ori: [0, 1]
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', -90], ['jumpto', 'red door'], ['rotate', 90], ['jumpto', 'soda can'], ['rotate', 180]]
final_observation: [{'name': 'truck', 'direction': 'front', 'distance': 'near', 'orientation': 'facing left'}, {'name': 'red door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on left wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [5.0, 8.0]
final_ori: [0, -1]
visible_match: True
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: 514e1fc99b9dfec8)
✅ Evaluation Results
id: 514e1fc99b9dfec8
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
basket is at front-right, slightly far; green door is at front-slight-left, mid distance, on left wall
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [15, 2]
final_ori: [0, 1]
room_id: 2
init_pos: [12, 5]
init_ori: [0, 1]
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', 90], ['jumpto', 'green door'], ['rotate', 90], ['jumpto', 'airplane'], ['rotate', 180]]
final_observation: [{'name': 'basket', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': None}, {'name': 'green door', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'on left wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [15.0, 2.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: use_gt_cogmap
Question 3 (ID: bf8d3e056f97abbf)
✅ Evaluation Results
id: bf8d3e056f97abbf
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
fridge is at front-right, mid distance
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [19, 4]
final_ori: [0, -1]
room_id: 2
init_pos: [12, 5]
init_ori: [0, 1]
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', 90], ['jumpto', 'green door'], ['jumpto', 'toilet'], ['rotate', 90]]
final_observation: [{'name': 'fridge', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [19.0, 4.0]
final_ori: [0, -1]
visible_match: True
score: 1
evaluation_mode: use_gt_cogmap
📊 Task: AlloMappingEvaluationTask
Question 1 (ID: 11808d88af254359)
✅ Evaluation Results
id: 11808d88af254359
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: basket, toilet, fridge.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[5, 1], [7, -1], [5, -3]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
threshold: 0.900
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: 2b6ee8e5881229c1)
✅ Evaluation Results
id: 2b6ee8e5881229c1
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: rubberduck, lego man, basket, bike, airplane.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-11, 1], [-3, 3], [5, 1], [-3, 1], [3, -3]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
threshold: 0.900
score: 1
evaluation_mode: use_gt_cogmap
Question 3 (ID: 6a481cb096f39335)
✅ Evaluation Results
id: 6a481cb096f39335
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: backpack, basket, lamp, airplane, rubberduck.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-1, -2], [5, 1], [-11, 5], [3, -3], [-11, 1]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
threshold: 0.900
score: 1
evaluation_mode: use_gt_cogmap
📊 Task: RotEvaluationTask
Question 1 (ID: 37b10f92ed55a226)
✅ Evaluation Results
id: 37b10f92ed55a226
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning clockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: basket, fridge, shoppingcart.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['basket', 'fridge', 'shoppingcart']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: 4b2dc57e77686a8b)
✅ Evaluation Results
id: 4b2dc57e77686a8b
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning counterclockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: backpack, lego man, rubberduck.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['lego man', 'rubberduck', 'backpack']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 3 (ID: 3af660d5e0016740)
✅ Evaluation Results
id: 3af660d5e0016740
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning clockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: airplane, backpack, lamp, toilet.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['toilet', 'airplane', 'backpack', 'lamp']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
📊 Task: Location2ViewEvaluationTask
Question 1 (ID: 3af268195886a8df)
✅ Evaluation Results
id: 3af268195886a8df
question: Treat the green door as the new 'origin' (0, 0).
You move to (2, -2) and face east.
What is the egocentric relation of toilet?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front-slight-left, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: 56728a7c1f765ce3)
✅ Evaluation Results
id: 56728a7c1f765ce3
question: Treat the green door as the new 'origin' (0, 0).
You move to (2, -2) and face north.
What is the egocentric relation of basket?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front-slight-right, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: use_gt_cogmap
Question 3 (ID: 843e8e1b199cd022)
✅ Evaluation Results
id: 843e8e1b199cd022
question: Treat the red door as the new 'origin' (0, 0).
You move to (-3, 4) and face south.
What is the egocentric relation of truck?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front-slight-left, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
📊 Task: View2LocationTextEvaluationTask
Question 1 (ID: 0627604bb51f9403)
✅ Evaluation Results
id: 0627604bb51f9403
question: You move to a new location and face east.
You observe: toilet is at front-slight-left, mid distance, facing forward; fridge is at front-right, near
Treat the green door as the new 'origin' (0, 0).
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [2, -2]
final_pos: [16, 3]
final_ori: [1, 0]
room_id: 2
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'toilet', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'fridge', 'direction': 'front-right', 'distance': 'near', 'orientation': None}]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: 9d51c9ea1c03280c)
✅ Evaluation Results
id: 9d51c9ea1c03280c
question: You move to a new location and face north.
You observe: basket is at front-slight-right, mid distance; green door is at front-left, mid distance, on left wall
Treat the green door as the new 'origin' (0, 0).
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [2, -2]
final_pos: [16, 3]
final_ori: [0, 1]
room_id: 2
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'basket', 'direction': 'front-slight-right', 'distance': 'mid distance', 'orientation': None}, {'name': 'green door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on left wall'}]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_gt_cogmap
Question 3 (ID: 231594f7927a5d40)
✅ Evaluation Results
id: 231594f7927a5d40
question: You move to a new location and face south.
You observe: truck is at front-slight-left, mid distance, facing left; rubberduck is at front-right, slightly far, facing backward; red door is at front-left, slightly far, on left wall
Treat the red door as the new 'origin' (0, 0).
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [-3, 4]
final_pos: [4, 9]
final_ori: [0, -1]
room_id: 3
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'truck', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing left'}, {'name': 'rubberduck', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing backward'}, {'name': 'red door', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'on left wall'}]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_gt_cogmap
📊 Evaluation (use_model_cogmap)
📊 Task: DirectionEvaluationTask
Question 1 (ID: 9d13d2040c8f0e75)
✅ Evaluation Results
id: 9d13d2040c8f0e75
question: You return to your starting position and face north.
From a Top-Down map, describe where basket is relative to toilet.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: north west, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Question 2 (ID: e2300d26c5895d2d)
✅ Evaluation Results
id: e2300d26c5895d2d
question: You return to your starting position and face north.
From a Top-Down map, describe where rubberduck is relative to lego man.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: west, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Question 3 (ID: c2a21d8364e4dca4)
✅ Evaluation Results
id: c2a21d8364e4dca4
question: You return to your starting position and face north.
From a Top-Down map, describe where shoppingcart is relative to airplane.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: north west, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
📊 Task: PovEvaluationTask
Question 1 (ID: 063960e28e84e1b8)
✅ Evaluation Results
id: 063960e28e84e1b8
question: Now you jump to rubberduck's direction, facing its direction.
Describe where lamp is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Question 2 (ID: dbbb7b93fba9d779)
✅ Evaluation Results
id: dbbb7b93fba9d779
question: Now you jump to lego man's direction, facing its direction.
Describe where shoppingcart is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front-left, slightly far
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Question 3 (ID: b2e43f1e4b97a13e)
✅ Evaluation Results
id: b2e43f1e4b97a13e
question: Now you jump to shoppingcart's direction, facing its direction.
Describe where bike is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front-right, slightly far
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
📊 Task: BackwardPovTextEvaluationTask
Question 1 (ID: 4022a79578213eb8)
✅ Evaluation Results
id: 4022a79578213eb8
question: Now you jump to an object's position, facing its direction.
You observe that shoppingcart is front-left, slightly far, facing right; red door is front-right, mid distance, on right wall; backpack is front-slight-left, slightly far, facing forward.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: lego man
final_pos: [9, 8]
final_ori: [0, -1]
final_observation: [{'name': 'shoppingcart', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'red door', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'on right wall'}, {'name': 'backpack', 'direction': 'front-slight-left', 'distance': 'slightly far', 'orientation': 'facing forward'}]
object_positions:
bike: [9.0, 6.0]
airplane: [15.0, 2.0]
rubberduck: [1.0, 6.0]
lego man: [9.0, 8.0]
toilet: [19.0, 4.0]
truck: [5.0, 6.0]
backpack: [11.0, 3.0]
shoppingcart: [13.0, 3.0]
fridge: [17.0, 2.0]
soda can: [5.0, 8.0]
basket: [17.0, 6.0]
lamp: [1.0, 10.0]
red door: [7.0, 5.0]
green door: [14.0, 5.0]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
room_id: 1
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_model_cogmap
Question 2 (ID: d217d8691521707e)
✅ Evaluation Results
id: d217d8691521707e
question: Now you jump to an object's position, facing its direction.
You observe that red door is front-right, mid distance, on front wall.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: truck
final_pos: [5, 6]
final_ori: [1, 0]
final_observation: [{'name': 'red door', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'on front wall'}]
object_positions:
bike: [9.0, 6.0]
airplane: [15.0, 2.0]
rubberduck: [1.0, 6.0]
lego man: [9.0, 8.0]
toilet: [19.0, 4.0]
truck: [5.0, 6.0]
backpack: [11.0, 3.0]
shoppingcart: [13.0, 3.0]
fridge: [17.0, 2.0]
soda can: [5.0, 8.0]
basket: [17.0, 6.0]
lamp: [1.0, 10.0]
red door: [7.0, 5.0]
green door: [14.0, 5.0]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
room_id: 3
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_model_cogmap
Question 3 (ID: 7599c12dc388ac1a)
✅ Evaluation Results
id: 7599c12dc388ac1a
question: Now you jump to an object's position, facing its direction.
You observe that lamp is front, mid distance.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: rubberduck
final_pos: [1, 6]
final_ori: [0, 1]
final_observation: [{'name': 'lamp', 'direction': 'front', 'distance': 'mid distance', 'orientation': None}]
object_positions:
bike: [9.0, 6.0]
airplane: [15.0, 2.0]
rubberduck: [1.0, 6.0]
lego man: [9.0, 8.0]
toilet: [19.0, 4.0]
truck: [5.0, 6.0]
backpack: [11.0, 3.0]
shoppingcart: [13.0, 3.0]
fridge: [17.0, 2.0]
soda can: [5.0, 8.0]
basket: [17.0, 6.0]
lamp: [1.0, 10.0]
red door: [7.0, 5.0]
green door: [14.0, 5.0]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
room_id: 3
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_model_cogmap
📊 Task: Action2ViewEvaluationTask
Question 1 (ID: e5146a0f564c765f)
✅ Evaluation Results
id: e5146a0f564c765f
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Rotate(90)
2. Jump to the object at front, near.
3. Jump to the object at front-right, slightly far.
After executing the actions, what is the ego relation of toilet relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-left, mid distance
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Question 2 (ID: 90f3cb1a03f64a70)
✅ Evaluation Results
id: 90f3cb1a03f64a70
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Rotate(90)
2. Jump to the object at front, near.
3. Jump to the object at front-right, slightly far.
4. Rotate(-180)
After executing the actions, what is the ego relation of green door relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-right, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Question 3 (ID: 7b132df57af38164)
✅ Evaluation Results
id: 7b132df57af38164
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Jump to the object at front-left, slightly far.
2. Rotate(90)
3. Jump to the object at front-right, slightly far.
4. Rotate(90)
After executing the actions, what is the ego relation of fridge relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
📊 Task: View2ActionTextEvaluationTask
Question 1 (ID: a1b55f183063b5a2)
✅ Evaluation Results
id: a1b55f183063b5a2
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
truck is at front, near, facing left; red door is at front-left, mid distance, on left wall
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [5, 8]
final_ori: [0, -1]
room_id: 3
init_pos: [12, 5]
init_ori: [0, 1]
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', -90], ['jumpto', 'red door'], ['rotate', 90], ['jumpto', 'soda can'], ['rotate', 180]]
final_observation: [{'name': 'truck', 'direction': 'front', 'distance': 'near', 'orientation': 'facing left'}, {'name': 'red door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on left wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: True
final_pos: [1.0, 6.0]
final_ori: [0, -1]
visible_match: False
score: 0
evaluation_mode: use_model_cogmap
Question 2 (ID: 514e1fc99b9dfec8)
✅ Evaluation Results
id: 514e1fc99b9dfec8
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
basket is at front-right, slightly far; green door is at front-slight-left, mid distance, on left wall
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [15, 2]
final_ori: [0, 1]
room_id: 2
init_pos: [12, 5]
init_ori: [0, 1]
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', 90], ['jumpto', 'green door'], ['rotate', 90], ['jumpto', 'airplane'], ['rotate', 180]]
final_observation: [{'name': 'basket', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': None}, {'name': 'green door', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'on left wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [15.0, 2.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: use_model_cogmap
Question 3 (ID: bf8d3e056f97abbf)
✅ Evaluation Results
id: bf8d3e056f97abbf
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
fridge is at front-right, mid distance
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [19, 4]
final_ori: [0, -1]
room_id: 2
init_pos: [12, 5]
init_ori: [0, 1]
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', 90], ['jumpto', 'green door'], ['jumpto', 'toilet'], ['rotate', 90]]
final_observation: [{'name': 'fridge', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: False
final_pos: [19.0, 4.0]
final_ori: [0, 1]
visible_match: False
score: 0
evaluation_mode: use_model_cogmap
📊 Task: AlloMappingEvaluationTask
Question 1 (ID: 11808d88af254359)
✅ Evaluation Results
id: 11808d88af254359
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: basket, toilet, fridge.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[5, 1], [7, -1], [5, -3]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.955
threshold: 0.900
score: 0.955
evaluation_mode: use_model_cogmap
Question 2 (ID: 2b6ee8e5881229c1)
✅ Evaluation Results
id: 2b6ee8e5881229c1
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: rubberduck, lego man, basket, bike, airplane.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-11, 1], [-3, 3], [5, 1], [-3, 1], [3, -3]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.965
threshold: 0.900
score: 0.965
evaluation_mode: use_model_cogmap
Question 3 (ID: 6a481cb096f39335)
✅ Evaluation Results
id: 6a481cb096f39335
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: backpack, basket, lamp, airplane, rubberduck.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-1, -2], [5, 1], [-11, 5], [3, -3], [-11, 1]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.778
threshold: 0.900
score: 0.778
evaluation_mode: use_model_cogmap
📊 Task: RotEvaluationTask
Question 1 (ID: 37b10f92ed55a226)
✅ Evaluation Results
id: 37b10f92ed55a226
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning clockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: basket, fridge, shoppingcart.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['basket', 'fridge', 'shoppingcart']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Question 2 (ID: 4b2dc57e77686a8b)
✅ Evaluation Results
id: 4b2dc57e77686a8b
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning counterclockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: backpack, lego man, rubberduck.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['lego man', 'rubberduck', 'backpack']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Question 3 (ID: 3af660d5e0016740)
✅ Evaluation Results
id: 3af660d5e0016740
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning clockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: airplane, backpack, lamp, toilet.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['toilet', 'airplane', 'backpack', 'lamp']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
📊 Task: Location2ViewEvaluationTask
Question 1 (ID: 3af268195886a8df)
✅ Evaluation Results
id: 3af268195886a8df
question: Treat the green door as the new 'origin' (0, 0).
You move to (2, -2) and face east.
What is the egocentric relation of toilet?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front-slight-left, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Question 2 (ID: 56728a7c1f765ce3)
✅ Evaluation Results
id: 56728a7c1f765ce3
question: Treat the green door as the new 'origin' (0, 0).
You move to (2, -2) and face north.
What is the egocentric relation of basket?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front-slight-right, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Question 3 (ID: 843e8e1b199cd022)
✅ Evaluation Results
id: 843e8e1b199cd022
question: Treat the red door as the new 'origin' (0, 0).
You move to (-3, 4) and face south.
What is the egocentric relation of truck?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front-slight-left, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: use_model_cogmap
📊 Task: View2LocationTextEvaluationTask
Question 1 (ID: 0627604bb51f9403)
✅ Evaluation Results
id: 0627604bb51f9403
question: You move to a new location and face east.
You observe: toilet is at front-slight-left, mid distance, facing forward; fridge is at front-right, near
Treat the green door as the new 'origin' (0, 0).
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [2, -2]
final_pos: [16, 3]
final_ori: [1, 0]
room_id: 2
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'toilet', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'fridge', 'direction': 'front-right', 'distance': 'near', 'orientation': None}]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_model_cogmap
Question 2 (ID: 9d51c9ea1c03280c)
✅ Evaluation Results
id: 9d51c9ea1c03280c
question: You move to a new location and face north.
You observe: basket is at front-slight-right, mid distance; green door is at front-left, mid distance, on left wall
Treat the green door as the new 'origin' (0, 0).
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [2, -2]
final_pos: [16, 3]
final_ori: [0, 1]
room_id: 2
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'basket', 'direction': 'front-slight-right', 'distance': 'mid distance', 'orientation': None}, {'name': 'green door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on left wall'}]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_model_cogmap
Question 3 (ID: 231594f7927a5d40)
✅ Evaluation Results
id: 231594f7927a5d40
question: You move to a new location and face south.
You observe: truck is at front-slight-left, mid distance, facing left; rubberduck is at front-right, slightly far, facing backward; red door is at front-left, slightly far, on left wall
Treat the red door as the new 'origin' (0, 0).
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [-3, 4]
final_pos: [4, 9]
final_ori: [0, -1]
room_id: 3
object_positions:
bike: [9, 6]
airplane: [15, 2]
rubberduck: [1, 6]
lego man: [9, 8]
toilet: [19, 4]
truck: [5, 6]
backpack: [11, 3]
shoppingcart: [13, 3]
fridge: [17, 2]
soda can: [5, 8]
basket: [17, 6]
lamp: [1, 10]
red door: [7, 5]
green door: [14, 5]
object_orientations:
bike: [0, -1]
airplane: [0, -1]
rubberduck: [0, 1]
lego man: [0, -1]
toilet: [1, 0]
truck: [1, 0]
backpack: [0, -1]
shoppingcart: [-1, 0]
red door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'truck', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing left'}, {'name': 'rubberduck', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing backward'}, {'name': 'red door', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'on left wall'}]
gate_info:
red door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
bike: 1
airplane: 2
rubberduck: 3
lego man: 1
toilet: 2
truck: 3
backpack: 1
shoppingcart: 1
fridge: 2
soda can: 3
basket: 2
lamp: 3
red door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.819
raw_score: 0.819
score: 0.819
evaluation_mode: use_model_cogmap