sample_run03
Select Configuration:
📊 Sample Metrics
🔍 Exploration
last_node_coverage: 1
last_edge_coverage: 0.512
n_exploration_steps: 10
action_counts:
move: 2
rotate: 6
return: 0
observe: 9
term: 1
forced_term: 0
query: 0
action_cost: 9
final_information_gain: 0.910
action_fail_ratio: 0
valid_action_ratio: 1
false_belief_steps: 6
false_belief_f1: 1
false_belief_f1_position: 1
false_belief_f1_facing: 1
false_belief_action_cost: 5
false_belief_action_cost_after_seen: 0
✅ Evaluation
overall:
n_total: 27
total_score: 17.634
avg_accuracy: 0.653
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
PovEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
Action2ViewEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
View2ActionTextEvaluationTask:
n_total: 3
task_score: 0
avg_accuracy: 0
AlloMappingEvaluationTask:
n_total: 3
task_score: 2.627
avg_accuracy: 0.876
RotEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
Location2ViewEvaluationTask:
n_total: 3
task_score: 1.500
avg_accuracy: 0.500
View2LocationTextEvaluationTask:
n_total: 3
task_score: 2.007
avg_accuracy: 0.669
✅ Evaluation (prompt_cogmap)
overall:
n_total: 27
total_score: 18.944
avg_accuracy: 0.702
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 1.500
avg_accuracy: 0.500
PovEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Action2ViewEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
View2ActionTextEvaluationTask:
n_total: 3
task_score: 1
avg_accuracy: 0.333
AlloMappingEvaluationTask:
n_total: 3
task_score: 2.167
avg_accuracy: 0.722
RotEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Location2ViewEvaluationTask:
n_total: 3
task_score: 1.500
avg_accuracy: 0.500
View2LocationTextEvaluationTask:
n_total: 3
task_score: 1.777
avg_accuracy: 0.592
✅ Evaluation (use_gt_cogmap)
overall:
n_total: 27
total_score: 26.639
avg_accuracy: 0.987
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
PovEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Action2ViewEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
View2ActionTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
AlloMappingEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
RotEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Location2ViewEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
View2LocationTextEvaluationTask:
n_total: 3
task_score: 2.639
avg_accuracy: 0.880
✅ Evaluation (use_model_cogmap)
overall:
n_total: 27
total_score: 20.119
avg_accuracy: 0.745
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
PovEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Action2ViewEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
View2ActionTextEvaluationTask:
n_total: 3
task_score: 0
avg_accuracy: 0
AlloMappingEvaluationTask:
n_total: 3
task_score: 2.479
avg_accuracy: 0.826
RotEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Location2ViewEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
View2LocationTextEvaluationTask:
n_total: 3
task_score: 2.639
avg_accuracy: 0.880
🧠 Cognitive Map
exploration:
error:
local_vs_gt_local_avg:
dir: 0.704
facing: 1
pos: 0.867
overall: 0.857
global_vs_gt_global_avg:
dir: 0.796
facing: 0.967
pos: 0.768
overall: 0.844
agent_vs_gt_agent_avg:
dir: 0.500
facing: 1
pos: 0.810
overall: 0.770
newly_observed_vs_gt_local_avg:
dir: 0.789
facing: 1
pos: 0.868
overall: 0.886
correctness:
last_global_vs_gt_full:
dir: 0.867
facing: 1
pos: 0.836
overall: 0.901
consistency:
local_vs_global_avg:
dir: 0.633
facing: 1
pos: 0.823
overall: 0.819
position_update_avg: 0.758
facing_update_avg: 0.958
position_stability_avg: 0.758
facing_stability_avg: 0.958
fog_probe:
f1_avg: 0.632
precision_avg: 0.644
recall_avg: 0.741
cogmap_fb:
metrics:
changed:
dir: None
pos: 0.632
facing: 1
overall: None
retention:
dir: None
pos: 0.264
facing: 0
overall: None
unchanged:
dir: 0.706
facing: 0.967
pos: 0.713
overall: 0.795
inertia: -0.002
inertia_list: [-0.004156680293742778, -0.0002946320464591308]
unchanged_retention:
dir: None
pos: 0.751
facing: 1
overall: None
unchanged_retention_minus_retention:
pos: 0.876
facing: 1
unchanged_exploration:
dir: 0.836
facing: 1
pos: 0.841
overall: 0.893
🧭 False Belief CogMap
inertia: -0.002
changed:
dir: None
pos: 0.632
facing: 1
overall: None
retention:
dir: None
pos: 0.264
facing: 0
overall: None
unchanged:
dir: 0.706
facing: 0.967
pos: 0.713
overall: 0.795
unchanged_retention:
dir: None
pos: 0.751
facing: 1
overall: None
unchanged_retention_minus_retention:
pos: 0.876
facing: 1
unchanged_exploration:
dir: 0.836
facing: 1
pos: 0.841
overall: 0.893
📊 Performance Charts
Cognitive Map (Update)
Cognitive Map (Full)
Cognitive Map (Self-Tracking)
FB CogMap (Unchanged)
Fog Probe F1
Fog Probe Precision
Fog Probe Recall
Position Update
Facing Update
Position Stability
Facing Stability
🌍 Exploration Phase
🔄 Turn 1
💬 Assistant Action
Actions: [Observe()]
Actions: [Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
0,
0
],
"facing": "north"
},
"teddybear": {
"position": [
2,
3
],
"facing": "east"
},
"bike": {
"position": [
2,
5
],
"facing": "east"
},
"vase": {
"position": [
-2,
3
]
}
}
🎯 Ground Truth (Observed)
{
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"agent": {
"position": [
0,
0
],
"facing": "north"
}
}
🎯 Ground Truth (Full)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"chair": {
"position": [
-5,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"desklamp": {
"position": [
-7,
2
],
"facing": "west"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"plant": {
"position": [
-9,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"basket": {
"position": [
-5,
4
],
"facing": "east"
},
"yellow door": {
"position": [
-3,
0
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
0,
0
],
"facing": "north"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"teddybear": {
"position": [
2,
3
],
"facing": "+x"
},
"bike": {
"position": [
1,
4
],
"facing": "+x"
},
"vase": {
"position": [
-2,
3
]
}
}
}
🎯 Ground Truth
{
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"teddybear": {
"position": [
2,
3
],
"facing": "+x"
},
"bike": {
"position": [
1,
4
],
"facing": "+x"
},
"vase": {
"position": [
-2,
3
]
}
}
}
🎯 Ground Truth
{
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
}
}
🗺️ Symbolic Fog Map
###################### #......############### #......#......#......# #......#..E...+......# #......#..F...#......# #......#A.B..C#......# #......+..*...#......# ########.....D#......# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: D
🎯 Ground Truth: D, A, C
🧠 Cognitive Map Metrics
Global:
dir: 1
facing: 1
pos: 0.844
overall: 0.948
Global (Full):
dir: 0.083
facing: 0.333
pos: 0.225
overall: 0.214
Local:
dir: 0.667
facing: 1
pos: 0.859
overall: 0.842
Local (Newly):
dir: 0.667
facing: 1
pos: 0.859
overall: 0.842
Fog Probe:
precision: 1
recall: 0.333
overall: 0.500
📈 Turn Metrics
node_coverage: 0.214
edge_coverage: 0.073
is_action_fail: False
step: 1
action_counts:
move: 0
rotate: 0
return: 0
observe: 1
term: 0
forced_term: 0
query: 0
information_gain: 0.186
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 2
💬 Assistant Action
Actions: [Rotate(90), Observe()]
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
0,
0
],
"facing": "east"
},
"teddybear": {
"position": [
2,
3
],
"facing": "east"
},
"bike": {
"position": [
2,
5
],
"facing": "east"
},
"vase": {
"position": [
-2,
3
]
},
"television": {
"position": [
3,
0
],
"facing": "south"
},
"green door": {
"position": [
5,
3
]
}
}
🎯 Ground Truth (Observed)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
0,
0
],
"facing": "east"
}
}
🎯 Ground Truth (Full)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"chair": {
"position": [
-5,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"desklamp": {
"position": [
-7,
2
],
"facing": "west"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"plant": {
"position": [
-9,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"basket": {
"position": [
-5,
4
],
"facing": "east"
},
"yellow door": {
"position": [
-3,
0
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
0,
0
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"television": {
"position": [
0,
3
],
"facing": "+x"
},
"green door": {
"position": [
-2,
6
]
}
}
}
🎯 Ground Truth
{
"television": {
"position": [
0,
3
],
"facing": "south"
},
"green door": {
"position": [
-3,
4
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"television": {
"position": [
0,
3
],
"facing": "+x"
},
"green door": {
"position": [
-2,
6
]
}
}
}
🎯 Ground Truth
{
"television": {
"position": [
0,
3
],
"facing": "south"
},
"green door": {
"position": [
-3,
4
],
"facing": "east"
}
}
🗺️ Symbolic Fog Map
###################### #......############### #......#......#......# #......#......+......# #......#.E....#......# #......#BFD...#......# #......+.C*...#......# ########..A...#......# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: A, C
🎯 Ground Truth: B, C, A
🧠 Cognitive Map Metrics
Global:
dir: 0.762
facing: 1
pos: 0.859
overall: 0.874
Global (Full):
dir: 0.133
facing: 0.444
pos: 0.344
overall: 0.307
Local:
dir: 0.333
facing: 1
pos: 0.787
overall: 0.707
Local (Newly):
dir: 0.333
facing: 1
pos: 0.787
overall: 0.707
Fog Probe:
precision: 1
recall: 0.667
overall: 0.800
📈 Turn Metrics
node_coverage: 0.357
edge_coverage: 0.122
is_action_fail: False
step: 2
action_counts:
move: 0
rotate: 1
return: 0
observe: 2
term: 0
forced_term: 0
query: 0
information_gain: 0.293
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 3
💬 Assistant Action
Actions: [JumpTo(green door), Observe()]
Actions: [JumpTo(green door), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
5,
3
],
"facing": "east"
},
"teddybear": {
"position": [
2,
3
],
"facing": "east"
},
"bike": {
"position": [
2,
5
],
"facing": "east"
},
"vase": {
"position": [
-2,
3
]
},
"television": {
"position": [
3,
0
],
"facing": "south"
},
"green door": {
"position": [
5,
3
]
},
"shoppingcart": {
"position": [
9,
1
],
"facing": "south"
},
"scooter": {
"position": [
8,
2
],
"facing": "east"
},
"shoes": {
"position": [
11,
0
],
"facing": "east"
},
"shelf": {
"position": [
10,
5
]
}
}
🎯 Ground Truth (Observed)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
4,
3
],
"facing": "east"
}
}
🎯 Ground Truth (Full)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"chair": {
"position": [
-5,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"desklamp": {
"position": [
-7,
2
],
"facing": "west"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"plant": {
"position": [
-9,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"basket": {
"position": [
-5,
4
],
"facing": "east"
},
"yellow door": {
"position": [
-3,
0
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
4,
3
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"scooter": {
"position": [
1,
3
],
"facing": "+y"
},
"shoppingcart": {
"position": [
3,
3
],
"facing": "+x"
},
"shoes": {
"position": [
3,
5
],
"facing": "+y"
},
"shelf": {
"position": [
-1,
4
]
}
}
}
🎯 Ground Truth
{
"shoppingcart": {
"position": [
3,
3
],
"facing": "south"
},
"shelf": {
"position": [
-1,
5
],
"facing": "east"
},
"shoes": {
"position": [
3,
5
],
"facing": "east"
},
"scooter": {
"position": [
1,
3
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"scooter": {
"position": [
1,
3
],
"facing": "+y"
},
"shoppingcart": {
"position": [
3,
3
],
"facing": "+x"
},
"shoes": {
"position": [
3,
5
],
"facing": "+y"
},
"shelf": {
"position": [
-1,
4
]
}
}
}
🎯 Ground Truth
{
"shoppingcart": {
"position": [
3,
3
],
"facing": "south"
},
"shelf": {
"position": [
-1,
5
],
"facing": "east"
},
"shoes": {
"position": [
3,
5
],
"facing": "east"
},
"scooter": {
"position": [
1,
3
],
"facing": "east"
}
}
🗺️ Symbolic Fog Map
###################### #......############### #......#......#......# #......#....F.*.....E# #......#....L.#.....J# #......#AG....#......# #......+......#......# ########.BC...#KHI.D.# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: A, B, C, G
🎯 Ground Truth: C, B, A, I, K, H
🧠 Cognitive Map Metrics
Global:
dir: 0.782
facing: 1
pos: 0.812
overall: 0.864
Global (Full):
dir: 0.358
facing: 0.778
pos: 0.541
overall: 0.559
Local:
dir: 0.900
facing: 1
pos: 0.927
overall: 0.942
Local (Newly):
dir: 0.900
facing: 1
pos: 0.927
overall: 0.942
Fog Probe:
precision: 0.750
recall: 0.500
overall: 0.600
📈 Turn Metrics
node_coverage: 0.643
edge_coverage: 0.220
is_action_fail: False
step: 3
action_counts:
move: 1
rotate: 1
return: 0
observe: 3
term: 0
forced_term: 0
query: 0
information_gain: 0.450
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 4
💬 Assistant Action
Actions: [Rotate(180), Observe()]
Actions: [Rotate(180), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
4,
3
],
"facing": "west"
},
"green door": {
"position": [
4,
3
]
},
"yellow door": {
"position": [
0,
0
]
},
"television": {
"position": [
3,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
3
],
"facing": "east"
},
"bike": {
"position": [
2,
4
],
"facing": "east"
},
"vase": {
"position": [
-1,
2
]
},
"shoppingcart": {
"position": [
8,
1
],
"facing": "south"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"shoes": {
"position": [
8,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
]
}
}
🎯 Ground Truth (Observed)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"yellow door": {
"position": [
-3,
0
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
4,
3
],
"facing": "west"
}
}
🎯 Ground Truth (Full)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"chair": {
"position": [
-5,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"desklamp": {
"position": [
-7,
2
],
"facing": "west"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"plant": {
"position": [
-9,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"basket": {
"position": [
-5,
4
],
"facing": "east"
},
"yellow door": {
"position": [
-3,
0
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
4,
3
],
"facing": "west"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"teddybear": {
"position": [
-1,
3
],
"facing": "-y"
},
"bike": {
"position": [
1,
3
],
"facing": "-y"
},
"vase": {
"position": [
-1,
6
]
},
"yellow door": {
"position": [
-3,
6
]
}
}
}
🎯 Ground Truth
{
"yellow door": {
"position": [
-3,
7
],
"facing": "east"
},
"bike": {
"position": [
1,
3
],
"facing": "east"
},
"teddybear": {
"position": [
-1,
3
],
"facing": "east"
},
"vase": {
"position": [
-1,
6
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"teddybear": {
"position": [
-1,
3
],
"facing": "-y"
},
"bike": {
"position": [
1,
3
],
"facing": "-y"
},
"vase": {
"position": [
-1,
6
]
},
"yellow door": {
"position": [
-3,
6
]
}
}
}
🎯 Ground Truth
{
"yellow door": {
"position": [
-3,
7
],
"facing": "east"
}
}
🗺️ Symbolic Fog Map
###################### #......############### #......#......#......# #......#......*B.....# #......#......#E.....# #......#......#......# #......+......#......# ########......#DCAF..# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: A, C, D, E, F
🎯 Ground Truth: A, C, D
🧠 Cognitive Map Metrics
Global:
dir: 0.697
facing: 1
pos: 0.819
overall: 0.839
Global (Full):
dir: 0.383
facing: 0.778
pos: 0.601
overall: 0.587
Local:
dir: 0.900
facing: 1
pos: 0.927
overall: 0.942
Local (Newly):
dir: 1
facing: 1
pos: 0.859
overall: 0.953
Fog Probe:
precision: 0.600
recall: 1
overall: 0.750
📈 Turn Metrics
node_coverage: 0.714
edge_coverage: 0.317
is_action_fail: False
step: 4
action_counts:
move: 1
rotate: 2
return: 0
observe: 4
term: 0
forced_term: 0
query: 0
information_gain: 0.578
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 5
💬 Assistant Action
Actions: [JumpTo(yellow door), Observe()]
Actions: [JumpTo(yellow door), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
2,
1
],
"facing": "west"
},
"green door": {
"position": [
6,
3
]
},
"yellow door": {
"position": [
2,
1
]
},
"television": {
"position": [
3,
0
],
"facing": "south"
},
"teddybear": {
"position": [
2,
3
],
"facing": "east"
},
"bike": {
"position": [
2,
5
],
"facing": "east"
},
"vase": {
"position": [
-1,
2
]
},
"shoppingcart": {
"position": [
11,
0
],
"facing": "south"
},
"scooter": {
"position": [
9,
0
],
"facing": "east"
},
"shoes": {
"position": [
11,
2
],
"facing": "east"
},
"shelf": {
"position": [
11,
4
]
},
"chair": {
"position": [
1,
1
],
"facing": "south"
},
"desklamp": {
"position": [
-2,
3
],
"facing": "west"
},
"plant": {
"position": [
-3,
3
]
}
}
🎯 Ground Truth (Observed)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"chair": {
"position": [
-5,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"desklamp": {
"position": [
-7,
2
],
"facing": "west"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"plant": {
"position": [
-9,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"yellow door": {
"position": [
-3,
0
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
-3,
0
],
"facing": "west"
}
}
🎯 Ground Truth (Full)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"chair": {
"position": [
-5,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"desklamp": {
"position": [
-7,
2
],
"facing": "west"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"plant": {
"position": [
-9,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"basket": {
"position": [
-5,
4
],
"facing": "east"
},
"yellow door": {
"position": [
-3,
0
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
-3,
0
],
"facing": "west"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"chair": {
"position": [
0,
1
],
"facing": "-x"
},
"desklamp": {
"position": [
2,
4
],
"facing": "+y"
},
"plant": {
"position": [
2,
6
]
}
}
}
🎯 Ground Truth
{
"plant": {
"position": [
2,
6
],
"facing": "east"
},
"chair": {
"position": [
0,
2
],
"facing": "south"
},
"desklamp": {
"position": [
2,
4
],
"facing": "west"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"chair": {
"position": [
0,
1
],
"facing": "-x"
},
"desklamp": {
"position": [
2,
4
],
"facing": "+y"
},
"plant": {
"position": [
2,
6
]
}
}
}
🎯 Ground Truth
{
"plant": {
"position": [
2,
6
],
"facing": "east"
},
"chair": {
"position": [
0,
2
],
"facing": "south"
},
"desklamp": {
"position": [
2,
4
],
"facing": "west"
}
}
🗺️ Symbolic Fog Map
###################### #....D.############### #......#......#......# #....B.#......+...K..# #G....J#......#...F..# #.E....#......#L..H..# #....A.*......#CI....# ########......#......# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: A, B, C, D, E, F, G, H, I, J, K, L
🎯 Ground Truth: C, I, L, D, B, J
🧠 Cognitive Map Metrics
Global:
dir: 0.638
facing: 1
pos: 0.567
overall: 0.735
Global (Full):
dir: 0.558
facing: 1
pos: 0.529
overall: 0.696
Local:
dir: 0.833
facing: 1
pos: 0.916
overall: 0.917
Local (Newly):
dir: 0.833
facing: 1
pos: 0.916
overall: 0.917
Fog Probe:
precision: 0.500
recall: 1
overall: 0.667
📈 Turn Metrics
node_coverage: 0.929
edge_coverage: 0.390
is_action_fail: False
step: 5
action_counts:
move: 2
rotate: 2
return: 0
observe: 5
term: 0
forced_term: 0
query: 0
information_gain: 0.682
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 6
💬 Assistant Action
Actions: [Rotate(180), Observe()]
Actions: [Rotate(180), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
-2,
1
],
"facing": "east"
},
"green door": {
"position": [
5,
4
]
},
"yellow door": {
"position": [
-2,
1
]
},
"television": {
"position": [
4,
0
],
"facing": "south"
},
"teddybear": {
"position": [
2,
3
],
"facing": "east"
},
"bike": {
"position": [
2,
5
],
"facing": "east"
},
"vase": {
"position": [
-2,
3
]
},
"shoppingcart": {
"position": [
8,
1
],
"facing": "south"
},
"scooter": {
"position": [
8,
3
],
"facing": "east"
},
"shoes": {
"position": [
10,
1
],
"facing": "east"
},
"shelf": {
"position": [
9,
5
]
},
"chair": {
"position": [
-3,
1
],
"facing": "south"
},
"desklamp": {
"position": [
-5,
4
],
"facing": "west"
},
"plant": {
"position": [
-6,
4
]
}
}
🎯 Ground Truth (Observed)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"chair": {
"position": [
-5,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"desklamp": {
"position": [
-7,
2
],
"facing": "west"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"plant": {
"position": [
-9,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"yellow door": {
"position": [
-3,
0
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
-3,
0
],
"facing": "east"
}
}
🎯 Ground Truth (Full)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"chair": {
"position": [
-5,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"desklamp": {
"position": [
-7,
2
],
"facing": "west"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"plant": {
"position": [
-9,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"basket": {
"position": [
-5,
4
],
"facing": "east"
},
"yellow door": {
"position": [
-3,
0
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
-3,
0
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"television": {
"position": [
0,
6
],
"facing": "+x"
},
"teddybear": {
"position": [
-1,
6
],
"facing": "+y"
},
"bike": {
"position": [
-3,
6
],
"facing": "+y"
},
"green door": {
"position": [
-5,
6
]
}
}
}
🎯 Ground Truth
{
"bike": {
"position": [
-4,
4
],
"facing": "east"
},
"teddybear": {
"position": [
-2,
4
],
"facing": "east"
},
"television": {
"position": [
0,
6
],
"facing": "south"
},
"green door": {
"position": [
-3,
7
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"television": {
"position": [
0,
6
],
"facing": "+x"
},
"teddybear": {
"position": [
-1,
6
],
"facing": "+y"
},
"bike": {
"position": [
-3,
6
],
"facing": "+y"
},
"green door": {
"position": [
-5,
6
]
}
}
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
###################### #..D..L############### #.H....#......#F...G.# #...IB.#......+......# #.A....#......#......# #......#......#J.....# #......*......#..C...# ########......#.KE...# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: C, E, F, G, J, K
🎯 Ground Truth: J, K, E, L, D, B
🧠 Cognitive Map Metrics
Global:
dir: 0.886
facing: 1
pos: 0.762
overall: 0.883
Global (Full):
dir: 0.775
facing: 1
pos: 0.711
overall: 0.829
Local:
dir: 0.500
facing: 1
pos: 0.746
overall: 0.749
Local (Newly):
(none)
Fog Probe:
precision: 0.500
recall: 0.500
overall: 0.500
📈 Turn Metrics
node_coverage: 0.929
edge_coverage: 0.463
is_action_fail: False
step: 6
action_counts:
move: 2
rotate: 3
return: 0
observe: 6
term: 0
forced_term: 0
query: 0
information_gain: 0.866
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 7
💬 Assistant Action
Actions: [Rotate(90), Observe()]
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
-3,
0
],
"facing": "south"
},
"green door": {
"position": [
4,
3
]
},
"yellow door": {
"position": [
-3,
0
]
},
"television": {
"position": [
4,
0
],
"facing": "south"
},
"vase": {
"position": [
-2,
3
]
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"shelf": {
"position": [
8,
4
]
},
"chair": {
"position": [
-4,
0
],
"facing": "south"
},
"desklamp": {
"position": [
-7,
2
],
"facing": "west"
},
"plant": {
"position": [
-8,
2
]
}
}
🎯 Ground Truth (Observed)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"chair": {
"position": [
-5,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"desklamp": {
"position": [
-7,
2
],
"facing": "west"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"plant": {
"position": [
-9,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"yellow door": {
"position": [
-3,
0
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
-3,
0
],
"facing": "south"
}
}
🎯 Ground Truth (Full)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"chair": {
"position": [
-5,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"desklamp": {
"position": [
-7,
2
],
"facing": "west"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"plant": {
"position": [
-9,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"basket": {
"position": [
-5,
4
],
"facing": "east"
},
"yellow door": {
"position": [
-3,
0
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
-3,
0
],
"facing": "south"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {}
}
🎯 Ground Truth
(no data)
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {}
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
###################### #......############### #...H.B#......#..L.EI# #...GC.#......+......# #.A....#......#......# #......#......#......# #....D.*......#.F....# ########......#.KJ...# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: A, B, C, E, F, G, H, I, J, K, L
🎯 Ground Truth: J, F, K, B, C, H
🧠 Cognitive Map Metrics
Global:
dir: 0.952
facing: 1
pos: 0.913
overall: 0.955
Global (Full):
dir: 0.833
facing: 1
pos: 0.853
overall: 0.895
Local:
(none)
Local (Newly):
(none)
Fog Probe:
precision: 0.545
recall: 1
overall: 0.706
📈 Turn Metrics
node_coverage: 0.929
edge_coverage: 0.463
is_action_fail: False
step: 7
action_counts:
move: 2
rotate: 4
return: 0
observe: 7
term: 0
forced_term: 0
query: 0
information_gain: 0.866
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 8
💬 Assistant Action
Actions: [Rotate(90), Observe()]
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
0,
-1
],
"facing": "west"
},
"green door": {
"position": [
6,
4
]
},
"yellow door": {
"position": [
0,
-1
]
},
"teddybear": {
"position": [
2,
3
],
"facing": "east"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
]
},
"television": {
"position": [
4,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
10,
3
],
"facing": "south"
},
"scooter": {
"position": [
9,
3
],
"facing": "east"
},
"shoes": {
"position": [
10,
2
],
"facing": "east"
},
"shelf": {
"position": [
10,
6
]
},
"chair": {
"position": [
-1,
-1
],
"facing": "south"
},
"desklamp": {
"position": [
-4,
1
],
"facing": "west"
},
"plant": {
"position": [
-4,
2
]
}
}
🎯 Ground Truth (Observed)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"chair": {
"position": [
-5,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"desklamp": {
"position": [
-7,
2
],
"facing": "west"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"plant": {
"position": [
-9,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"yellow door": {
"position": [
-3,
0
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
-3,
0
],
"facing": "west"
}
}
🎯 Ground Truth (Full)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"chair": {
"position": [
-5,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"desklamp": {
"position": [
-7,
2
],
"facing": "west"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"plant": {
"position": [
-9,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"basket": {
"position": [
-5,
4
],
"facing": "east"
},
"yellow door": {
"position": [
-3,
0
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
-3,
0
],
"facing": "west"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"chair": {
"position": [
0,
1
],
"facing": "-x"
},
"desklamp": {
"position": [
2,
4
],
"facing": "+y"
},
"plant": {
"position": [
2,
6
]
}
}
}
🎯 Ground Truth
{
"plant": {
"position": [
2,
6
],
"facing": "east"
},
"chair": {
"position": [
0,
2
],
"facing": "south"
},
"desklamp": {
"position": [
2,
4
],
"facing": "west"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"chair": {
"position": [
0,
1
],
"facing": "-x"
},
"desklamp": {
"position": [
2,
4
],
"facing": "+y"
},
"plant": {
"position": [
2,
6
]
}
}
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
###################### #.....C############### #...L..#......#K....F# #...A..#......+......# #.E...G#......#......# #......#......#.I....# #...J..*......#B.....# ########......#.HD...# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: A, B, C, E, F, G, I, J, K, L
🎯 Ground Truth: H, B, D, G, C, L
🧠 Cognitive Map Metrics
Global:
dir: 0.686
facing: 1
pos: 0.650
overall: 0.779
Global (Full):
dir: 0.600
facing: 1
pos: 0.607
overall: 0.736
Local:
dir: 0.833
facing: 1
pos: 0.916
overall: 0.917
Local (Newly):
(none)
Fog Probe:
precision: 0.400
recall: 0.667
overall: 0.500
📈 Turn Metrics
node_coverage: 0.929
edge_coverage: 0.463
is_action_fail: False
step: 8
action_counts:
move: 2
rotate: 5
return: 0
observe: 8
term: 0
forced_term: 0
query: 0
information_gain: 0.866
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 9
💬 Assistant Action
Actions: [Rotate(90), Observe()]
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
1,
0
],
"facing": "north"
},
"teddybear": {
"position": [
2,
3
],
"facing": "east"
},
"bike": {
"position": [
2,
5
],
"facing": "east"
},
"vase": {
"position": [
-1,
2
]
},
"television": {
"position": [
3,
0
],
"facing": "east"
},
"green door": {
"position": [
5,
4
]
},
"shoppingcart": {
"position": [
8,
2
],
"facing": "east"
},
"scooter": {
"position": [
8,
3
],
"facing": "east"
},
"shoes": {
"position": [
9,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
5
]
},
"yellow door": {
"position": [
1,
0
]
},
"chair": {
"position": [
0,
0
],
"facing": "west"
},
"desklamp": {
"position": [
-3,
3
],
"facing": "west"
},
"plant": {
"position": [
-2,
3
]
},
"basket": {
"position": [
-2,
5
]
}
}
🎯 Ground Truth (Observed)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"chair": {
"position": [
-5,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"desklamp": {
"position": [
-7,
2
],
"facing": "west"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"plant": {
"position": [
-9,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"basket": {
"position": [
-5,
4
],
"facing": "east"
},
"yellow door": {
"position": [
-3,
0
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
-3,
0
],
"facing": "north"
}
}
🎯 Ground Truth (Full)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"chair": {
"position": [
-5,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"desklamp": {
"position": [
-7,
2
],
"facing": "west"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"plant": {
"position": [
-9,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"basket": {
"position": [
-5,
4
],
"facing": "east"
},
"yellow door": {
"position": [
-3,
0
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
-3,
0
],
"facing": "north"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"bike": {
"position": [
3,
4
],
"facing": "+x"
},
"vase": {
"position": [
1,
3
]
},
"basket": {
"position": [
-3,
4
]
}
}
}
🎯 Ground Truth
{
"bike": {
"position": [
4,
4
],
"facing": "east"
},
"basket": {
"position": [
-2,
4
],
"facing": "east"
},
"vase": {
"position": [
1,
2
],
"facing": "east"
}
}
📊 Cognitive Map JSONs
🤖 Predicted
{
"origin": "agent",
"objects": {
"bike": {
"position": [
3,
4
],
"facing": "+x"
},
"vase": {
"position": [
1,
3
]
},
"basket": {
"position": [
-3,
4
]
}
}
}
🎯 Ground Truth
{
"basket": {
"position": [
-2,
4
],
"facing": "east"
}
}
🗺️ Symbolic Fog Map
###################### #......############### #......#......#...D..# #......#......+......# #......#......#......# #......#......#..F...# #......*......#A.....# ########......#EB...C# ######################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: A, B, C, D, E, F
🎯 Ground Truth: A, B, E
🧠 Cognitive Map Metrics
Global:
dir: 0.692
facing: 0.667
pos: 0.616
overall: 0.658
Global (Full):
dir: 0.692
facing: 0.667
pos: 0.616
overall: 0.658
Local:
dir: 0.667
facing: 1
pos: 0.859
overall: 0.842
Local (Newly):
dir: 1
facing: 1
pos: 0.859
overall: 0.953
Fog Probe:
precision: 0.500
recall: 1
overall: 0.667
📈 Turn Metrics
node_coverage: 1
edge_coverage: 0.512
is_action_fail: False
step: 9
action_counts:
move: 2
rotate: 6
return: 0
observe: 9
term: 0
forced_term: 0
query: 0
information_gain: 0.910
reward: -0.100
is_done: False
is_valid_action: True
🔄 Turn 10
💬 Assistant Action
Actions: [Term()]
Actions: [Term()]
📊 Cognitive Map JSONs
🤖 Predicted
{
"agent": {
"position": [
-2,
1
],
"facing": "north"
},
"yellow door": {
"position": [
-2,
1
]
},
"green door": {
"position": [
4,
3
]
},
"television": {
"position": [
3,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-1,
2
]
},
"shoppingcart": {
"position": [
8,
1
],
"facing": "south"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"shoes": {
"position": [
10,
1
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
]
},
"chair": {
"position": [
-4,
1
],
"facing": "south"
},
"desklamp": {
"position": [
-7,
4
],
"facing": "west"
},
"plant": {
"position": [
-8,
3
]
},
"basket": {
"position": [
-5,
6
]
}
}
🎯 Ground Truth (Observed)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"chair": {
"position": [
-5,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"desklamp": {
"position": [
-7,
2
],
"facing": "west"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"plant": {
"position": [
-9,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"basket": {
"position": [
-5,
4
],
"facing": "east"
},
"yellow door": {
"position": [
-3,
0
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
-3,
0
],
"facing": "north"
}
}
🎯 Ground Truth (Full)
{
"television": {
"position": [
3,
0
],
"facing": "south"
},
"shoppingcart": {
"position": [
7,
0
],
"facing": "south"
},
"chair": {
"position": [
-5,
0
],
"facing": "south"
},
"teddybear": {
"position": [
1,
2
],
"facing": "east"
},
"scooter": {
"position": [
7,
2
],
"facing": "east"
},
"desklamp": {
"position": [
-7,
2
],
"facing": "west"
},
"bike": {
"position": [
1,
4
],
"facing": "east"
},
"vase": {
"position": [
-2,
2
],
"facing": "east"
},
"shoes": {
"position": [
9,
0
],
"facing": "east"
},
"plant": {
"position": [
-9,
2
],
"facing": "east"
},
"shelf": {
"position": [
9,
4
],
"facing": "east"
},
"basket": {
"position": [
-5,
4
],
"facing": "east"
},
"yellow door": {
"position": [
-3,
0
],
"facing": "east"
},
"green door": {
"position": [
4,
3
],
"facing": "east"
},
"agent": {
"position": [
-3,
0
],
"facing": "north"
}
}
🧠 Cognitive Map Metrics
Global:
dir: 0.867
facing: 1
pos: 0.836
overall: 0.901
Global (Full):
dir: 0.867
facing: 1
pos: 0.836
overall: 0.901
Local:
(none)
Local (Newly):
(none)
Fog Probe:
(none)
📈 Turn Metrics
node_coverage: 1
edge_coverage: 0.512
is_action_fail: False
step: 10
action_counts:
move: 2
rotate: 6
return: 0
observe: 9
term: 1
forced_term: 0
query: 0
information_gain: 0.910
reward: -0.100
is_done: True
is_valid_action: True
🧭 False Belief Exploration
🔄 FB Turn 11
💬 Assistant Action
Actions: [Observe()]
Actions: [Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Changed: plant:
dir: None
pos: 0.739
facing: None
overall: None
Retention: plant:
dir: None
pos: 0.316
facing: None
overall: None
Unchanged (all):
dir: 0.864
facing: 0.833
pos: 0.819
overall: 0.839
🧾 False Belief Object Sets
all_objects: ['basket', 'bike', 'chair', 'desklamp', 'green door', 'plant', 'scooter', 'shelf', 'shoes', 'shoppingcart', 'teddybear', 'television', 'vase', 'yellow door']
all_changed_objects: ['bike', 'desklamp', 'plant', 'television']
newly_observed_changed (this turn): ['plant']
unchanged_objects: ['basket', 'chair', 'green door', 'scooter', 'shelf', 'shoes', 'shoppingcart', 'teddybear', 'vase', 'yellow door']
pred_changed_keys: ['plant']
pred_unchanged_keys: ['basket', 'bike', 'chair', 'desklamp', 'green door', 'plant', 'scooter', 'shelf', 'shoes', 'shoppingcart', 'teddybear', 'television', 'vase', 'yellow door']
🧭 False Belief Info
step: 1
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'television', 'ori': True}, {'name': 'desklamp', 'ori': True}, {'name': 'plant', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: ['plant']
newly_observed_unchanged_objects: ['vase', 'teddybear']
🔄 FB Turn 12
💬 Assistant Action
Actions: [Rotate(90), Observe()]
Actions: [Rotate(90), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Changed: television:
dir: None
pos: None
facing: 1
overall: None
Retention: television:
dir: None
pos: None
facing: 0
overall: None
Unchanged (all):
dir: 0.667
facing: 1
pos: 0.719
overall: 0.795
🧾 False Belief Object Sets
all_objects: ['basket', 'bike', 'chair', 'desklamp', 'green door', 'plant', 'scooter', 'shelf', 'shoes', 'shoppingcart', 'teddybear', 'television', 'vase', 'yellow door']
all_changed_objects: ['bike', 'desklamp', 'plant', 'television']
newly_observed_changed (this turn): ['television']
unchanged_objects: ['basket', 'chair', 'green door', 'scooter', 'shelf', 'shoes', 'shoppingcart', 'teddybear', 'vase', 'yellow door']
pred_changed_keys: ['television']
pred_unchanged_keys: ['basket', 'bike', 'chair', 'desklamp', 'green door', 'plant', 'scooter', 'shelf', 'shoes', 'shoppingcart', 'teddybear', 'television', 'vase', 'yellow door']
🧭 False Belief Info
step: 2
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'television', 'ori': True}, {'name': 'desklamp', 'ori': True}, {'name': 'plant', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: ['television']
newly_observed_unchanged_objects: ['green door']
🔄 FB Turn 13
💬 Assistant Action
Actions: [JumpTo(green door), Observe()]
Actions: [JumpTo(green door), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Changed: bike:
dir: None
pos: 0.526
facing: None
overall: None
Retention: bike:
dir: None
pos: 0.213
facing: None
overall: None
Unchanged (all):
dir: 0.727
facing: 1
pos: 0.724
overall: 0.817
🧾 False Belief Object Sets
all_objects: ['basket', 'bike', 'chair', 'desklamp', 'green door', 'plant', 'scooter', 'shelf', 'shoes', 'shoppingcart', 'teddybear', 'television', 'vase', 'yellow door']
all_changed_objects: ['bike', 'desklamp', 'plant', 'television']
newly_observed_changed (this turn): ['bike']
unchanged_objects: ['basket', 'chair', 'green door', 'scooter', 'shelf', 'shoes', 'shoppingcart', 'teddybear', 'vase', 'yellow door']
pred_changed_keys: ['bike']
pred_unchanged_keys: ['basket', 'bike', 'chair', 'desklamp', 'green door', 'plant', 'scooter', 'shelf', 'shoes', 'shoppingcart', 'teddybear', 'television', 'vase', 'yellow door']
🧭 False Belief Info
step: 3
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'television', 'ori': True}, {'name': 'desklamp', 'ori': True}, {'name': 'plant', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: ['bike']
newly_observed_unchanged_objects: ['shoes', 'shelf', 'scooter', 'shoppingcart']
🔄 FB Turn 14
💬 Assistant Action
Actions: [Rotate(180), Observe()]
Actions: [Rotate(180), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Unchanged (all):
dir: 0.667
facing: 1
pos: 0.636
overall: 0.768
🧾 False Belief Object Sets
all_objects: ['basket', 'bike', 'chair', 'desklamp', 'green door', 'plant', 'scooter', 'shelf', 'shoes', 'shoppingcart', 'teddybear', 'television', 'vase', 'yellow door']
all_changed_objects: ['bike', 'desklamp', 'plant', 'television']
newly_observed_changed (this turn): []
unchanged_objects: ['basket', 'chair', 'green door', 'scooter', 'shelf', 'shoes', 'shoppingcart', 'teddybear', 'vase', 'yellow door']
pred_changed_keys: []
pred_unchanged_keys: ['basket', 'bike', 'chair', 'desklamp', 'green door', 'plant', 'scooter', 'shelf', 'shoes', 'shoppingcart', 'teddybear', 'television', 'vase', 'yellow door']
🧭 False Belief Info
step: 4
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'television', 'ori': True}, {'name': 'desklamp', 'ori': True}, {'name': 'plant', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: []
newly_observed_unchanged_objects: ['yellow door']
🔄 FB Turn 15
💬 Assistant Action
Actions: [JumpTo(yellow door), Observe()]
Actions: [JumpTo(yellow door), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Changed: desklamp:
dir: None
pos: None
facing: 1
overall: None
Retention: desklamp:
dir: None
pos: None
facing: 0
overall: None
Unchanged (all):
dir: 0.606
facing: 1
pos: 0.665
overall: 0.757
🧾 False Belief Object Sets
all_objects: ['basket', 'bike', 'chair', 'desklamp', 'green door', 'plant', 'scooter', 'shelf', 'shoes', 'shoppingcart', 'teddybear', 'television', 'vase', 'yellow door']
all_changed_objects: ['bike', 'desklamp', 'plant', 'television']
newly_observed_changed (this turn): ['desklamp']
unchanged_objects: ['basket', 'chair', 'green door', 'scooter', 'shelf', 'shoes', 'shoppingcart', 'teddybear', 'vase', 'yellow door']
pred_changed_keys: ['desklamp']
pred_unchanged_keys: ['basket', 'bike', 'chair', 'desklamp', 'green door', 'plant', 'scooter', 'shelf', 'shoes', 'shoppingcart', 'teddybear', 'television', 'vase', 'yellow door']
🧭 False Belief Info
step: 5
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'television', 'ori': True}, {'name': 'desklamp', 'ori': True}, {'name': 'plant', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: ['desklamp']
newly_observed_unchanged_objects: ['chair']
🔄 FB Turn 16
💬 Assistant Action
Actions: [Term(changes="television: orientation, plant: position, bike: position, desklamp: orientation")]
Actions: [Term(changes="television: orientation, plant: position, bike: position, desklamp: orientation")]
📈 Turn Metrics
reward: 1
is_done: True
is_valid_action: True
success: 1
🧭 False Belief Info
step: 6
correctly_identified_changes: 1
f1_overall: 1
f1_position: 1
f1_facing: 1
action_cost: 5
action_cost_after_seen: 0
ground_truth_changes: [{'name': 'television', 'ori': True}, {'name': 'desklamp', 'ori': True}, {'name': 'plant', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: [{'name': 'television', 'ori': True}, {'name': 'plant', 'pos': True}, {'name': 'bike', 'pos': True}, {'name': 'desklamp', 'ori': True}]
newly_observed_changed_objects: []
newly_observed_unchanged_objects: []
📊 Evaluation (default)
📊 Task: DirectionEvaluationTask
Question 1 (ID: a5560bd263efdb31)
✅ Evaluation Results
id: a5560bd263efdb31
question: You return to your starting position and face north.
From a Top-Down map, describe where basket is relative to vase.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: north west, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Question 2 (ID: bb2523a0048c8980)
✅ Evaluation Results
id: bb2523a0048c8980
question: You return to your starting position and face north.
From a Top-Down map, describe where plant is relative to basket.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: south west, slightly far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 3 (ID: daa53364e07a0ab1)
✅ Evaluation Results
id: daa53364e07a0ab1
question: You return to your starting position and face north.
From a Top-Down map, describe where desklamp is relative to chair.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: north west, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
📊 Task: PovEvaluationTask
Question 1 (ID: 3f86078c87f16910)
✅ Evaluation Results
id: 3f86078c87f16910
question: Now you jump to scooter's direction, facing its direction.
Describe where shelf is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 2 (ID: 18c74f89f0b249a8)
✅ Evaluation Results
id: 18c74f89f0b249a8
question: Now you jump to desklamp's direction, facing its direction.
Describe where plant is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front, near
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 3 (ID: 581f2fdf25b85ca0)
✅ Evaluation Results
id: 581f2fdf25b85ca0
question: Now you jump to scooter's direction, facing its direction.
Describe where shoes is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front-right, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
📊 Task: BackwardPovTextEvaluationTask
Question 1 (ID: 98f3ab2f400036ba)
✅ Evaluation Results
id: 98f3ab2f400036ba
question: Now you jump to an object's position, facing its direction.
You observe that shoes is front-right, mid distance, facing forward; shelf is front-left, mid distance.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: scooter
final_pos: [17, 4]
final_ori: [1, 0]
final_observation: [{'name': 'shoes', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'shelf', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': None}]
object_positions:
television: [13.0, 2.0]
shoppingcart: [17.0, 2.0]
chair: [5.0, 2.0]
teddybear: [11.0, 4.0]
scooter: [17.0, 4.0]
desklamp: [3.0, 4.0]
bike: [11.0, 6.0]
vase: [8.0, 4.0]
shoes: [19.0, 2.0]
plant: [1.0, 4.0]
shelf: [19.0, 6.0]
basket: [5.0, 6.0]
yellow door: [7.0, 2.0]
green door: [14.0, 5.0]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
room_id: 2
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: default
Question 2 (ID: 06116bb98f50440a)
✅ Evaluation Results
id: 06116bb98f50440a
question: Now you jump to an object's position, facing its direction.
You observe that television is front-right, mid distance, facing right; green door is front-slight-left, mid distance, on front wall.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: teddybear
final_pos: [11, 4]
final_ori: [1, 0]
final_observation: [{'name': 'television', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'green door', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'on front wall'}]
object_positions:
television: [13.0, 2.0]
shoppingcart: [17.0, 2.0]
chair: [5.0, 2.0]
teddybear: [11.0, 4.0]
scooter: [17.0, 4.0]
desklamp: [3.0, 4.0]
bike: [11.0, 6.0]
vase: [8.0, 4.0]
shoes: [19.0, 2.0]
plant: [1.0, 4.0]
shelf: [19.0, 6.0]
basket: [5.0, 6.0]
yellow door: [7.0, 2.0]
green door: [14.0, 5.0]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
room_id: 1
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: False
view_match: False
score: 0
evaluation_mode: default
Question 3 (ID: a0575673764e4662)
✅ Evaluation Results
id: a0575673764e4662
question: Now you jump to an object's position, facing its direction.
You observe that plant is front, near.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: desklamp
final_pos: [3, 4]
final_ori: [-1, 0]
final_observation: [{'name': 'plant', 'direction': 'front', 'distance': 'near', 'orientation': None}]
object_positions:
television: [13.0, 2.0]
shoppingcart: [17.0, 2.0]
chair: [5.0, 2.0]
teddybear: [11.0, 4.0]
scooter: [17.0, 4.0]
desklamp: [3.0, 4.0]
bike: [11.0, 6.0]
vase: [8.0, 4.0]
shoes: [19.0, 2.0]
plant: [1.0, 4.0]
shelf: [19.0, 6.0]
basket: [5.0, 6.0]
yellow door: [7.0, 2.0]
green door: [14.0, 5.0]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
room_id: 3
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: default
📊 Task: Action2ViewEvaluationTask
Question 1 (ID: e3a718b9ba658f8a)
✅ Evaluation Results
id: e3a718b9ba658f8a
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Rotate(-90)
2. Jump to the object at front, mid distance.
3. Jump to the object at front-slight-right, slightly far.
4. Rotate(-180)
After executing the actions, what is the ego relation of yellow door relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-slight-right, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 2 (ID: 4ddf26d26bf2de51)
✅ Evaluation Results
id: 4ddf26d26bf2de51
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Jump to the object at front-right, mid distance.
2. Rotate(90)
3. Jump to the object at front-right, mid distance.
4. Rotate(-90)
After executing the actions, what is the ego relation of bike relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 3 (ID: 73bdb5bede08b131)
✅ Evaluation Results
id: 73bdb5bede08b131
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Jump to the object at front-slight-right, slightly far.
2. Rotate(-180)
3. Jump to the object at front-right, slightly far.
4. Rotate(-90)
After executing the actions, what is the ego relation of television relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
📊 Task: View2ActionTextEvaluationTask
Question 1 (ID: 9f3daf6b05eca7d1)
✅ Evaluation Results
id: 9f3daf6b05eca7d1
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
yellow door is at front-slight-right, slightly far, on front wall; chair is at front-right, slightly far, facing right; desklamp is at front, near, facing backward
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [1, 4]
final_ori: [1, 0]
room_id: 3
init_pos: [10, 2]
init_ori: [0, 1]
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', -90], ['jumpto', 'yellow door'], ['jumpto', 'plant'], ['rotate', 180]]
final_observation: [{'name': 'yellow door', 'direction': 'front-slight-right', 'distance': 'slightly far', 'orientation': 'on front wall'}, {'name': 'chair', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'desklamp', 'direction': 'front', 'distance': 'near', 'orientation': 'facing backward'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: False
final_pos: [1.0, 4.0]
final_ori: [0, -1]
visible_match: False
score: 0
evaluation_mode: default
Question 2 (ID: 5b0522617e98fcb6)
✅ Evaluation Results
id: 5b0522617e98fcb6
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
desklamp is at front-left, mid distance, facing forward; plant is at front-left, slightly far
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [5, 6]
final_ori: [-1, 0]
room_id: 3
init_pos: [10, 2]
init_ori: [0, 1]
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', -90], ['jumpto', 'yellow door'], ['rotate', 90], ['jumpto', 'basket'], ['rotate', -90]]
final_observation: [{'name': 'desklamp', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'plant', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: False
final_pos: [5.0, 2.0]
final_ori: [1, 0]
visible_match: False
score: 0
evaluation_mode: default
Question 3 (ID: c9fb26112af06791)
✅ Evaluation Results
id: c9fb26112af06791
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
green door is at front-right, slightly far, on front wall
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [17, 2]
final_ori: [-1, 0]
room_id: 2
init_pos: [10, 2]
init_ori: [0, 1]
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', 90], ['jumpto', 'green door'], ['jumpto', 'shoppingcart'], ['rotate', 180]]
final_observation: [{'name': 'green door', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: False
final_pos: [8.0, 4.0]
final_ori: [1, 0]
visible_match: False
score: 0
evaluation_mode: default
📊 Task: AlloMappingEvaluationTask
Question 1 (ID: 2cc0be1d05b6551a)
✅ Evaluation Results
id: 2cc0be1d05b6551a
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: basket, vase, chair.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-5, 4], [-2, 2], [-5, 0]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.841
threshold: 0.900
score: 0.841
evaluation_mode: default
Question 2 (ID: 9523d4db4cc7f611)
✅ Evaluation Results
id: 9523d4db4cc7f611
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: shelf, shoppingcart, basket, scooter.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[9, 4], [7, 0], [-5, 4], [7, 2]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.833
threshold: 0.900
score: 0.833
evaluation_mode: default
Question 3 (ID: 934a5d170d934897)
✅ Evaluation Results
id: 934a5d170d934897
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: chair, bike, vase, shoppingcart, teddybear.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-5, 0], [1, 4], [-2, 2], [7, 0], [1, 2]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.953
threshold: 0.900
score: 0.953
evaluation_mode: default
📊 Task: RotEvaluationTask
Question 1 (ID: fa250972cf6336ae)
✅ Evaluation Results
id: fa250972cf6336ae
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning counterclockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: plant, shoes, vase.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['vase', 'plant', 'shoes']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: default
Question 2 (ID: bb289996eb948eb5)
✅ Evaluation Results
id: bb289996eb948eb5
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning clockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: bike, desklamp, scooter.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['bike', 'scooter', 'desklamp']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 3 (ID: 849efca6f71b0548)
✅ Evaluation Results
id: 849efca6f71b0548
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning counterclockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: chair, shelf, vase.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['vase', 'chair', 'shelf']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
📊 Task: RotDualEvaluationTask
Question 1 (ID: 9c32cc1590bebf7a)
✅ Evaluation Results
id: 9c32cc1590bebf7a
question: You return to your starting position and face north.
You performed a complete 360° rotation in place.
Assume all walls are removed (you can see through walls), so every object is visible.
During the rotation, these objects appeared directly in front of you in this order:
vase, plant, shoes
Based on this sequence, in which direction did you rotate?
Answer format: clockwise or counterclockwise
Example: clockwise
action: None
answer: counterclockwise
task_type: RotDualEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 2 (ID: b329ee5b46d5266a)
✅ Evaluation Results
id: b329ee5b46d5266a
question: You return to your starting position and face north.
You performed a complete 360° rotation in place.
Assume all walls are removed (you can see through walls), so every object is visible.
During the rotation, these objects appeared directly in front of you in this order:
bike, scooter, desklamp
Based on this sequence, in which direction did you rotate?
Answer format: clockwise or counterclockwise
Example: clockwise
action: None
answer: clockwise
task_type: RotDualEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 3 (ID: 70141be403b87c4e)
✅ Evaluation Results
id: 70141be403b87c4e
question: You return to your starting position and face north.
You performed a complete 360° rotation in place.
Assume all walls are removed (you can see through walls), so every object is visible.
During the rotation, these objects appeared directly in front of you in this order:
vase, chair, shelf
Based on this sequence, in which direction did you rotate?
Answer format: clockwise or counterclockwise
Example: clockwise
action: None
answer: counterclockwise
task_type: RotDualEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
📊 Task: Location2ViewEvaluationTask
Question 1 (ID: 7a18b9e1a601f3c8)
✅ Evaluation Results
id: 7a18b9e1a601f3c8
question: Still treat your initial position as origin (0, 0)
You move to (-2, 1) and face east.
What is the egocentric relation of bike?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2LocationEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Question 2 (ID: 8497629827c2666b)
✅ Evaluation Results
id: 8497629827c2666b
question: Still treat your initial position as origin (0, 0)
You move to (-1, 2) and face west.
What is the egocentric relation of vase?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front, near
task_type: Action2LocationEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Question 3 (ID: 2be81510a6e4a0ff)
✅ Evaluation Results
id: 2be81510a6e4a0ff
question: Still treat your initial position as origin (0, 0)
You move to (0, 4) and face east.
What is the egocentric relation of bike?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front, near
task_type: Action2LocationEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: default
📊 Task: View2LocationTextEvaluationTask
Question 1 (ID: 71d44ca0438f07d8)
✅ Evaluation Results
id: 71d44ca0438f07d8
question: You move to a new location and face east.
You observe: bike is at front-left, slightly far, facing forward; teddybear is at front-slight-left, mid distance, facing forward; television is at front-slight-right, slightly far, facing right
Still treat your initial position as origin (0, 0)
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [-2, 1]
final_pos: [8, 3]
final_ori: [1, 0]
room_id: 1
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'bike', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing forward'}, {'name': 'teddybear', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'television', 'direction': 'front-slight-right', 'distance': 'slightly far', 'orientation': 'facing right'}]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: Location2ActionTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.368
raw_score: 0.368
score: 0.368
evaluation_mode: default
Question 2 (ID: abe453e8057600a8)
✅ Evaluation Results
id: abe453e8057600a8
question: You move to a new location and face west.
You observe: vase is at front, near; yellow door is at front-left, mid distance, on front wall
Still treat your initial position as origin (0, 0)
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [-1, 2]
final_pos: [9, 4]
final_ori: [-1, 0]
room_id: 1
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'vase', 'direction': 'front', 'distance': 'near', 'orientation': None}, {'name': 'yellow door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on front wall'}]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: Location2ActionTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.639
raw_score: 0.639
score: 0.639
evaluation_mode: default
Question 3 (ID: dafbe7465cd09199)
✅ Evaluation Results
id: dafbe7465cd09199
question: You move to a new location and face east.
You observe: bike is at front, near, facing forward; green door is at front-slight-right, slightly far, on front wall
Still treat your initial position as origin (0, 0)
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [0, 4]
final_pos: [10, 6]
final_ori: [1, 0]
room_id: 1
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'bike', 'direction': 'front', 'distance': 'near', 'orientation': 'facing forward'}, {'name': 'green door', 'direction': 'front-slight-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: Location2ActionTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: default
📊 Evaluation (prompt_cogmap)
📊 Task: DirectionEvaluationTask
Question 1 (ID: a5560bd263efdb31)
✅ Evaluation Results
id: a5560bd263efdb31
question: You return to your starting position and face north.
From a Top-Down map, describe where basket is relative to vase.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: north west, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Question 2 (ID: bb2523a0048c8980)
✅ Evaluation Results
id: bb2523a0048c8980
question: You return to your starting position and face north.
From a Top-Down map, describe where plant is relative to basket.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: south west, slightly far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Question 3 (ID: daa53364e07a0ab1)
✅ Evaluation Results
id: daa53364e07a0ab1
question: You return to your starting position and face north.
From a Top-Down map, describe where desklamp is relative to chair.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: north west, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
📊 Task: PovEvaluationTask
Question 1 (ID: 3f86078c87f16910)
✅ Evaluation Results
id: 3f86078c87f16910
question: Now you jump to scooter's direction, facing its direction.
Describe where shelf is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Question 2 (ID: 18c74f89f0b249a8)
✅ Evaluation Results
id: 18c74f89f0b249a8
question: Now you jump to desklamp's direction, facing its direction.
Describe where plant is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front, near
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Question 3 (ID: 581f2fdf25b85ca0)
✅ Evaluation Results
id: 581f2fdf25b85ca0
question: Now you jump to scooter's direction, facing its direction.
Describe where shoes is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front-right, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
📊 Task: BackwardPovTextEvaluationTask
Question 1 (ID: 98f3ab2f400036ba)
✅ Evaluation Results
id: 98f3ab2f400036ba
question: Now you jump to an object's position, facing its direction.
You observe that shoes is front-right, mid distance, facing forward; shelf is front-left, mid distance.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: scooter
final_pos: [17, 4]
final_ori: [1, 0]
final_observation: [{'name': 'shoes', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'shelf', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': None}]
object_positions:
television: [13.0, 2.0]
shoppingcart: [17.0, 2.0]
chair: [5.0, 2.0]
teddybear: [11.0, 4.0]
scooter: [17.0, 4.0]
desklamp: [3.0, 4.0]
bike: [11.0, 6.0]
vase: [8.0, 4.0]
shoes: [19.0, 2.0]
plant: [1.0, 4.0]
shelf: [19.0, 6.0]
basket: [5.0, 6.0]
yellow door: [7.0, 2.0]
green door: [14.0, 5.0]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
room_id: 2
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: prompt_cogmap
Question 2 (ID: 06116bb98f50440a)
✅ Evaluation Results
id: 06116bb98f50440a
question: Now you jump to an object's position, facing its direction.
You observe that television is front-right, mid distance, facing right; green door is front-slight-left, mid distance, on front wall.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: teddybear
final_pos: [11, 4]
final_ori: [1, 0]
final_observation: [{'name': 'television', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'green door', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'on front wall'}]
object_positions:
television: [13.0, 2.0]
shoppingcart: [17.0, 2.0]
chair: [5.0, 2.0]
teddybear: [11.0, 4.0]
scooter: [17.0, 4.0]
desklamp: [3.0, 4.0]
bike: [11.0, 6.0]
vase: [8.0, 4.0]
shoes: [19.0, 2.0]
plant: [1.0, 4.0]
shelf: [19.0, 6.0]
basket: [5.0, 6.0]
yellow door: [7.0, 2.0]
green door: [14.0, 5.0]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
room_id: 1
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: prompt_cogmap
Question 3 (ID: a0575673764e4662)
✅ Evaluation Results
id: a0575673764e4662
question: Now you jump to an object's position, facing its direction.
You observe that plant is front, near.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: desklamp
final_pos: [3, 4]
final_ori: [-1, 0]
final_observation: [{'name': 'plant', 'direction': 'front', 'distance': 'near', 'orientation': None}]
object_positions:
television: [13.0, 2.0]
shoppingcart: [17.0, 2.0]
chair: [5.0, 2.0]
teddybear: [11.0, 4.0]
scooter: [17.0, 4.0]
desklamp: [3.0, 4.0]
bike: [11.0, 6.0]
vase: [8.0, 4.0]
shoes: [19.0, 2.0]
plant: [1.0, 4.0]
shelf: [19.0, 6.0]
basket: [5.0, 6.0]
yellow door: [7.0, 2.0]
green door: [14.0, 5.0]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
room_id: 3
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: prompt_cogmap
📊 Task: Action2ViewEvaluationTask
Question 1 (ID: e3a718b9ba658f8a)
✅ Evaluation Results
id: e3a718b9ba658f8a
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Rotate(-90)
2. Jump to the object at front, mid distance.
3. Jump to the object at front-slight-right, slightly far.
4. Rotate(-180)
After executing the actions, what is the ego relation of yellow door relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-slight-right, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Question 2 (ID: 4ddf26d26bf2de51)
✅ Evaluation Results
id: 4ddf26d26bf2de51
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Jump to the object at front-right, mid distance.
2. Rotate(90)
3. Jump to the object at front-right, mid distance.
4. Rotate(-90)
After executing the actions, what is the ego relation of bike relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: prompt_cogmap
Question 3 (ID: 73bdb5bede08b131)
✅ Evaluation Results
id: 73bdb5bede08b131
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Jump to the object at front-slight-right, slightly far.
2. Rotate(-180)
3. Jump to the object at front-right, slightly far.
4. Rotate(-90)
After executing the actions, what is the ego relation of television relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
📊 Task: View2ActionTextEvaluationTask
Question 1 (ID: 9f3daf6b05eca7d1)
✅ Evaluation Results
id: 9f3daf6b05eca7d1
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
yellow door is at front-slight-right, slightly far, on front wall; chair is at front-right, slightly far, facing right; desklamp is at front, near, facing backward
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [1, 4]
final_ori: [1, 0]
room_id: 3
init_pos: [10, 2]
init_ori: [0, 1]
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', -90], ['jumpto', 'yellow door'], ['jumpto', 'plant'], ['rotate', 180]]
final_observation: [{'name': 'yellow door', 'direction': 'front-slight-right', 'distance': 'slightly far', 'orientation': 'on front wall'}, {'name': 'chair', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'desklamp', 'direction': 'front', 'distance': 'near', 'orientation': 'facing backward'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [1.0, 4.0]
final_ori: [1, 0]
visible_match: True
score: 1
evaluation_mode: prompt_cogmap
Question 2 (ID: 5b0522617e98fcb6)
✅ Evaluation Results
id: 5b0522617e98fcb6
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
desklamp is at front-left, mid distance, facing forward; plant is at front-left, slightly far
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [5, 6]
final_ori: [-1, 0]
room_id: 3
init_pos: [10, 2]
init_ori: [0, 1]
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', -90], ['jumpto', 'yellow door'], ['rotate', 90], ['jumpto', 'basket'], ['rotate', -90]]
final_observation: [{'name': 'desklamp', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'plant', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: False
final_pos: [5.0, 2.0]
final_ori: [0, 1]
visible_match: False
score: 0
evaluation_mode: prompt_cogmap
Question 3 (ID: c9fb26112af06791)
✅ Evaluation Results
id: c9fb26112af06791
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
green door is at front-right, slightly far, on front wall
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [17, 2]
final_ori: [-1, 0]
room_id: 2
init_pos: [10, 2]
init_ori: [0, 1]
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', 90], ['jumpto', 'green door'], ['jumpto', 'shoppingcart'], ['rotate', 180]]
final_observation: [{'name': 'green door', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: False
final_pos: [13.0, 2.0]
final_ori: [0, 1]
visible_match: False
score: 0
evaluation_mode: prompt_cogmap
📊 Task: AlloMappingEvaluationTask
Question 1 (ID: 2cc0be1d05b6551a)
✅ Evaluation Results
id: 2cc0be1d05b6551a
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: basket, vase, chair.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-5, 4], [-2, 2], [-5, 0]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.719
threshold: 0.900
score: 0.719
evaluation_mode: prompt_cogmap
Question 2 (ID: 9523d4db4cc7f611)
✅ Evaluation Results
id: 9523d4db4cc7f611
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: shelf, shoppingcart, basket, scooter.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[9, 4], [7, 0], [-5, 4], [7, 2]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.758
threshold: 0.900
score: 0.758
evaluation_mode: prompt_cogmap
Question 3 (ID: 934a5d170d934897)
✅ Evaluation Results
id: 934a5d170d934897
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: chair, bike, vase, shoppingcart, teddybear.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-5, 0], [1, 4], [-2, 2], [7, 0], [1, 2]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.690
threshold: 0.900
score: 0.690
evaluation_mode: prompt_cogmap
📊 Task: RotEvaluationTask
Question 1 (ID: fa250972cf6336ae)
✅ Evaluation Results
id: fa250972cf6336ae
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning counterclockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: plant, shoes, vase.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['vase', 'plant', 'shoes']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Question 2 (ID: bb289996eb948eb5)
✅ Evaluation Results
id: bb289996eb948eb5
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning clockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: bike, desklamp, scooter.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['bike', 'scooter', 'desklamp']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Question 3 (ID: 849efca6f71b0548)
✅ Evaluation Results
id: 849efca6f71b0548
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning counterclockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: chair, shelf, vase.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['vase', 'chair', 'shelf']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
📊 Task: Location2ViewEvaluationTask
Question 1 (ID: 7a18b9e1a601f3c8)
✅ Evaluation Results
id: 7a18b9e1a601f3c8
question: Still treat your initial position as origin (0, 0)
You move to (-2, 1) and face east.
What is the egocentric relation of bike?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front-left, slightly far
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Question 2 (ID: 8497629827c2666b)
✅ Evaluation Results
id: 8497629827c2666b
question: Still treat your initial position as origin (0, 0)
You move to (-1, 2) and face west.
What is the egocentric relation of vase?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front, near
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Question 3 (ID: 2be81510a6e4a0ff)
✅ Evaluation Results
id: 2be81510a6e4a0ff
question: Still treat your initial position as origin (0, 0)
You move to (0, 4) and face east.
What is the egocentric relation of bike?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front, near
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: prompt_cogmap
📊 Task: View2LocationTextEvaluationTask
Question 1 (ID: 71d44ca0438f07d8)
✅ Evaluation Results
id: 71d44ca0438f07d8
question: You move to a new location and face east.
You observe: bike is at front-left, slightly far, facing forward; teddybear is at front-slight-left, mid distance, facing forward; television is at front-slight-right, slightly far, facing right
Still treat your initial position as origin (0, 0)
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [-2, 1]
final_pos: [8, 3]
final_ori: [1, 0]
room_id: 1
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'bike', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing forward'}, {'name': 'teddybear', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'television', 'direction': 'front-slight-right', 'distance': 'slightly far', 'orientation': 'facing right'}]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.531
raw_score: 0.531
score: 0.531
evaluation_mode: prompt_cogmap
Question 2 (ID: abe453e8057600a8)
✅ Evaluation Results
id: abe453e8057600a8
question: You move to a new location and face west.
You observe: vase is at front, near; yellow door is at front-left, mid distance, on front wall
Still treat your initial position as origin (0, 0)
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [-1, 2]
final_pos: [9, 4]
final_ori: [-1, 0]
room_id: 1
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'vase', 'direction': 'front', 'distance': 'near', 'orientation': None}, {'name': 'yellow door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on front wall'}]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.639
raw_score: 0.639
score: 0.639
evaluation_mode: prompt_cogmap
Question 3 (ID: dafbe7465cd09199)
✅ Evaluation Results
id: dafbe7465cd09199
question: You move to a new location and face east.
You observe: bike is at front, near, facing forward; green door is at front-slight-right, slightly far, on front wall
Still treat your initial position as origin (0, 0)
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [0, 4]
final_pos: [10, 6]
final_ori: [1, 0]
room_id: 1
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'bike', 'direction': 'front', 'distance': 'near', 'orientation': 'facing forward'}, {'name': 'green door', 'direction': 'front-slight-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.607
raw_score: 0.607
score: 0.607
evaluation_mode: prompt_cogmap
📊 Evaluation (use_gt_cogmap)
📊 Task: DirectionEvaluationTask
Question 1 (ID: a5560bd263efdb31)
✅ Evaluation Results
id: a5560bd263efdb31
question: You return to your starting position and face north.
From a Top-Down map, describe where basket is relative to vase.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: north west, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: bb2523a0048c8980)
✅ Evaluation Results
id: bb2523a0048c8980
question: You return to your starting position and face north.
From a Top-Down map, describe where plant is relative to basket.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: south west, slightly far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 3 (ID: daa53364e07a0ab1)
✅ Evaluation Results
id: daa53364e07a0ab1
question: You return to your starting position and face north.
From a Top-Down map, describe where desklamp is relative to chair.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: north west, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
📊 Task: PovEvaluationTask
Question 1 (ID: 3f86078c87f16910)
✅ Evaluation Results
id: 3f86078c87f16910
question: Now you jump to scooter's direction, facing its direction.
Describe where shelf is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: 18c74f89f0b249a8)
✅ Evaluation Results
id: 18c74f89f0b249a8
question: Now you jump to desklamp's direction, facing its direction.
Describe where plant is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front, near
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 3 (ID: 581f2fdf25b85ca0)
✅ Evaluation Results
id: 581f2fdf25b85ca0
question: Now you jump to scooter's direction, facing its direction.
Describe where shoes is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front-right, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
📊 Task: BackwardPovTextEvaluationTask
Question 1 (ID: 98f3ab2f400036ba)
✅ Evaluation Results
id: 98f3ab2f400036ba
question: Now you jump to an object's position, facing its direction.
You observe that shoes is front-right, mid distance, facing forward; shelf is front-left, mid distance.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: scooter
final_pos: [17, 4]
final_ori: [1, 0]
final_observation: [{'name': 'shoes', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'shelf', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': None}]
object_positions:
television: [13.0, 2.0]
shoppingcart: [17.0, 2.0]
chair: [5.0, 2.0]
teddybear: [11.0, 4.0]
scooter: [17.0, 4.0]
desklamp: [3.0, 4.0]
bike: [11.0, 6.0]
vase: [8.0, 4.0]
shoes: [19.0, 2.0]
plant: [1.0, 4.0]
shelf: [19.0, 6.0]
basket: [5.0, 6.0]
yellow door: [7.0, 2.0]
green door: [14.0, 5.0]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
room_id: 2
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: 06116bb98f50440a)
✅ Evaluation Results
id: 06116bb98f50440a
question: Now you jump to an object's position, facing its direction.
You observe that television is front-right, mid distance, facing right; green door is front-slight-left, mid distance, on front wall.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: teddybear
final_pos: [11, 4]
final_ori: [1, 0]
final_observation: [{'name': 'television', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'green door', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'on front wall'}]
object_positions:
television: [13.0, 2.0]
shoppingcart: [17.0, 2.0]
chair: [5.0, 2.0]
teddybear: [11.0, 4.0]
scooter: [17.0, 4.0]
desklamp: [3.0, 4.0]
bike: [11.0, 6.0]
vase: [8.0, 4.0]
shoes: [19.0, 2.0]
plant: [1.0, 4.0]
shelf: [19.0, 6.0]
basket: [5.0, 6.0]
yellow door: [7.0, 2.0]
green door: [14.0, 5.0]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
room_id: 1
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_gt_cogmap
Question 3 (ID: a0575673764e4662)
✅ Evaluation Results
id: a0575673764e4662
question: Now you jump to an object's position, facing its direction.
You observe that plant is front, near.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: desklamp
final_pos: [3, 4]
final_ori: [-1, 0]
final_observation: [{'name': 'plant', 'direction': 'front', 'distance': 'near', 'orientation': None}]
object_positions:
television: [13.0, 2.0]
shoppingcart: [17.0, 2.0]
chair: [5.0, 2.0]
teddybear: [11.0, 4.0]
scooter: [17.0, 4.0]
desklamp: [3.0, 4.0]
bike: [11.0, 6.0]
vase: [8.0, 4.0]
shoes: [19.0, 2.0]
plant: [1.0, 4.0]
shelf: [19.0, 6.0]
basket: [5.0, 6.0]
yellow door: [7.0, 2.0]
green door: [14.0, 5.0]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
room_id: 3
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_gt_cogmap
📊 Task: Action2ViewEvaluationTask
Question 1 (ID: e3a718b9ba658f8a)
✅ Evaluation Results
id: e3a718b9ba658f8a
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Rotate(-90)
2. Jump to the object at front, mid distance.
3. Jump to the object at front-slight-right, slightly far.
4. Rotate(-180)
After executing the actions, what is the ego relation of yellow door relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-slight-right, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: 4ddf26d26bf2de51)
✅ Evaluation Results
id: 4ddf26d26bf2de51
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Jump to the object at front-right, mid distance.
2. Rotate(90)
3. Jump to the object at front-right, mid distance.
4. Rotate(-90)
After executing the actions, what is the ego relation of bike relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 3 (ID: 73bdb5bede08b131)
✅ Evaluation Results
id: 73bdb5bede08b131
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Jump to the object at front-slight-right, slightly far.
2. Rotate(-180)
3. Jump to the object at front-right, slightly far.
4. Rotate(-90)
After executing the actions, what is the ego relation of television relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
📊 Task: View2ActionTextEvaluationTask
Question 1 (ID: 9f3daf6b05eca7d1)
✅ Evaluation Results
id: 9f3daf6b05eca7d1
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
yellow door is at front-slight-right, slightly far, on front wall; chair is at front-right, slightly far, facing right; desklamp is at front, near, facing backward
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [1, 4]
final_ori: [1, 0]
room_id: 3
init_pos: [10, 2]
init_ori: [0, 1]
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', -90], ['jumpto', 'yellow door'], ['jumpto', 'plant'], ['rotate', 180]]
final_observation: [{'name': 'yellow door', 'direction': 'front-slight-right', 'distance': 'slightly far', 'orientation': 'on front wall'}, {'name': 'chair', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'desklamp', 'direction': 'front', 'distance': 'near', 'orientation': 'facing backward'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [1.0, 4.0]
final_ori: [1, 0]
visible_match: True
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: 5b0522617e98fcb6)
✅ Evaluation Results
id: 5b0522617e98fcb6
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
desklamp is at front-left, mid distance, facing forward; plant is at front-left, slightly far
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [5, 6]
final_ori: [-1, 0]
room_id: 3
init_pos: [10, 2]
init_ori: [0, 1]
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', -90], ['jumpto', 'yellow door'], ['rotate', 90], ['jumpto', 'basket'], ['rotate', -90]]
final_observation: [{'name': 'desklamp', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'plant', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [5.0, 6.0]
final_ori: [-1, 0]
visible_match: True
score: 1
evaluation_mode: use_gt_cogmap
Question 3 (ID: c9fb26112af06791)
✅ Evaluation Results
id: c9fb26112af06791
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
green door is at front-right, slightly far, on front wall
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [17, 2]
final_ori: [-1, 0]
room_id: 2
init_pos: [10, 2]
init_ori: [0, 1]
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', 90], ['jumpto', 'green door'], ['jumpto', 'shoppingcart'], ['rotate', 180]]
final_observation: [{'name': 'green door', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [17.0, 2.0]
final_ori: [-1, 0]
visible_match: True
score: 1
evaluation_mode: use_gt_cogmap
📊 Task: AlloMappingEvaluationTask
Question 1 (ID: 2cc0be1d05b6551a)
✅ Evaluation Results
id: 2cc0be1d05b6551a
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: basket, vase, chair.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-5, 4], [-2, 2], [-5, 0]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
threshold: 0.900
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: 9523d4db4cc7f611)
✅ Evaluation Results
id: 9523d4db4cc7f611
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: shelf, shoppingcart, basket, scooter.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[9, 4], [7, 0], [-5, 4], [7, 2]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
threshold: 0.900
score: 1
evaluation_mode: use_gt_cogmap
Question 3 (ID: 934a5d170d934897)
✅ Evaluation Results
id: 934a5d170d934897
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: chair, bike, vase, shoppingcart, teddybear.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-5, 0], [1, 4], [-2, 2], [7, 0], [1, 2]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
threshold: 0.900
score: 1
evaluation_mode: use_gt_cogmap
📊 Task: RotEvaluationTask
Question 1 (ID: fa250972cf6336ae)
✅ Evaluation Results
id: fa250972cf6336ae
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning counterclockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: plant, shoes, vase.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['vase', 'plant', 'shoes']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: bb289996eb948eb5)
✅ Evaluation Results
id: bb289996eb948eb5
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning clockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: bike, desklamp, scooter.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['bike', 'scooter', 'desklamp']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 3 (ID: 849efca6f71b0548)
✅ Evaluation Results
id: 849efca6f71b0548
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning counterclockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: chair, shelf, vase.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['vase', 'chair', 'shelf']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
📊 Task: Location2ViewEvaluationTask
Question 1 (ID: 7a18b9e1a601f3c8)
✅ Evaluation Results
id: 7a18b9e1a601f3c8
question: Still treat your initial position as origin (0, 0)
You move to (-2, 1) and face east.
What is the egocentric relation of bike?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front-left, slightly far
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: 8497629827c2666b)
✅ Evaluation Results
id: 8497629827c2666b
question: Still treat your initial position as origin (0, 0)
You move to (-1, 2) and face west.
What is the egocentric relation of vase?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front, near
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Question 3 (ID: 2be81510a6e4a0ff)
✅ Evaluation Results
id: 2be81510a6e4a0ff
question: Still treat your initial position as origin (0, 0)
You move to (0, 4) and face east.
What is the egocentric relation of bike?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front, near
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
📊 Task: View2LocationTextEvaluationTask
Question 1 (ID: 71d44ca0438f07d8)
✅ Evaluation Results
id: 71d44ca0438f07d8
question: You move to a new location and face east.
You observe: bike is at front-left, slightly far, facing forward; teddybear is at front-slight-left, mid distance, facing forward; television is at front-slight-right, slightly far, facing right
Still treat your initial position as origin (0, 0)
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [-2, 1]
final_pos: [8, 3]
final_ori: [1, 0]
room_id: 1
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'bike', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing forward'}, {'name': 'teddybear', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'television', 'direction': 'front-slight-right', 'distance': 'slightly far', 'orientation': 'facing right'}]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_gt_cogmap
Question 2 (ID: abe453e8057600a8)
✅ Evaluation Results
id: abe453e8057600a8
question: You move to a new location and face west.
You observe: vase is at front, near; yellow door is at front-left, mid distance, on front wall
Still treat your initial position as origin (0, 0)
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [-1, 2]
final_pos: [9, 4]
final_ori: [-1, 0]
room_id: 1
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'vase', 'direction': 'front', 'distance': 'near', 'orientation': None}, {'name': 'yellow door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on front wall'}]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.639
raw_score: 0.639
score: 0.639
evaluation_mode: use_gt_cogmap
Question 3 (ID: dafbe7465cd09199)
✅ Evaluation Results
id: dafbe7465cd09199
question: You move to a new location and face east.
You observe: bike is at front, near, facing forward; green door is at front-slight-right, slightly far, on front wall
Still treat your initial position as origin (0, 0)
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [0, 4]
final_pos: [10, 6]
final_ori: [1, 0]
room_id: 1
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'bike', 'direction': 'front', 'distance': 'near', 'orientation': 'facing forward'}, {'name': 'green door', 'direction': 'front-slight-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_gt_cogmap
📊 Evaluation (use_model_cogmap)
📊 Task: DirectionEvaluationTask
Question 1 (ID: a5560bd263efdb31)
✅ Evaluation Results
id: a5560bd263efdb31
question: You return to your starting position and face north.
From a Top-Down map, describe where basket is relative to vase.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: north west, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Question 2 (ID: bb2523a0048c8980)
✅ Evaluation Results
id: bb2523a0048c8980
question: You return to your starting position and face north.
From a Top-Down map, describe where plant is relative to basket.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: south west, slightly far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Question 3 (ID: daa53364e07a0ab1)
✅ Evaluation Results
id: daa53364e07a0ab1
question: You return to your starting position and face north.
From a Top-Down map, describe where desklamp is relative to chair.
Answer format: <cardinal direction>, <distance>
Example: north-west, near
action: None
answer: north west, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
📊 Task: PovEvaluationTask
Question 1 (ID: 3f86078c87f16910)
✅ Evaluation Results
id: 3f86078c87f16910
question: Now you jump to scooter's direction, facing its direction.
Describe where shelf is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Question 2 (ID: 18c74f89f0b249a8)
✅ Evaluation Results
id: 18c74f89f0b249a8
question: Now you jump to desklamp's direction, facing its direction.
Describe where plant is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front, near
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Question 3 (ID: 581f2fdf25b85ca0)
✅ Evaluation Results
id: 581f2fdf25b85ca0
question: Now you jump to scooter's direction, facing its direction.
Describe where shoes is relative to you.
Answer format: <ego direction>, <distance>
Example: front-left, near
action: None
answer: front-right, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
📊 Task: BackwardPovTextEvaluationTask
Question 1 (ID: 98f3ab2f400036ba)
✅ Evaluation Results
id: 98f3ab2f400036ba
question: Now you jump to an object's position, facing its direction.
You observe that shoes is front-right, mid distance, facing forward; shelf is front-left, mid distance.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: scooter
final_pos: [17, 4]
final_ori: [1, 0]
final_observation: [{'name': 'shoes', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'shelf', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': None}]
object_positions:
television: [13.0, 2.0]
shoppingcart: [17.0, 2.0]
chair: [5.0, 2.0]
teddybear: [11.0, 4.0]
scooter: [17.0, 4.0]
desklamp: [3.0, 4.0]
bike: [11.0, 6.0]
vase: [8.0, 4.0]
shoes: [19.0, 2.0]
plant: [1.0, 4.0]
shelf: [19.0, 6.0]
basket: [5.0, 6.0]
yellow door: [7.0, 2.0]
green door: [14.0, 5.0]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
room_id: 2
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_model_cogmap
Question 2 (ID: 06116bb98f50440a)
✅ Evaluation Results
id: 06116bb98f50440a
question: Now you jump to an object's position, facing its direction.
You observe that television is front-right, mid distance, facing right; green door is front-slight-left, mid distance, on front wall.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: teddybear
final_pos: [11, 4]
final_ori: [1, 0]
final_observation: [{'name': 'television', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'green door', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'on front wall'}]
object_positions:
television: [13.0, 2.0]
shoppingcart: [17.0, 2.0]
chair: [5.0, 2.0]
teddybear: [11.0, 4.0]
scooter: [17.0, 4.0]
desklamp: [3.0, 4.0]
bike: [11.0, 6.0]
vase: [8.0, 4.0]
shoes: [19.0, 2.0]
plant: [1.0, 4.0]
shelf: [19.0, 6.0]
basket: [5.0, 6.0]
yellow door: [7.0, 2.0]
green door: [14.0, 5.0]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
room_id: 1
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_model_cogmap
Question 3 (ID: a0575673764e4662)
✅ Evaluation Results
id: a0575673764e4662
question: Now you jump to an object's position, facing its direction.
You observe that plant is front, near.
Which object are you standing at?
Answer format: <object_name>
Example: lamp
action: None
answer:
answer: desklamp
final_pos: [3, 4]
final_ori: [-1, 0]
final_observation: [{'name': 'plant', 'direction': 'front', 'distance': 'near', 'orientation': None}]
object_positions:
television: [13.0, 2.0]
shoppingcart: [17.0, 2.0]
chair: [5.0, 2.0]
teddybear: [11.0, 4.0]
scooter: [17.0, 4.0]
desklamp: [3.0, 4.0]
bike: [11.0, 6.0]
vase: [8.0, 4.0]
shoes: [19.0, 2.0]
plant: [1.0, 4.0]
shelf: [19.0, 6.0]
basket: [5.0, 6.0]
yellow door: [7.0, 2.0]
green door: [14.0, 5.0]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
room_id: 3
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_model_cogmap
📊 Task: Action2ViewEvaluationTask
Question 1 (ID: e3a718b9ba658f8a)
✅ Evaluation Results
id: e3a718b9ba658f8a
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Rotate(-90)
2. Jump to the object at front, mid distance.
3. Jump to the object at front-slight-right, slightly far.
4. Rotate(-180)
After executing the actions, what is the ego relation of yellow door relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-slight-right, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Question 2 (ID: 4ddf26d26bf2de51)
✅ Evaluation Results
id: 4ddf26d26bf2de51
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Jump to the object at front-right, mid distance.
2. Rotate(90)
3. Jump to the object at front-right, mid distance.
4. Rotate(-90)
After executing the actions, what is the ego relation of bike relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Question 3 (ID: 73bdb5bede08b131)
✅ Evaluation Results
id: 73bdb5bede08b131
question: You return to your starting position and face north.
You will execute the following action sequence:
1. Jump to the object at front-slight-right, slightly far.
2. Rotate(-180)
3. Jump to the object at front-right, slightly far.
4. Rotate(-90)
After executing the actions, what is the ego relation of television relative to you?
Answer format: <ego direction>, <distance>
Example: front, near
action: None
answer: front, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
📊 Task: View2ActionTextEvaluationTask
Question 1 (ID: 9f3daf6b05eca7d1)
✅ Evaluation Results
id: 9f3daf6b05eca7d1
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
yellow door is at front-slight-right, slightly far, on front wall; chair is at front-right, slightly far, facing right; desklamp is at front, near, facing backward
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [1, 4]
final_ori: [1, 0]
room_id: 3
init_pos: [10, 2]
init_ori: [0, 1]
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', -90], ['jumpto', 'yellow door'], ['jumpto', 'plant'], ['rotate', 180]]
final_observation: [{'name': 'yellow door', 'direction': 'front-slight-right', 'distance': 'slightly far', 'orientation': 'on front wall'}, {'name': 'chair', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'desklamp', 'direction': 'front', 'distance': 'near', 'orientation': 'facing backward'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: False
final_pos: [1.0, 4.0]
final_ori: [0, -1]
visible_match: False
score: 0
evaluation_mode: use_model_cogmap
Question 2 (ID: 5b0522617e98fcb6)
✅ Evaluation Results
id: 5b0522617e98fcb6
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
desklamp is at front-left, mid distance, facing forward; plant is at front-left, slightly far
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [5, 6]
final_ori: [-1, 0]
room_id: 3
init_pos: [10, 2]
init_ori: [0, 1]
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', -90], ['jumpto', 'yellow door'], ['rotate', 90], ['jumpto', 'basket'], ['rotate', -90]]
final_observation: [{'name': 'desklamp', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'plant', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: False
final_pos: [5.0, 2.0]
final_ori: [0, 1]
visible_match: False
score: 0
evaluation_mode: use_model_cogmap
Question 3 (ID: c9fb26112af06791)
✅ Evaluation Results
id: c9fb26112af06791
question: You return to your starting position and face north.
Then you have executed an action sequence and changed to a new location and facing direction.
You observe the following:
green door is at front-right, slightly far, on front wall
What action sequence led to this final view? The action sequence must be valid and only contain move actions.
Answer format: <sequence of move actions>
Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [17, 2]
final_ori: [-1, 0]
room_id: 2
init_pos: [10, 2]
init_ori: [0, 1]
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
minimal_plan: [['rotate', 90], ['jumpto', 'green door'], ['jumpto', 'shoppingcart'], ['rotate', 180]]
final_observation: [{'name': 'green door', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: False
final_pos: [13.0, 2.0]
final_ori: [0, 1]
visible_match: False
score: 0
evaluation_mode: use_model_cogmap
📊 Task: AlloMappingEvaluationTask
Question 1 (ID: 2cc0be1d05b6551a)
✅ Evaluation Results
id: 2cc0be1d05b6551a
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: basket, vase, chair.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-5, 4], [-2, 2], [-5, 0]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.784
threshold: 0.900
score: 0.784
evaluation_mode: use_model_cogmap
Question 2 (ID: 9523d4db4cc7f611)
✅ Evaluation Results
id: 9523d4db4cc7f611
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: shelf, shoppingcart, basket, scooter.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[9, 4], [7, 0], [-5, 4], [7, 2]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.827
threshold: 0.900
score: 0.827
evaluation_mode: use_model_cogmap
Question 3 (ID: 934a5d170d934897)
✅ Evaluation Results
id: 934a5d170d934897
question: Treat your starting position as the origin (0, 0) while facing north.
Report allocentric coordinates using (x right/east, y up/north).
Objects: chair, bike, vase, shoppingcart, teddybear.
Answer format: (x0, y0); (x1, y1); ... in the same order.
Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-5, 0], [1, 4], [-2, 2], [7, 0], [1, 2]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.868
threshold: 0.900
score: 0.868
evaluation_mode: use_model_cogmap
📊 Task: RotEvaluationTask
Question 1 (ID: fa250972cf6336ae)
✅ Evaluation Results
id: fa250972cf6336ae
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning counterclockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: plant, shoes, vase.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['vase', 'plant', 'shoes']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Question 2 (ID: bb289996eb948eb5)
✅ Evaluation Results
id: bb289996eb948eb5
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning clockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: bike, desklamp, scooter.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['bike', 'scooter', 'desklamp']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Question 3 (ID: 849efca6f71b0548)
✅ Evaluation Results
id: 849efca6f71b0548
question: You return to your starting position and face north.
You will perform a full 360-degree rotation by continuously turning counterclockwise in place.
Assume all walls are removed (you can see through walls), so every object is visible.
Focus on this set of objects: chair, shelf, vase.
List them in the exact order they appear directly ahead while you rotate.
If two objects share a bearing, place the nearer one first.
Answer format: <object_name1>, <object_name2>, ...
Example: mug, sofa, plant
action: None
answer: ['vase', 'chair', 'shelf']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
📊 Task: Location2ViewEvaluationTask
Question 1 (ID: 7a18b9e1a601f3c8)
✅ Evaluation Results
id: 7a18b9e1a601f3c8
question: Still treat your initial position as origin (0, 0)
You move to (-2, 1) and face east.
What is the egocentric relation of bike?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front-left, slightly far
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Question 2 (ID: 8497629827c2666b)
✅ Evaluation Results
id: 8497629827c2666b
question: Still treat your initial position as origin (0, 0)
You move to (-1, 2) and face west.
What is the egocentric relation of vase?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front, near
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Question 3 (ID: 2be81510a6e4a0ff)
✅ Evaluation Results
id: 2be81510a6e4a0ff
question: Still treat your initial position as origin (0, 0)
You move to (0, 4) and face east.
What is the egocentric relation of bike?
Answer format: <direction>, <distance>
Example: front, near
action: None
answer: front, near
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
📊 Task: View2LocationTextEvaluationTask
Question 1 (ID: 71d44ca0438f07d8)
✅ Evaluation Results
id: 71d44ca0438f07d8
question: You move to a new location and face east.
You observe: bike is at front-left, slightly far, facing forward; teddybear is at front-slight-left, mid distance, facing forward; television is at front-slight-right, slightly far, facing right
Still treat your initial position as origin (0, 0)
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [-2, 1]
final_pos: [8, 3]
final_ori: [1, 0]
room_id: 1
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'bike', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing forward'}, {'name': 'teddybear', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'television', 'direction': 'front-slight-right', 'distance': 'slightly far', 'orientation': 'facing right'}]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_model_cogmap
Question 2 (ID: abe453e8057600a8)
✅ Evaluation Results
id: abe453e8057600a8
question: You move to a new location and face west.
You observe: vase is at front, near; yellow door is at front-left, mid distance, on front wall
Still treat your initial position as origin (0, 0)
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [-1, 2]
final_pos: [9, 4]
final_ori: [-1, 0]
room_id: 1
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'vase', 'direction': 'front', 'distance': 'near', 'orientation': None}, {'name': 'yellow door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on front wall'}]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.639
raw_score: 0.639
score: 0.639
evaluation_mode: use_model_cogmap
Question 3 (ID: dafbe7465cd09199)
✅ Evaluation Results
id: dafbe7465cd09199
question: You move to a new location and face east.
You observe: bike is at front, near, facing forward; green door is at front-slight-right, slightly far, on front wall
Still treat your initial position as origin (0, 0)
What is your new 2D coordinate (x, y)?
Answer format: (x, y)
Example: (2, -1)
action: None
answer:
coord: [0, 4]
final_pos: [10, 6]
final_ori: [1, 0]
room_id: 1
object_positions:
television: [13, 2]
shoppingcart: [17, 2]
chair: [5, 2]
teddybear: [11, 4]
scooter: [17, 4]
desklamp: [3, 4]
bike: [11, 6]
vase: [8, 4]
shoes: [19, 2]
plant: [1, 4]
shelf: [19, 6]
basket: [5, 6]
yellow door: [7, 2]
green door: [14, 5]
object_orientations:
television: [0, -1]
shoppingcart: [0, -1]
chair: [0, -1]
teddybear: [1, 0]
scooter: [1, 0]
desklamp: [-1, 0]
bike: [1, 0]
shoes: [1, 0]
yellow door: [1, 0]
green door: [1, 0]
final_observation: [{'name': 'bike', 'direction': 'front', 'distance': 'near', 'orientation': 'facing forward'}, {'name': 'green door', 'direction': 'front-slight-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
gate_info:
yellow door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
green door:
room_ids: [1, 2]
ori_by_room:
1: [-1, 0]
2: [1, 0]
object_rooms:
television: 1
shoppingcart: 2
chair: 3
teddybear: 1
scooter: 2
desklamp: 3
bike: 1
vase: 1
shoes: 2
plant: 3
shelf: 2
basket: 3
yellow door: [3, 1]
green door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_model_cogmap