Model: gemini-3-pro-preview

sample_run05

Select Configuration:

📊 Sample Metrics

🔍 Exploration

last_node_coverage: 1
last_edge_coverage: 0.512
n_exploration_steps: 13
action_counts:
move: 6
rotate: 8
return: 0
observe: 12
term: 1
forced_term: 0
query: 0
action_cost: 12
final_information_gain: 0.927
action_fail_ratio: 0
valid_action_ratio: 1
false_belief_steps: 9
false_belief_f1: 1
false_belief_f1_position: 1
false_belief_f1_facing: 1
false_belief_action_cost: 7
false_belief_action_cost_after_seen: 1

✅ Evaluation

overall:
n_total: 27
total_score: 21.823
avg_accuracy: 0.808
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
PovEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Action2ViewEvaluationTask:
n_total: 3
task_score: 1.500
avg_accuracy: 0.500
View2ActionTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
AlloMappingEvaluationTask:
n_total: 3
task_score: 2.524
avg_accuracy: 0.841
RotEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Location2ViewEvaluationTask:
n_total: 3
task_score: 1.500
avg_accuracy: 0.500
View2LocationTextEvaluationTask:
n_total: 3
task_score: 2.800
avg_accuracy: 0.933

✅ Evaluation (prompt_cogmap)

overall:
n_total: 27
total_score: 20.071
avg_accuracy: 0.743
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
PovEvaluationTask:
n_total: 3
task_score: 1.500
avg_accuracy: 0.500
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Action2ViewEvaluationTask:
n_total: 3
task_score: 1
avg_accuracy: 0.333
View2ActionTextEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
AlloMappingEvaluationTask:
n_total: 3
task_score: 2.471
avg_accuracy: 0.824
RotEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Location2ViewEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
View2LocationTextEvaluationTask:
n_total: 3
task_score: 2.599
avg_accuracy: 0.866

✅ Evaluation (use_gt_cogmap)

overall:
n_total: 27
total_score: 24.800
avg_accuracy: 0.919
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
PovEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Action2ViewEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
View2ActionTextEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
AlloMappingEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
RotEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Location2ViewEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
View2LocationTextEvaluationTask:
n_total: 3
task_score: 2.800
avg_accuracy: 0.933

✅ Evaluation (use_model_cogmap)

overall:
n_total: 27
total_score: 21.207
avg_accuracy: 0.785
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
PovEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Action2ViewEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
View2ActionTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
AlloMappingEvaluationTask:
n_total: 3
task_score: 2.089
avg_accuracy: 0.696
RotEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Location2ViewEvaluationTask:
n_total: 3
task_score: 0.500
avg_accuracy: 0.167
View2LocationTextEvaluationTask:
n_total: 3
task_score: 2.618
avg_accuracy: 0.873

🧠 Cognitive Map

exploration:
error:
local_vs_gt_local_avg:
dir: 0.856
facing: 1
pos: 0.905
overall: 0.920
global_vs_gt_global_avg:
dir: 0.762
facing: 0.937
pos: 0.768
overall: 0.822
agent_vs_gt_agent_avg:
dir: 0.769
facing: 1
pos: 0.837
overall: 0.869
newly_observed_vs_gt_local_avg:
dir: 0.911
facing: 1
pos: 0.891
overall: 0.934
correctness:
last_global_vs_gt_full:
dir: 0.658
facing: 1
pos: 0.595
overall: 0.751
consistency:
local_vs_global_avg:
dir: 0.774
facing: 0.889
pos: 0.872
overall: 0.845
position_update_avg: 0.499
facing_update_avg: 0.918
position_stability_avg: 0.499
facing_stability_avg: 0.918
fog_probe:
f1_avg: 0.631
precision_avg: 0.725
recall_avg: 0.574
cogmap_fb:
metrics:
changed:
dir: None
pos: 0.798
facing: 1
overall: None
retention:
dir: None
pos: 0.371
facing: 0
overall: None
unchanged:
dir: 0.614
facing: 0.812
pos: 0.632
overall: 0.686
inertia: -0.011
inertia_list: [-0.03948227577910927, 0.016703211047044725]
unchanged_retention:
dir: None
pos: 0.690
facing: 1
overall: None
unchanged_retention_minus_retention:
pos: 0.711
facing: 1
unchanged_exploration:
dir: 0.636
facing: 1
pos: 0.602
overall: 0.746

🧭 False Belief CogMap

inertia: -0.011
changed:
dir: None
pos: 0.798
facing: 1
overall: None
retention:
dir: None
pos: 0.371
facing: 0
overall: None
unchanged:
dir: 0.614
facing: 0.812
pos: 0.632
overall: 0.686
unchanged_retention:
dir: None
pos: 0.690
facing: 1
overall: None
unchanged_retention_minus_retention:
pos: 0.711
facing: 1
unchanged_exploration:
dir: 0.636
facing: 1
pos: 0.602
overall: 0.746

📊 Performance Charts

Cognitive Map (Update)
Global Update Metrics
Cognitive Map (Full)
Global Full Metrics
Cognitive Map (Self-Tracking)
Global Self-Tracking Metrics
FB CogMap (Unchanged)
False Belief CogMap Unchanged per Turn
Fog Probe F1
Fog Probe F1 per Turn
Fog Probe Precision
Fog Probe Precision per Turn
Fog Probe Recall
Fog Probe Recall per Turn
Position Update
Position Update per Turn
Facing Update
Facing Update per Turn
Position Stability
Position Stability per Turn
Facing Stability
Facing Stability per Turn

🌍 Exploration Phase

🔄 Turn 1

💬 Assistant Action
Actions: [Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Observed)
{
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Full)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "bike": {
    "position": [
      2,
      -5
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      0,
      -3
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {}
}
🎯 Ground Truth
(no data)
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {}
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
################
#########......#
#########......#
##DBC...#......#
##.....E#......#
##*.....#......#
##.....F+......#
##......########
##.A....########
######+#########
#......#########
#......#########
#......#########
#......#########
#......#########
#......#########
################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: A, E, F
🎯 Ground Truth: E, F, A
🧠 Cognitive Map Metrics
Global:
dir: 1
facing: 1
pos: 1
overall: 1
Global (Full):
dir: 0.008
facing: 0.083
pos: 0.067
overall: 0.053
Local:
(none)
Local (Newly):
(none)
Fog Probe:
precision: 1
recall: 1
overall: 1
📈 Turn Metrics
node_coverage: 0
edge_coverage: 0
is_action_fail: False
step: 1
action_counts:
move: 0
rotate: 0
return: 0
observe: 1
term: 0
forced_term: 0
query: 0
information_gain: 0
reward: -0.100
is_done: False
is_valid_action: True
Current state
State at Turn 1

🔄 Turn 2

💬 Assistant Action
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "east"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "yellow door": {
    "position": [
      5,
      -1
    ]
  },
  "blue door": {
    "position": [
      4,
      -3
    ]
  }
}
🎯 Ground Truth (Observed)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "east"
  }
}
🎯 Ground Truth (Full)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "bike": {
    "position": [
      2,
      -5
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      0,
      -3
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "east"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "shoes": {
      "position": [
        2,
        5
      ],
      "facing": "-y"
    },
    "rubberduck": {
      "position": [
        2,
        3
      ],
      "facing": "-x"
    },
    "alarmclock": {
      "position": [
        -2,
        3
      ],
      "facing": "-y"
    },
    "yellow door": {
      "position": [
        2,
        7
      ],
      "facing": "-y"
    },
    "blue door": {
      "position": [
        5,
        6
      ],
      "facing": "-x"
    }
  }
}
🎯 Ground Truth
{
  "shoes": {
    "position": [
      1,
      4
    ],
    "facing": "west"
  },
  "yellow door": {
    "position": [
      1,
      6
    ],
    "facing": "east"
  },
  "alarmclock": {
    "position": [
      -1,
      2
    ],
    "facing": "west"
  },
  "blue door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "rubberduck": {
    "position": [
      1,
      2
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "shoes": {
      "position": [
        2,
        5
      ],
      "facing": "-y"
    },
    "rubberduck": {
      "position": [
        2,
        3
      ],
      "facing": "-x"
    },
    "alarmclock": {
      "position": [
        -2,
        3
      ],
      "facing": "-y"
    },
    "yellow door": {
      "position": [
        2,
        7
      ],
      "facing": "-y"
    },
    "blue door": {
      "position": [
        5,
        6
      ],
      "facing": "-x"
    }
  }
}
🎯 Ground Truth
{
  "shoes": {
    "position": [
      1,
      4
    ],
    "facing": "west"
  },
  "yellow door": {
    "position": [
      1,
      6
    ],
    "facing": "east"
  },
  "alarmclock": {
    "position": [
      -1,
      2
    ],
    "facing": "west"
  },
  "blue door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "rubberduck": {
    "position": [
      1,
      2
    ],
    "facing": "north"
  }
}
🗺️ Symbolic Fog Map
################
#########......#
#########......#
##....C.#......#
##.....D#......#
##*.....#......#
##FA....+......#
##.E....########
##B.....########
######+#########
#......#########
#......#########
#......#########
#......#########
#......#########
#......#########
################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: B, E, F
🎯 Ground Truth: E, F, B
🧠 Cognitive Map Metrics
Global:
dir: 0.952
facing: 1
pos: 0.931
overall: 0.961
Global (Full):
dir: 0.167
facing: 0.333
pos: 0.372
overall: 0.291
Local:
dir: 0.867
facing: 1
pos: 0.819
overall: 0.895
Local (Newly):
dir: 0.867
facing: 1
pos: 0.819
overall: 0.895
Fog Probe:
precision: 1
recall: 1
overall: 1
📈 Turn Metrics
node_coverage: 0.357
edge_coverage: 0.122
is_action_fail: False
step: 2
action_counts:
move: 0
rotate: 1
return: 0
observe: 2
term: 0
forced_term: 0
query: 0
information_gain: 0.318
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 2
Current state
State at Turn 2

🔄 Turn 3

💬 Assistant Action
Actions: [JumpTo(yellow door), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      6,
      -2
    ],
    "facing": "east"
  },
  "alarmclock": {
    "position": [
      3,
      2
    ],
    "facing": "west"
  },
  "rubberduck": {
    "position": [
      3,
      -2
    ],
    "facing": "north"
  },
  "shoes": {
    "position": [
      5,
      -2
    ],
    "facing": "west"
  },
  "yellow door": {
    "position": [
      6,
      -2
    ]
  },
  "blue door": {
    "position": [
      4,
      -4
    ]
  },
  "laptop": {
    "position": [
      9,
      0
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      11,
      2
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      11,
      0
    ],
    "facing": "north"
  },
  "books": {
    "position": [
      9,
      -2
    ]
  }
}
🎯 Ground Truth (Observed)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  }
}
🎯 Ground Truth (Full)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "bike": {
    "position": [
      2,
      -5
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      0,
      -3
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "laptop": {
      "position": [
        -2,
        4
      ],
      "facing": "+y"
    },
    "airplane": {
      "position": [
        -3,
        5
      ],
      "facing": "+x"
    },
    "backpack": {
      "position": [
        -1,
        6
      ],
      "facing": "-x"
    },
    "books": {
      "position": [
        0,
        3
      ]
    }
  }
}
🎯 Ground Truth
{
  "backpack": {
    "position": [
      -2,
      6
    ],
    "facing": "north"
  },
  "laptop": {
    "position": [
      -2,
      4
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      -4,
      6
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      0,
      4
    ],
    "facing": "east"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "laptop": {
      "position": [
        -2,
        4
      ],
      "facing": "+y"
    },
    "airplane": {
      "position": [
        -3,
        5
      ],
      "facing": "+x"
    },
    "backpack": {
      "position": [
        -1,
        6
      ],
      "facing": "-x"
    },
    "books": {
      "position": [
        0,
        3
      ]
    }
  }
}
🎯 Ground Truth
{
  "backpack": {
    "position": [
      -2,
      6
    ],
    "facing": "north"
  },
  "laptop": {
    "position": [
      -2,
      4
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      -4,
      6
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      0,
      4
    ],
    "facing": "east"
  }
}
🗺️ Symbolic Fog Map
################
#########......#
#########DEF...#
##..L...#......#
##..H...#......#
##......#C...A.#
##B.....*...J..#
##.G....########
##K..I..########
######+#########
#......#########
#......#########
#......#########
#......#########
#......#########
#......#########
################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: B, G, H, I, K
🎯 Ground Truth: B, G, K, E, D, F
🧠 Cognitive Map Metrics
Global:
dir: 0.909
facing: 1
pos: 0.855
overall: 0.921
Global (Full):
dir: 0.417
facing: 0.583
pos: 0.570
overall: 0.523
Local:
dir: 0.600
facing: 1
pos: 0.883
overall: 0.828
Local (Newly):
dir: 0.600
facing: 1
pos: 0.883
overall: 0.828
Fog Probe:
precision: 0.600
recall: 0.500
overall: 0.545
📈 Turn Metrics
node_coverage: 0.643
edge_coverage: 0.220
is_action_fail: False
step: 3
action_counts:
move: 1
rotate: 1
return: 0
observe: 3
term: 0
forced_term: 0
query: 0
information_gain: 0.550
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 3
Current state
State at Turn 3

🔄 Turn 4

💬 Assistant Action
Actions: [JumpTo(books), Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      8,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      5,
      -1
    ],
    "facing": "west"
  },
  "rubberduck": {
    "position": [
      3,
      -2
    ],
    "facing": "north"
  },
  "alarmclock": {
    "position": [
      3,
      2
    ],
    "facing": "west"
  },
  "yellow door": {
    "position": [
      5,
      0
    ]
  },
  "blue door": {
    "position": [
      2,
      -4
    ]
  },
  "laptop": {
    "position": [
      7,
      2
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      5,
      3
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      9,
      3
    ],
    "facing": "north"
  },
  "books": {
    "position": [
      8,
      0
    ]
  }
}
🎯 Ground Truth (Observed)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      10,
      -1
    ],
    "facing": "south"
  }
}
🎯 Ground Truth (Full)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "bike": {
    "position": [
      2,
      -5
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      0,
      -3
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      10,
      -1
    ],
    "facing": "south"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {}
}
🎯 Ground Truth
(no data)
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {}
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
################
#########.....L#
#########.F....#
##.I...D#EB....#
##......#.....A#
##......#.....J#
##......+...*..#
##HK.C..########
##.G....########
######+#########
#......#########
#......#########
#......#########
#......#########
#......#########
#......#########
################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: C, G, H, I, K
🎯 Ground Truth: K, H, G, F, B, E
🧠 Cognitive Map Metrics
Global:
dir: 0.564
facing: 1
pos: 0.685
overall: 0.750
Global (Full):
dir: 0.258
facing: 0.583
pos: 0.457
overall: 0.433
Local:
(none)
Local (Newly):
(none)
Fog Probe:
precision: 0.600
recall: 0.500
overall: 0.545
📈 Turn Metrics
node_coverage: 0.643
edge_coverage: 0.220
is_action_fail: False
step: 4
action_counts:
move: 2
rotate: 2
return: 0
observe: 4
term: 0
forced_term: 0
query: 0
information_gain: 0.550
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 4
Current state
State at Turn 4

🔄 Turn 5

💬 Assistant Action
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      8,
      0
    ],
    "facing": "west"
  },
  "yellow door": {
    "position": [
      5,
      0
    ]
  },
  "blue door": {
    "position": [
      2,
      -4
    ]
  },
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "books": {
    "position": [
      8,
      0
    ]
  },
  "laptop": {
    "position": [
      7,
      0
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      7,
      1
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      6,
      1
    ],
    "facing": "south"
  }
}
🎯 Ground Truth (Observed)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      10,
      -1
    ],
    "facing": "west"
  }
}
🎯 Ground Truth (Full)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "bike": {
    "position": [
      2,
      -5
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      0,
      -3
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      10,
      -1
    ],
    "facing": "west"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "yellow door": {
      "position": [
        0,
        3
      ]
    }
  }
}
🎯 Ground Truth
{
  "yellow door": {
    "position": [
      0,
      4
    ],
    "facing": "east"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "yellow door": {
      "position": [
        0,
        3
      ]
    }
  }
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
################
#########.FK...#
#########L.....#
##.D....#...A..#
##......#......#
##......#.I....#
##H.G...+C..*..#
##.EJ...########
##..B...########
######+#########
#......#########
#......#########
#......#########
#......#########
#......#########
#......#########
################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: B, E, F, H, L
🎯 Ground Truth: H, E, B, K, F, L
🧠 Cognitive Map Metrics
Global:
dir: 0.691
facing: 1
pos: 0.688
overall: 0.793
Global (Full):
dir: 0.317
facing: 0.583
pos: 0.459
overall: 0.453
Local:
dir: 1
facing: 1
pos: 0.883
overall: 0.961
Local (Newly):
(none)
Fog Probe:
precision: 1
recall: 0.833
overall: 0.909
📈 Turn Metrics
node_coverage: 0.643
edge_coverage: 0.220
is_action_fail: False
step: 5
action_counts:
move: 2
rotate: 3
return: 0
observe: 5
term: 0
forced_term: 0
query: 0
information_gain: 0.550
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 5
Current state
State at Turn 5

🔄 Turn 6

💬 Assistant Action
Actions: [JumpTo(yellow door), Rotate(-90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      6,
      -1
    ],
    "facing": "south"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ]
  },
  "blue door": {
    "position": [
      4,
      -4
    ]
  },
  "shoes": {
    "position": [
      5,
      -2
    ],
    "facing": "west"
  },
  "rubberduck": {
    "position": [
      2,
      -2
    ],
    "facing": "north"
  },
  "alarmclock": {
    "position": [
      2,
      2
    ],
    "facing": "west"
  },
  "books": {
    "position": [
      9,
      -1
    ]
  },
  "laptop": {
    "position": [
      9,
      2
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      9,
      3
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      10,
      0
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Observed)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      6,
      -1
    ],
    "facing": "south"
  }
}
🎯 Ground Truth (Full)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "bike": {
    "position": [
      2,
      -5
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      0,
      -3
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      6,
      -1
    ],
    "facing": "south"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "blue door": {
      "position": [
        2,
        3
      ],
      "facing": "-y"
    }
  }
}
🎯 Ground Truth
{
  "blue door": {
    "position": [
      2,
      3
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "blue door": {
      "position": [
        2,
        3
      ],
      "facing": "-y"
    }
  }
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
################
#########.GB...#
#########..I...#
##K.....#......#
##......#C.E...#
##....L.#......#
##......*.D....#
##HJ..A.########
##..F...########
######+#########
#......#########
#......#########
#......#########
#......#########
#......#########
#......#########
################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: B, F, G, H, I, J
🎯 Ground Truth: J, H, F, I, G, B
🧠 Cognitive Map Metrics
Global:
dir: 0.818
facing: 1
pos: 0.835
overall: 0.885
Global (Full):
dir: 0.375
facing: 0.583
pos: 0.557
overall: 0.505
Local:
dir: 1
facing: 1
pos: 1
overall: 1
Local (Newly):
(none)
Fog Probe:
precision: 1
recall: 1
overall: 1
📈 Turn Metrics
node_coverage: 0.643
edge_coverage: 0.244
is_action_fail: False
step: 6
action_counts:
move: 3
rotate: 4
return: 0
observe: 6
term: 0
forced_term: 0
query: 0
information_gain: 0.563
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 6
Current state
State at Turn 6

🔄 Turn 7

💬 Assistant Action
Actions: [JumpTo(blue door), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      3,
      -3
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "west"
  },
  "rubberduck": {
    "position": [
      4,
      -3
    ],
    "facing": "north"
  },
  "alarmclock": {
    "position": [
      3,
      1
    ],
    "facing": "west"
  },
  "yellow door": {
    "position": [
      4,
      -1
    ]
  },
  "blue door": {
    "position": [
      3,
      -3
    ]
  },
  "laptop": {
    "position": [
      6,
      0
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      6,
      1
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      7,
      1
    ],
    "facing": "north"
  },
  "books": {
    "position": [
      7,
      0
    ]
  },
  "shoppingcart": {
    "position": [
      3,
      -7
    ],
    "facing": "south"
  },
  "red chair": {
    "position": [
      1,
      -5
    ],
    "facing": "south"
  }
}
🎯 Ground Truth (Observed)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      -4
    ],
    "facing": "south"
  }
}
🎯 Ground Truth (Full)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "bike": {
    "position": [
      2,
      -5
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      0,
      -3
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      -4
    ],
    "facing": "south"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "shoppingcart": {
      "position": [
        0,
        5
      ],
      "facing": "+y"
    },
    "red chair": {
      "position": [
        2,
        3
      ],
      "facing": "+y"
    }
  }
}
🎯 Ground Truth
{
  "red chair": {
    "position": [
      2,
      3
    ],
    "facing": "south"
  },
  "shoppingcart": {
    "position": [
      0,
      5
    ],
    "facing": "south"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "shoppingcart": {
      "position": [
        0,
        5
      ],
      "facing": "+y"
    },
    "red chair": {
      "position": [
        2,
        3
      ],
      "facing": "+y"
    }
  }
}
🎯 Ground Truth
{
  "red chair": {
    "position": [
      2,
      3
    ],
    "facing": "south"
  },
  "shoppingcart": {
    "position": [
      0,
      5
    ],
    "facing": "south"
  }
}
🗺️ Symbolic Fog Map
################
#########F.Q...#
#########.D....#
##......#...A..#
##......#C.....#
##.K....#......#
##.....O+.....R#
##G.....########
##.HP.L.########
######*#########
#.I.J..#########
#E.....#########
#...M..#########
#.N....#########
#......#########
#....B.#########
################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R
🤖 Predicted: D, E, F, G, H, I
🎯 Ground Truth: G, P, H, I, J, E, Q, F, D
🧠 Cognitive Map Metrics
Global:
dir: 0.654
facing: 1
pos: 0.676
overall: 0.776
Global (Full):
dir: 0.425
facing: 0.750
pos: 0.540
overall: 0.572
Local:
dir: 1
facing: 1
pos: 1
overall: 1
Local (Newly):
dir: 1
facing: 1
pos: 1
overall: 1
Fog Probe:
precision: 1
recall: 0.667
overall: 0.800
📈 Turn Metrics
node_coverage: 0.786
edge_coverage: 0.293
is_action_fail: False
step: 7
action_counts:
move: 4
rotate: 4
return: 0
observe: 7
term: 0
forced_term: 0
query: 0
information_gain: 0.647
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 7
Current state
State at Turn 7

🔄 Turn 8

💬 Assistant Action
Actions: [JumpTo(shoppingcart), Rotate(-90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      3,
      -10
    ],
    "facing": "east"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ]
  },
  "blue door": {
    "position": [
      3,
      -5
    ]
  },
  "books": {
    "position": [
      10,
      -1
    ]
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "shoppingcart": {
    "position": [
      3,
      -10
    ],
    "facing": "south"
  },
  "red chair": {
    "position": [
      1,
      -7
    ],
    "facing": "south"
  }
}
🎯 Ground Truth (Observed)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      -9
    ],
    "facing": "east"
  }
}
🎯 Ground Truth (Full)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "bike": {
    "position": [
      2,
      -5
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      0,
      -3
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      -9
    ],
    "facing": "east"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {}
}
🎯 Ground Truth
(no data)
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {}
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
################
#########PA.M..#
#########......#
##......#L.....#
##......#...K..#
##......#Q.....#
##E...G.+......#
##....J.########
##C.FR..########
######+#########
#D..B..#########
#O.....#########
#......#########
#....H.#########
#.....*#########
#.I..N.#########
################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R
🤖 Predicted: C, E, F, O, P, R
🎯 Ground Truth: E, F, C, D, O, B, A, M, P
🧠 Cognitive Map Metrics
Global:
dir: 0.949
facing: 1
pos: 0.910
overall: 0.953
Global (Full):
dir: 0.617
facing: 0.750
pos: 0.728
overall: 0.698
Local:
(none)
Local (Newly):
(none)
Fog Probe:
precision: 0.833
recall: 0.556
overall: 0.667
📈 Turn Metrics
node_coverage: 0.786
edge_coverage: 0.293
is_action_fail: False
step: 8
action_counts:
move: 5
rotate: 5
return: 0
observe: 8
term: 0
forced_term: 0
query: 0
information_gain: 0.647
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 8
Current state
State at Turn 8

🔄 Turn 9

💬 Assistant Action
Actions: [Rotate(180), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      3,
      -11
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      3,
      2
    ],
    "facing": "west"
  },
  "shoes": {
    "position": [
      5,
      -2
    ],
    "facing": "west"
  },
  "rubberduck": {
    "position": [
      5,
      -4
    ],
    "facing": "north"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      3,
      -6
    ],
    "facing": "south"
  },
  "airplane": {
    "position": [
      7,
      3
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      9,
      3
    ],
    "facing": "north"
  },
  "laptop": {
    "position": [
      9,
      -1
    ],
    "facing": "east"
  },
  "books": {
    "position": [
      11,
      -1
    ]
  },
  "shoppingcart": {
    "position": [
      3,
      -11
    ],
    "facing": "south"
  },
  "red chair": {
    "position": [
      1,
      -9
    ],
    "facing": "south"
  },
  "desklamp": {
    "position": [
      -1,
      -7
    ],
    "facing": "south"
  }
}
🎯 Ground Truth (Observed)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      -9
    ],
    "facing": "west"
  }
}
🎯 Ground Truth (Full)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "bike": {
    "position": [
      2,
      -5
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      0,
      -3
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      -9
    ],
    "facing": "west"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "red chair": {
      "position": [
        2,
        3
      ],
      "facing": "-x"
    },
    "desklamp": {
      "position": [
        1,
        5
      ],
      "facing": "-x"
    }
  }
}
🎯 Ground Truth
{
  "desklamp": {
    "position": [
      1,
      5
    ],
    "facing": "south"
  },
  "red chair": {
    "position": [
      2,
      2
    ],
    "facing": "south"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "red chair": {
      "position": [
        2,
        3
      ],
      "facing": "-x"
    },
    "desklamp": {
      "position": [
        1,
        5
      ],
      "facing": "-x"
    }
  }
}
🎯 Ground Truth
{
  "desklamp": {
    "position": [
      1,
      5
    ],
    "facing": "south"
  }
}
🗺️ Symbolic Fog Map
################
#########.F.L.O#
#########.....E#
##.G....#.K....#
##..MJ..#......#
##......#......#
##......+....A.#
##IB....########
##H.....########
######+#########
#..CN..#########
#......#########
#......#########
#......#########
#.....*#########
#...DP.#########
################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P
🤖 Predicted: B, C, H, I
🎯 Ground Truth: H, B, I, C, N, F, L, K
🧠 Cognitive Map Metrics
Global:
dir: 0.637
facing: 1
pos: 0.726
overall: 0.788
Global (Full):
dir: 0.483
facing: 0.833
pos: 0.629
overall: 0.648
Local:
dir: 0.667
facing: 1
pos: 0.916
overall: 0.861
Local (Newly):
dir: 1
facing: 1
pos: 1
overall: 1
Fog Probe:
precision: 1
recall: 0.500
overall: 0.667
📈 Turn Metrics
node_coverage: 0.857
edge_coverage: 0.341
is_action_fail: False
step: 9
action_counts:
move: 5
rotate: 6
return: 0
observe: 9
term: 0
forced_term: 0
query: 0
information_gain: 0.704
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 9
Current state
State at Turn 9

🔄 Turn 10

💬 Assistant Action
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      4,
      -8
    ],
    "facing": "north"
  },
  "alarmclock": {
    "position": [
      2,
      2
    ],
    "facing": "west"
  },
  "rubberduck": {
    "position": [
      2,
      -2
    ],
    "facing": "north"
  },
  "shoes": {
    "position": [
      2,
      -1
    ],
    "facing": "west"
  },
  "yellow door": {
    "position": [
      5,
      0
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -3
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      8,
      0
    ]
  },
  "laptop": {
    "position": [
      7,
      3
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      6,
      3
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      8,
      2
    ],
    "facing": "north"
  },
  "shoppingcart": {
    "position": [
      4,
      -8
    ],
    "facing": "south"
  },
  "red chair": {
    "position": [
      2,
      -5
    ],
    "facing": "south"
  },
  "desklamp": {
    "position": [
      0,
      -7
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      2,
      -3
    ],
    "facing": "east"
  }
}
🎯 Ground Truth (Observed)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "bike": {
    "position": [
      2,
      -5
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      -9
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Full)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "bike": {
    "position": [
      2,
      -5
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      0,
      -3
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      -9
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "bike": {
      "position": [
        -4,
        5
      ],
      "facing": "+x"
    },
    "red chair": {
      "position": [
        -2,
        3
      ],
      "facing": "-y"
    },
    "blue door": {
      "position": [
        0,
        5
      ],
      "facing": "-y"
    }
  }
}
🎯 Ground Truth
{
  "red chair": {
    "position": [
      -2,
      2
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      -2,
      4
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      0,
      5
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "bike": {
      "position": [
        -4,
        5
      ],
      "facing": "+x"
    },
    "red chair": {
      "position": [
        -2,
        3
      ],
      "facing": "-y"
    },
    "blue door": {
      "position": [
        0,
        5
      ],
      "facing": "-y"
    }
  }
}
🎯 Ground Truth
{
  "bike": {
    "position": [
      -2,
      4
    ],
    "facing": "east"
  }
}
🗺️ Symbolic Fog Map
################
#########E....I#
#########C.K...#
##.....B#......#
##......#.L..D.#
##....G.#......#
##....J.+......#
##FA....########
##..H...########
######+#########
#......#########
#......#########
#......#########
#......#########
#.....*#########
#......#########
################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: A, B, F
🎯 Ground Truth: A, F, H, C, K, E
🧠 Cognitive Map Metrics
Global:
dir: 0.667
facing: 1
pos: 0.728
overall: 0.798
Global (Full):
dir: 0.583
facing: 0.917
pos: 0.679
overall: 0.726
Local:
dir: 0.667
facing: 1
pos: 0.839
overall: 0.835
Local (Newly):
dir: 1
facing: 1
pos: 0.758
overall: 0.919
Fog Probe:
precision: 0.667
recall: 0.333
overall: 0.444
📈 Turn Metrics
node_coverage: 0.929
edge_coverage: 0.366
is_action_fail: False
step: 10
action_counts:
move: 5
rotate: 7
return: 0
observe: 10
term: 0
forced_term: 0
query: 0
information_gain: 0.759
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 10
Current state
State at Turn 10

🔄 Turn 11

💬 Assistant Action
Actions: [JumpTo(blue door), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "grid": "approximate",
  "agent": {
    "position": [
      2,
      -4
    ],
    "facing": "north"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "backward"
  },
  "rubberduck": {
    "position": [
      2,
      -2
    ],
    "facing": "left"
  },
  "alarmclock": {
    "position": [
      1,
      1
    ],
    "facing": "backward"
  },
  "yellow door": {
    "position": [
      4,
      0
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      2,
      -4
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      6,
      2
    ],
    "facing": "forward"
  },
  "airplane": {
    "position": [
      6,
      3
    ],
    "facing": "right"
  },
  "backpack": {
    "position": [
      5,
      2
    ],
    "facing": "left"
  },
  "books": {
    "position": [
      7,
      0
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      2,
      -7
    ],
    "facing": "forward"
  },
  "red chair": {
    "position": [
      0,
      -7
    ],
    "facing": "forward"
  },
  "bike": {
    "position": [
      0,
      -6
    ],
    "facing": "right"
  },
  "desklamp": {
    "position": [
      0,
      -8
    ],
    "facing": "left"
  }
}
🎯 Ground Truth (Observed)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "bike": {
    "position": [
      2,
      -5
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Full)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "bike": {
    "position": [
      2,
      -5
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      0,
      -3
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "shoes": {
      "position": [
        0,
        3
      ],
      "facing": "-x"
    },
    "rubberduck": {
      "position": [
        -2,
        3
      ],
      "facing": "+y"
    },
    "alarmclock": {
      "position": [
        -2,
        5
      ],
      "facing": "-x"
    },
    "yellow door": {
      "position": [
        3,
        3
      ]
    }
  }
}
🎯 Ground Truth
{
  "shoes": {
    "position": [
      0,
      3
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      -2,
      5
    ],
    "facing": "west"
  },
  "yellow door": {
    "position": [
      2,
      3
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      -2,
      3
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "shoes": {
      "position": [
        0,
        3
      ],
      "facing": "-x"
    },
    "rubberduck": {
      "position": [
        -2,
        3
      ],
      "facing": "+y"
    },
    "alarmclock": {
      "position": [
        -2,
        5
      ],
      "facing": "-x"
    },
    "yellow door": {
      "position": [
        3,
        3
      ]
    }
  }
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
################
#########..FEG.#
#########......#
##.IA...#.B....#
##.....K#..L...#
##......#......#
##......+....C.#
##.H....########
##JD....########
######*#########
#......#########
#......#########
#......#########
#......#########
#......#########
#......#########
################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F, G, H, I, J, K, L
🤖 Predicted: I, A
🎯 Ground Truth: J, D, H, E, B, F
🧠 Cognitive Map Metrics
Global:
dir: 0.686
facing: 0.182
pos: 0.669
overall: 0.512
Global (Full):
dir: 0.600
facing: 0.167
pos: 0.624
overall: 0.464
Local:
dir: 0.900
facing: 1
pos: 0.940
overall: 0.947
Local (Newly):
(none)
Fog Probe:
precision: 0
recall: 0
overall: 0
📈 Turn Metrics
node_coverage: 0.929
edge_coverage: 0.439
is_action_fail: False
step: 11
action_counts:
move: 6
rotate: 7
return: 0
observe: 11
term: 0
forced_term: 0
query: 0
information_gain: 0.878
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 11
Current state
State at Turn 11

🔄 Turn 12

💬 Assistant Action
Actions: [Rotate(-90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      2,
      -4
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      2,
      2
    ],
    "facing": "west"
  },
  "yellow door": {
    "position": [
      4,
      0
    ],
    "facing": "west"
  },
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "rubberduck": {
    "position": [
      3,
      -2
    ],
    "facing": "north"
  },
  "blue door": {
    "position": [
      2,
      -4
    ],
    "facing": "north"
  },
  "pan": {
    "position": [
      -2,
      -2
    ],
    "facing": "west"
  },
  "books": {
    "position": [
      7,
      -1
    ]
  },
  "backpack": {
    "position": [
      7,
      2
    ],
    "facing": "north"
  },
  "laptop": {
    "position": [
      6,
      1
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      5,
      2
    ],
    "facing": "south"
  },
  "shoppingcart": {
    "position": [
      2,
      -8
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      0,
      -6
    ],
    "facing": "east"
  },
  "red chair": {
    "position": [
      0,
      -7
    ],
    "facing": "south"
  },
  "desklamp": {
    "position": [
      -1,
      -7
    ],
    "facing": "south"
  }
}
🎯 Ground Truth (Observed)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "bike": {
    "position": [
      2,
      -5
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      0,
      -3
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      -4
    ],
    "facing": "west"
  }
}
🎯 Ground Truth (Full)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "bike": {
    "position": [
      2,
      -5
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      0,
      -3
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      -4
    ],
    "facing": "west"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "bike": {
      "position": [
        -2,
        2
      ],
      "facing": "-y"
    },
    "pan": {
      "position": [
        1,
        5
      ],
      "facing": "+y"
    },
    "desklamp": {
      "position": [
        -3,
        4
      ],
      "facing": "-x"
    }
  }
}
🎯 Ground Truth
{
  "desklamp": {
    "position": [
      -4,
      5
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "pan": {
    "position": [
      1,
      4
    ],
    "facing": "west"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "bike": {
      "position": [
        -2,
        2
      ],
      "facing": "-y"
    },
    "pan": {
      "position": [
        1,
        5
      ],
      "facing": "+y"
    },
    "desklamp": {
      "position": [
        -3,
        4
      ],
      "facing": "-x"
    }
  }
}
🎯 Ground Truth
{
  "pan": {
    "position": [
      1,
      4
    ],
    "facing": "west"
  }
}
🗺️ Symbolic Fog Map
################
#########DF....#
#########.E....#
##......#...A..#
##......#.C....#
##......#B.....#
##......+......#
##......########
##......########
######*#########
#......#########
#......#########
#......#########
#......#########
#......#########
#......#########
################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: (none)
🎯 Ground Truth: F, D, E
🧠 Cognitive Map Metrics
Global:
dir: 0.725
facing: 1
pos: 0.687
overall: 0.804
Global (Full):
dir: 0.725
facing: 1
pos: 0.687
overall: 0.804
Local:
dir: 1
facing: 1
pos: 0.867
overall: 0.956
Local (Newly):
dir: 1
facing: 1
pos: 0.883
overall: 0.961
Fog Probe:
precision: 0
recall: 0
overall: 0
📈 Turn Metrics
node_coverage: 1
edge_coverage: 0.512
is_action_fail: False
step: 12
action_counts:
move: 6
rotate: 8
return: 0
observe: 12
term: 0
forced_term: 0
query: 0
information_gain: 0.927
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 12
Current state
State at Turn 12

🔄 Turn 13

💬 Assistant Action
Actions: [Term()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      0,
      -3
    ],
    "facing": "west"
  },
  "shoes": {
    "position": [
      2,
      -1
    ],
    "facing": "west"
  },
  "rubberduck": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "alarmclock": {
    "position": [
      1,
      1
    ],
    "facing": "west"
  },
  "pan": {
    "position": [
      -2,
      -1
    ],
    "facing": "west"
  },
  "yellow door": {
    "position": [
      3,
      0
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      0,
      -3
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      6,
      0
    ]
  },
  "laptop": {
    "position": [
      6,
      2
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      5,
      3
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      5,
      1
    ],
    "facing": "north"
  },
  "shoppingcart": {
    "position": [
      0,
      -6
    ],
    "facing": "south"
  },
  "red chair": {
    "position": [
      -2,
      -5
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      -2,
      -4
    ],
    "facing": "east"
  },
  "desklamp": {
    "position": [
      -1,
      -5
    ],
    "facing": "south"
  }
}
🎯 Ground Truth (Observed)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "bike": {
    "position": [
      2,
      -5
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      0,
      -3
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      -4
    ],
    "facing": "west"
  }
}
🎯 Ground Truth (Full)
{
  "shoes": {
    "position": [
      4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      -9
    ],
    "facing": "south"
  },
  "laptop": {
    "position": [
      10,
      1
    ],
    "facing": "east"
  },
  "rubberduck": {
    "position": [
      2,
      -1
    ],
    "facing": "north"
  },
  "bike": {
    "position": [
      2,
      -5
    ],
    "facing": "east"
  },
  "airplane": {
    "position": [
      12,
      3
    ],
    "facing": "south"
  },
  "pan": {
    "position": [
      0,
      -3
    ],
    "facing": "west"
  },
  "alarmclock": {
    "position": [
      2,
      1
    ],
    "facing": "west"
  },
  "red chair": {
    "position": [
      2,
      -7
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      12,
      1
    ],
    "facing": "north"
  },
  "desklamp": {
    "position": [
      -1,
      -8
    ],
    "facing": "south"
  },
  "books": {
    "position": [
      10,
      -1
    ],
    "facing": "east"
  },
  "yellow door": {
    "position": [
      6,
      -1
    ],
    "facing": "east"
  },
  "blue door": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      -4
    ],
    "facing": "west"
  }
}
🧠 Cognitive Map Metrics
Global:
dir: 0.658
facing: 1
pos: 0.595
overall: 0.751
Global (Full):
dir: 0.658
facing: 1
pos: 0.595
overall: 0.751
Local:
(none)
Local (Newly):
(none)
Fog Probe:
(none)
📈 Turn Metrics
node_coverage: 1
edge_coverage: 0.512
is_action_fail: False
step: 13
action_counts:
move: 6
rotate: 8
return: 0
observe: 12
term: 1
forced_term: 0
query: 0
information_gain: 0.927
reward: -0.100
is_done: True
is_valid_action: True
Previous state
State before Turn 13
Current state
State at Turn 13

🧭 False Belief Exploration

🔄 FB Turn 14

💬 Assistant Action
Actions: [Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Unchanged (all):
dir: 0.667
facing: 1
pos: 0.766
overall: 0.811
🧾 False Belief Object Sets
all_objects: ['airplane', 'alarmclock', 'backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'pan', 'red chair', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
all_changed_objects: ['airplane', 'alarmclock', 'pan', 'red chair']
newly_observed_changed (this turn): []
unchanged_objects: ['backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
pred_changed_keys: []
pred_unchanged_keys: ['airplane', 'alarmclock', 'backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'pan', 'red chair', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
🧭 False Belief Info
step: 1
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'red chair', 'ori': True}, {'name': 'pan', 'ori': True}, {'name': 'alarmclock', 'pos': True}, {'name': 'airplane', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: []
newly_observed_unchanged_objects: []
Current state
State at Turn 1

🔄 FB Turn 15

💬 Assistant Action
Actions: [Rotate(90), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Changed: airplane:
dir: None
pos: 0.839
facing: None
overall: None
Retention: airplane:
dir: None
pos: 0.416
facing: None
overall: None
Unchanged (all):
dir: 0.152
facing: 0.375
pos: 0.389
overall: 0.305
🧾 False Belief Object Sets
all_objects: ['airplane', 'alarmclock', 'backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'pan', 'red chair', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
all_changed_objects: ['airplane', 'alarmclock', 'pan', 'red chair']
newly_observed_changed (this turn): ['airplane']
unchanged_objects: ['backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
pred_changed_keys: ['airplane']
pred_unchanged_keys: ['airplane', 'blue door', 'rubberduck', 'shoes', 'yellow door']
🧭 False Belief Info
step: 2
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'red chair', 'ori': True}, {'name': 'pan', 'ori': True}, {'name': 'alarmclock', 'pos': True}, {'name': 'airplane', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: ['airplane']
newly_observed_unchanged_objects: ['yellow door', 'rubberduck', 'shoes', 'blue door']
Previous state
State before Turn 2
Current state
State at Turn 2

🔄 FB Turn 16

💬 Assistant Action
Actions: [JumpTo(yellow door), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Unchanged (all):
dir: 0.652
facing: 1
pos: 0.766
overall: 0.806
🧾 False Belief Object Sets
all_objects: ['airplane', 'alarmclock', 'backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'pan', 'red chair', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
all_changed_objects: ['airplane', 'alarmclock', 'pan', 'red chair']
newly_observed_changed (this turn): []
unchanged_objects: ['backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
pred_changed_keys: []
pred_unchanged_keys: ['airplane', 'alarmclock', 'backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'pan', 'red chair', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
🧭 False Belief Info
step: 3
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'red chair', 'ori': True}, {'name': 'pan', 'ori': True}, {'name': 'alarmclock', 'pos': True}, {'name': 'airplane', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: []
newly_observed_unchanged_objects: ['books', 'backpack', 'laptop']
Previous state
State before Turn 3
Current state
State at Turn 3

🔄 FB Turn 17

💬 Assistant Action
Actions: [Rotate(180), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Changed: pan:
dir: None
pos: None
facing: 1
overall: None
Retention: pan:
dir: None
pos: None
facing: 0
overall: None
Unchanged (all):
dir: 0.652
facing: 0.875
pos: 0.520
overall: 0.682
🧾 False Belief Object Sets
all_objects: ['airplane', 'alarmclock', 'backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'pan', 'red chair', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
all_changed_objects: ['airplane', 'alarmclock', 'pan', 'red chair']
newly_observed_changed (this turn): ['pan']
unchanged_objects: ['backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
pred_changed_keys: ['pan']
pred_unchanged_keys: ['airplane', 'alarmclock', 'backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'pan', 'red chair', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
🧭 False Belief Info
step: 4
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'red chair', 'ori': True}, {'name': 'pan', 'ori': True}, {'name': 'alarmclock', 'pos': True}, {'name': 'airplane', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: ['pan']
newly_observed_unchanged_objects: []
Previous state
State before Turn 4
Current state
State at Turn 4

🔄 FB Turn 18

💬 Assistant Action
Actions: [Rotate(-90), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Unchanged (all):
dir: 0.667
facing: 0.250
pos: 0.668
overall: 0.528
🧾 False Belief Object Sets
all_objects: ['airplane', 'alarmclock', 'backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'pan', 'red chair', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
all_changed_objects: ['airplane', 'alarmclock', 'pan', 'red chair']
newly_observed_changed (this turn): []
unchanged_objects: ['backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
pred_changed_keys: []
pred_unchanged_keys: ['airplane', 'alarmclock', 'backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'pan', 'red chair', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
🧭 False Belief Info
step: 5
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'red chair', 'ori': True}, {'name': 'pan', 'ori': True}, {'name': 'alarmclock', 'pos': True}, {'name': 'airplane', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: []
newly_observed_unchanged_objects: []
Previous state
State before Turn 5
Current state
State at Turn 5

🔄 FB Turn 19

💬 Assistant Action
Actions: [JumpTo(blue door), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Changed: red chair:
dir: None
pos: None
facing: 1
overall: None
Retention: red chair:
dir: None
pos: None
facing: 0
overall: None
Changed: alarmclock:
dir: None
pos: 0.758
facing: None
overall: None
Retention: alarmclock:
dir: None
pos: 0.325
facing: None
overall: None
Unchanged (all):
dir: 0.773
facing: 1
pos: 0.733
overall: 0.835
🧾 False Belief Object Sets
all_objects: ['airplane', 'alarmclock', 'backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'pan', 'red chair', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
all_changed_objects: ['airplane', 'alarmclock', 'pan', 'red chair']
newly_observed_changed (this turn): ['alarmclock', 'red chair']
unchanged_objects: ['backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
pred_changed_keys: ['alarmclock', 'red chair']
pred_unchanged_keys: ['airplane', 'alarmclock', 'backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'pan', 'red chair', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
🧭 False Belief Info
step: 6
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'red chair', 'ori': True}, {'name': 'pan', 'ori': True}, {'name': 'alarmclock', 'pos': True}, {'name': 'airplane', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: ['red chair', 'alarmclock']
newly_observed_unchanged_objects: ['shoppingcart']
Previous state
State before Turn 6
Current state
State at Turn 6

🔄 FB Turn 20

💬 Assistant Action
Actions: [Query(pan)]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Unchanged (all):
dir: 0.697
facing: 1
pos: 0.668
overall: 0.788
🧾 False Belief Object Sets
all_objects: ['airplane', 'alarmclock', 'backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'pan', 'red chair', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
all_changed_objects: ['airplane', 'alarmclock', 'pan', 'red chair']
newly_observed_changed (this turn): []
unchanged_objects: ['backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
pred_changed_keys: []
pred_unchanged_keys: ['airplane', 'alarmclock', 'backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'pan', 'red chair', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
🧭 False Belief Info
step: 7
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'red chair', 'ori': True}, {'name': 'pan', 'ori': True}, {'name': 'alarmclock', 'pos': True}, {'name': 'airplane', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: []
newly_observed_unchanged_objects: []
Previous state
State before Turn 7
Current state
State at Turn 7

🔄 FB Turn 21

💬 Assistant Action
Actions: [Rotate(90), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Unchanged (all):
dir: 0.652
facing: 1
pos: 0.550
overall: 0.734
🧾 False Belief Object Sets
all_objects: ['airplane', 'alarmclock', 'backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'pan', 'red chair', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
all_changed_objects: ['airplane', 'alarmclock', 'pan', 'red chair']
newly_observed_changed (this turn): []
unchanged_objects: ['backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
pred_changed_keys: []
pred_unchanged_keys: ['airplane', 'alarmclock', 'backpack', 'bike', 'blue door', 'books', 'desklamp', 'laptop', 'pan', 'red chair', 'rubberduck', 'shoes', 'shoppingcart', 'yellow door']
🧭 False Belief Info
step: 8
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'red chair', 'ori': True}, {'name': 'pan', 'ori': True}, {'name': 'alarmclock', 'pos': True}, {'name': 'airplane', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: []
newly_observed_unchanged_objects: ['desklamp', 'bike']
Previous state
State before Turn 8
Current state
State at Turn 8

🔄 FB Turn 22

💬 Assistant Action
Actions: [Term(changes="airplane: position, alarmclock: position, red chair: orientation, pan: orientation")]
📈 Turn Metrics
reward: 1
is_done: True
is_valid_action: True
success: 1
🧭 False Belief Info
step: 9
correctly_identified_changes: 1
f1_overall: 1
f1_position: 1
f1_facing: 1
action_cost: 7
action_cost_after_seen: 1
ground_truth_changes: [{'name': 'red chair', 'ori': True}, {'name': 'pan', 'ori': True}, {'name': 'alarmclock', 'pos': True}, {'name': 'airplane', 'pos': True}]
reported_changes: [{'name': 'airplane', 'pos': True}, {'name': 'alarmclock', 'pos': True}, {'name': 'red chair', 'ori': True}, {'name': 'pan', 'ori': True}]
newly_observed_changed_objects: []
newly_observed_unchanged_objects: []
Previous state
State before Turn 9
Current state
State at Turn 9

📊 Evaluation (default)

📊 Task: DirectionEvaluationTask

Question 1 (ID: fd2887aae65af114)

✅ Evaluation Results
id: fd2887aae65af114
question: You return to your starting position and face north. From a Top-Down map, describe where backpack is relative to books. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north east, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q1: DirectionEvaluationTask

Question 2 (ID: 47e25a4138337ec9)

✅ Evaluation Results
id: 47e25a4138337ec9
question: You return to your starting position and face north. From a Top-Down map, describe where books is relative to pan. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: east, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Evaluation state
Q2: DirectionEvaluationTask

Question 3 (ID: 05aacd3852716124)

✅ Evaluation Results
id: 05aacd3852716124
question: You return to your starting position and face north. From a Top-Down map, describe where desklamp is relative to books. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: south west, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q3: DirectionEvaluationTask

📊 Task: PovEvaluationTask

Question 1 (ID: dd32b518994cc0cd)

✅ Evaluation Results
id: dd32b518994cc0cd
question: Now you jump to rubberduck's direction, facing its direction. Describe where alarmclock is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front, near
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Evaluation state
Q1: PovEvaluationTask

Question 2 (ID: b3bd1bdceefdf76a)

✅ Evaluation Results
id: b3bd1bdceefdf76a
question: Now you jump to red chair's direction, facing its direction. Describe where shoppingcart is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q2: PovEvaluationTask

Question 3 (ID: 9f4070bd1a76781d)

✅ Evaluation Results
id: 9f4070bd1a76781d
question: Now you jump to laptop's direction, facing its direction. Describe where backpack is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front, near
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Evaluation state
Q3: PovEvaluationTask

📊 Task: BackwardPovTextEvaluationTask

Question 1 (ID: e0bba9ba020fcf43)

✅ Evaluation Results
id: e0bba9ba020fcf43
question: Now you jump to an object's position, facing its direction. You observe that alarmclock is front, near, facing left. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: rubberduck
final_pos: [4, 10]
final_ori: [0, 1]
final_observation: [{'name': 'alarmclock', 'direction': 'front', 'distance': 'near', 'orientation': 'facing left'}]
object_positions:
shoes: [6.0, 10.0]
shoppingcart: [6.0, 2.0]
laptop: [12.0, 12.0]
rubberduck: [4.0, 10.0]
bike: [4.0, 6.0]
airplane: [14.0, 14.0]
pan: [2.0, 8.0]
alarmclock: [4.0, 12.0]
red chair: [4.0, 4.0]
backpack: [14.0, 12.0]
desklamp: [1.0, 3.0]
books: [12.0, 10.0]
yellow door: [8.0, 10.0]
blue door: [6.0, 7.0]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
room_id: 1
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: default
Evaluation state
Q1: BackwardPovTextEvaluationTask

Question 2 (ID: 5ee18081cd74702f)

✅ Evaluation Results
id: 5ee18081cd74702f
question: Now you jump to an object's position, facing its direction. You observe that airplane is front-left, mid distance, facing right; backpack is front, near, facing left. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: laptop
final_pos: [12, 12]
final_ori: [1, 0]
final_observation: [{'name': 'airplane', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'backpack', 'direction': 'front', 'distance': 'near', 'orientation': 'facing left'}]
object_positions:
shoes: [6.0, 10.0]
shoppingcart: [6.0, 2.0]
laptop: [12.0, 12.0]
rubberduck: [4.0, 10.0]
bike: [4.0, 6.0]
airplane: [14.0, 14.0]
pan: [2.0, 8.0]
alarmclock: [4.0, 12.0]
red chair: [4.0, 4.0]
backpack: [14.0, 12.0]
desklamp: [1.0, 3.0]
books: [12.0, 10.0]
yellow door: [8.0, 10.0]
blue door: [6.0, 7.0]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
room_id: 3
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: default
Evaluation state
Q2: BackwardPovTextEvaluationTask

Question 3 (ID: 48ef54e966ae6b2b)

✅ Evaluation Results
id: 48ef54e966ae6b2b
question: Now you jump to an object's position, facing its direction. You observe that blue door is front-left, mid distance, on left wall. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: bike
final_pos: [4, 6]
final_ori: [1, 0]
final_observation: [{'name': 'blue door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on left wall'}]
object_positions:
shoes: [6.0, 10.0]
shoppingcart: [6.0, 2.0]
laptop: [12.0, 12.0]
rubberduck: [4.0, 10.0]
bike: [4.0, 6.0]
airplane: [14.0, 14.0]
pan: [2.0, 8.0]
alarmclock: [4.0, 12.0]
red chair: [4.0, 4.0]
backpack: [14.0, 12.0]
desklamp: [1.0, 3.0]
books: [12.0, 10.0]
yellow door: [8.0, 10.0]
blue door: [6.0, 7.0]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
room_id: 2
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: default
Evaluation state
Q3: BackwardPovTextEvaluationTask

📊 Task: Action2ViewEvaluationTask

Question 1 (ID: 746bd0e0e6ba5a86)

✅ Evaluation Results
id: 746bd0e0e6ba5a86
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(90) 2. Jump to the object at front-left, mid distance. 3. Jump to the object at front-right, slightly far. 4. Rotate(-180) After executing the actions, what is the ego relation of shoes relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front, near
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Evaluation state
Q1: Action2ViewEvaluationTask

Question 2 (ID: dd857cdb010f8d2c)

✅ Evaluation Results
id: dd857cdb010f8d2c
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(90) 2. Jump to the object at front-right, slightly far. 3. Rotate(-180) 4. Jump to the object at front-left, mid distance. After executing the actions, what is the ego relation of desklamp relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Evaluation state
Q2: Action2ViewEvaluationTask

Question 3 (ID: f648d1cc302e836a)

✅ Evaluation Results
id: f648d1cc302e836a
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(90) 2. Jump to the object at front-right, mid distance. 3. Jump to the object at front, mid distance. 4. Rotate(-180) After executing the actions, what is the ego relation of shoes relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front, near
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Evaluation state
Q3: Action2ViewEvaluationTask

📊 Task: View2ActionTextEvaluationTask

Question 1 (ID: a64c44dde1456896)

✅ Evaluation Results
id: a64c44dde1456896
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: bike is at front-right, slightly far, facing right What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [1, 3]
final_ori: [0, 1]
room_id: 2
init_pos: [2, 11]
init_ori: [0, 1]
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
minimal_plan: [['rotate', 90], ['jumpto', 'blue door'], ['rotate', 180], ['jumpto', 'desklamp'], ['rotate', 90]]
final_observation: [{'name': 'bike', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing right'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [1.0, 3.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: default
Evaluation state
Q1: View2ActionTextEvaluationTask

Question 2 (ID: c61a5d692c908db6)

✅ Evaluation Results
id: c61a5d692c908db6
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: bike is at front, near, facing right; blue door is at front-right, mid distance, on front wall What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [4, 4]
final_ori: [0, 1]
room_id: 2
init_pos: [2, 11]
init_ori: [0, 1]
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
minimal_plan: [['rotate', 180], ['jumpto', 'blue door'], ['jumpto', 'red chair'], ['rotate', 180]]
final_observation: [{'name': 'bike', 'direction': 'front', 'distance': 'near', 'orientation': 'facing right'}, {'name': 'blue door', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'on front wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [4.0, 4.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: default
Evaluation state
Q2: View2ActionTextEvaluationTask

Question 3 (ID: 84f10bed2583426f)

✅ Evaluation Results
id: 84f10bed2583426f
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: airplane is at front-left, slightly far, facing right; backpack is at front-slight-left, slightly far, facing left; laptop is at front-left, slightly far, facing forward What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [8, 10]
final_ori: [1, 0]
room_id: [1, 3]
init_pos: [2, 11]
init_ori: [0, 1]
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
minimal_plan: [['rotate', 90], ['jumpto', 'initial_pos']]
final_observation: [{'name': 'airplane', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'backpack', 'direction': 'front-slight-left', 'distance': 'slightly far', 'orientation': 'facing left'}, {'name': 'laptop', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing forward'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [8.0, 10.0]
final_ori: [1, 0]
visible_match: True
score: 1
evaluation_mode: default
Evaluation state
Q3: View2ActionTextEvaluationTask

📊 Task: AlloMappingEvaluationTask

Question 1 (ID: d07c09cead29a5ab)

✅ Evaluation Results
id: d07c09cead29a5ab
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: backpack, books, shoppingcart, rubberduck. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[12, 1], [10, -1], [4, -9], [2, -1]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.804
threshold: 0.900
score: 0.804
evaluation_mode: default
Evaluation state
Q1: AlloMappingEvaluationTask

Question 2 (ID: c301d9dd74a52e57)

✅ Evaluation Results
id: c301d9dd74a52e57
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: laptop, alarmclock, desklamp, rubberduck, red chair. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[10, 1], [2, 1], [-1, -8], [2, -1], [2, -7]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.811
threshold: 0.900
score: 0.811
evaluation_mode: default
Evaluation state
Q2: AlloMappingEvaluationTask

Question 3 (ID: 4ade2017a10e41fb)

✅ Evaluation Results
id: 4ade2017a10e41fb
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: books, shoppingcart, laptop, alarmclock, bike. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[10, -1], [4, -9], [10, 1], [2, 1], [2, -5]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.909
threshold: 0.900
score: 0.909
evaluation_mode: default
Evaluation state
Q3: AlloMappingEvaluationTask

📊 Task: RotEvaluationTask

Question 1 (ID: d41a4cb1c4ae88e8)

✅ Evaluation Results
id: d41a4cb1c4ae88e8
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning counterclockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: desklamp, rubberduck, shoppingcart. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['desklamp', 'shoppingcart', 'rubberduck']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q1: RotEvaluationTask

Question 2 (ID: 35aed4a89a7998dd)

✅ Evaluation Results
id: 35aed4a89a7998dd
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: backpack, pan, rubberduck. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['backpack', 'rubberduck', 'pan']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q2: RotEvaluationTask

Question 3 (ID: 070834d55928c793)

✅ Evaluation Results
id: 070834d55928c793
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: backpack, rubberduck, shoppingcart. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['backpack', 'rubberduck', 'shoppingcart']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q3: RotEvaluationTask

📊 Task: RotDualEvaluationTask

Question 1 (ID: 51578be02150882b)

✅ Evaluation Results
id: 51578be02150882b
question: You return to your starting position and face north. You performed a complete 360° rotation in place. Assume all walls are removed (you can see through walls), so every object is visible. During the rotation, these objects appeared directly in front of you in this order: desklamp, shoppingcart, rubberduck Based on this sequence, in which direction did you rotate? Answer format: clockwise or counterclockwise Example: clockwise
action: None
answer: counterclockwise
task_type: RotDualEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q1: RotDualEvaluationTask

Question 2 (ID: a232cb4d3c853647)

✅ Evaluation Results
id: a232cb4d3c853647
question: You return to your starting position and face north. You performed a complete 360° rotation in place. Assume all walls are removed (you can see through walls), so every object is visible. During the rotation, these objects appeared directly in front of you in this order: backpack, rubberduck, pan Based on this sequence, in which direction did you rotate? Answer format: clockwise or counterclockwise Example: clockwise
action: None
answer: clockwise
task_type: RotDualEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q2: RotDualEvaluationTask

Question 3 (ID: 83fd00114d015923)

✅ Evaluation Results
id: 83fd00114d015923
question: You return to your starting position and face north. You performed a complete 360° rotation in place. Assume all walls are removed (you can see through walls), so every object is visible. During the rotation, these objects appeared directly in front of you in this order: backpack, rubberduck, shoppingcart Based on this sequence, in which direction did you rotate? Answer format: clockwise or counterclockwise Example: clockwise
action: None
answer: clockwise
task_type: RotDualEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q3: RotDualEvaluationTask

📊 Task: Location2ViewEvaluationTask

Question 1 (ID: 12e5714b691ffd9d)

✅ Evaluation Results
id: 12e5714b691ffd9d
question: Still treat your initial position as origin (0, 0) You move to (4, -2) and face north. What is the egocentric relation of shoes? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front, near
task_type: Action2LocationEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q1: Location2ViewEvaluationTask

Question 2 (ID: 37203b723cb2ed85)

✅ Evaluation Results
id: 37203b723cb2ed85
question: Still treat your initial position as origin (0, 0) You move to (4, 2) and face south. What is the egocentric relation of rubberduck? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-right, mid distance
task_type: Action2LocationEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: default
Evaluation state
Q2: Location2ViewEvaluationTask

Question 3 (ID: 1e0d4b92e81bdd63)

✅ Evaluation Results
id: 1e0d4b92e81bdd63
question: Still treat your initial position as origin (0, 0) You move to (5, 0) and face west. What is the egocentric relation of pan? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2LocationEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Evaluation state
Q3: Location2ViewEvaluationTask

📊 Task: View2LocationTextEvaluationTask

Question 1 (ID: 7eb7d98b38d92500)

✅ Evaluation Results
id: 7eb7d98b38d92500
question: You move to a new location and face north. You observe: shoes is at front, near, facing left; alarmclock is at front-left, mid distance, facing left Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [4, -2]
final_pos: [6, 9]
final_ori: [0, 1]
room_id: 1
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'shoes', 'direction': 'front', 'distance': 'near', 'orientation': 'facing left'}, {'name': 'alarmclock', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing left'}]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: Location2ActionTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: default
Evaluation state
Q1: View2LocationTextEvaluationTask

Question 2 (ID: 4190e9e80fa5f994)

✅ Evaluation Results
id: 4190e9e80fa5f994
question: You move to a new location and face south. You observe: rubberduck is at front-right, mid distance, facing backward; blue door is at front, slightly far, on front wall; pan is at front-right, slightly far, facing right Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [4, 2]
final_pos: [6, 13]
final_ori: [0, -1]
room_id: 1
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'rubberduck', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'blue door', 'direction': 'front', 'distance': 'slightly far', 'orientation': 'on front wall'}, {'name': 'pan', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing right'}]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: Location2ActionTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.800
raw_score: 0.800
score: 0.800
evaluation_mode: default
Evaluation state
Q2: View2LocationTextEvaluationTask

Question 3 (ID: e006710c90936706)

✅ Evaluation Results
id: e006710c90936706
question: You move to a new location and face west. You observe: pan is at front-left, slightly far, facing forward; alarmclock is at front-slight-right, mid distance, facing forward; rubberduck is at front-slight-left, mid distance, facing right Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [5, 0]
final_pos: [7, 11]
final_ori: [-1, 0]
room_id: 1
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'pan', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing forward'}, {'name': 'alarmclock', 'direction': 'front-slight-right', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'rubberduck', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing right'}]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: Location2ActionTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: default
Evaluation state
Q3: View2LocationTextEvaluationTask

📊 Evaluation (prompt_cogmap)

📊 Task: DirectionEvaluationTask

Question 1 (ID: fd2887aae65af114)

✅ Evaluation Results
id: fd2887aae65af114
question: You return to your starting position and face north. From a Top-Down map, describe where backpack is relative to books. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north east, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Evaluation state
Q1: DirectionEvaluationTask

Question 2 (ID: 47e25a4138337ec9)

✅ Evaluation Results
id: 47e25a4138337ec9
question: You return to your starting position and face north. From a Top-Down map, describe where books is relative to pan. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: east, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q2: DirectionEvaluationTask

Question 3 (ID: 05aacd3852716124)

✅ Evaluation Results
id: 05aacd3852716124
question: You return to your starting position and face north. From a Top-Down map, describe where desklamp is relative to books. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: south west, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q3: DirectionEvaluationTask

📊 Task: PovEvaluationTask

Question 1 (ID: dd32b518994cc0cd)

✅ Evaluation Results
id: dd32b518994cc0cd
question: Now you jump to rubberduck's direction, facing its direction. Describe where alarmclock is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front, near
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: prompt_cogmap
Evaluation state
Q1: PovEvaluationTask

Question 2 (ID: b3bd1bdceefdf76a)

✅ Evaluation Results
id: b3bd1bdceefdf76a
question: Now you jump to red chair's direction, facing its direction. Describe where shoppingcart is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q2: PovEvaluationTask

Question 3 (ID: 9f4070bd1a76781d)

✅ Evaluation Results
id: 9f4070bd1a76781d
question: Now you jump to laptop's direction, facing its direction. Describe where backpack is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front, near
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Evaluation state
Q3: PovEvaluationTask

📊 Task: BackwardPovTextEvaluationTask

Question 1 (ID: e0bba9ba020fcf43)

✅ Evaluation Results
id: e0bba9ba020fcf43
question: Now you jump to an object's position, facing its direction. You observe that alarmclock is front, near, facing left. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: rubberduck
final_pos: [4, 10]
final_ori: [0, 1]
final_observation: [{'name': 'alarmclock', 'direction': 'front', 'distance': 'near', 'orientation': 'facing left'}]
object_positions:
shoes: [6.0, 10.0]
shoppingcart: [6.0, 2.0]
laptop: [12.0, 12.0]
rubberduck: [4.0, 10.0]
bike: [4.0, 6.0]
airplane: [14.0, 14.0]
pan: [2.0, 8.0]
alarmclock: [4.0, 12.0]
red chair: [4.0, 4.0]
backpack: [14.0, 12.0]
desklamp: [1.0, 3.0]
books: [12.0, 10.0]
yellow door: [8.0, 10.0]
blue door: [6.0, 7.0]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
room_id: 1
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q1: BackwardPovTextEvaluationTask

Question 2 (ID: 5ee18081cd74702f)

✅ Evaluation Results
id: 5ee18081cd74702f
question: Now you jump to an object's position, facing its direction. You observe that airplane is front-left, mid distance, facing right; backpack is front, near, facing left. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: laptop
final_pos: [12, 12]
final_ori: [1, 0]
final_observation: [{'name': 'airplane', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'backpack', 'direction': 'front', 'distance': 'near', 'orientation': 'facing left'}]
object_positions:
shoes: [6.0, 10.0]
shoppingcart: [6.0, 2.0]
laptop: [12.0, 12.0]
rubberduck: [4.0, 10.0]
bike: [4.0, 6.0]
airplane: [14.0, 14.0]
pan: [2.0, 8.0]
alarmclock: [4.0, 12.0]
red chair: [4.0, 4.0]
backpack: [14.0, 12.0]
desklamp: [1.0, 3.0]
books: [12.0, 10.0]
yellow door: [8.0, 10.0]
blue door: [6.0, 7.0]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
room_id: 3
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q2: BackwardPovTextEvaluationTask

Question 3 (ID: 48ef54e966ae6b2b)

✅ Evaluation Results
id: 48ef54e966ae6b2b
question: Now you jump to an object's position, facing its direction. You observe that blue door is front-left, mid distance, on left wall. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: bike
final_pos: [4, 6]
final_ori: [1, 0]
final_observation: [{'name': 'blue door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on left wall'}]
object_positions:
shoes: [6.0, 10.0]
shoppingcart: [6.0, 2.0]
laptop: [12.0, 12.0]
rubberduck: [4.0, 10.0]
bike: [4.0, 6.0]
airplane: [14.0, 14.0]
pan: [2.0, 8.0]
alarmclock: [4.0, 12.0]
red chair: [4.0, 4.0]
backpack: [14.0, 12.0]
desklamp: [1.0, 3.0]
books: [12.0, 10.0]
yellow door: [8.0, 10.0]
blue door: [6.0, 7.0]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
room_id: 2
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q3: BackwardPovTextEvaluationTask

📊 Task: Action2ViewEvaluationTask

Question 1 (ID: dd857cdb010f8d2c)

✅ Evaluation Results
id: dd857cdb010f8d2c
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(90) 2. Jump to the object at front-right, slightly far. 3. Rotate(-180) 4. Jump to the object at front-left, mid distance. After executing the actions, what is the ego relation of desklamp relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Evaluation state
Q1: Action2ViewEvaluationTask

Question 2 (ID: f648d1cc302e836a)

✅ Evaluation Results
id: f648d1cc302e836a
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(90) 2. Jump to the object at front-right, mid distance. 3. Jump to the object at front, mid distance. 4. Rotate(-180) After executing the actions, what is the ego relation of shoes relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front, near
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: prompt_cogmap
Evaluation state
Q2: Action2ViewEvaluationTask

Question 3 (ID: 746bd0e0e6ba5a86)

✅ Evaluation Results
id: 746bd0e0e6ba5a86
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(90) 2. Jump to the object at front-left, mid distance. 3. Jump to the object at front-right, slightly far. 4. Rotate(-180) After executing the actions, what is the ego relation of shoes relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front, near
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Evaluation state
Q3: Action2ViewEvaluationTask

📊 Task: View2ActionTextEvaluationTask

Question 1 (ID: a64c44dde1456896)

✅ Evaluation Results
id: a64c44dde1456896
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: bike is at front-right, slightly far, facing right What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [1, 3]
final_ori: [0, 1]
room_id: 2
init_pos: [2, 11]
init_ori: [0, 1]
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
minimal_plan: [['rotate', 90], ['jumpto', 'blue door'], ['rotate', 180], ['jumpto', 'desklamp'], ['rotate', 90]]
final_observation: [{'name': 'bike', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing right'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [1.0, 3.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q1: View2ActionTextEvaluationTask

Question 2 (ID: c61a5d692c908db6)

✅ Evaluation Results
id: c61a5d692c908db6
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: bike is at front, near, facing right; blue door is at front-right, mid distance, on front wall What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [4, 4]
final_ori: [0, 1]
room_id: 2
init_pos: [2, 11]
init_ori: [0, 1]
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
minimal_plan: [['rotate', 180], ['jumpto', 'blue door'], ['jumpto', 'red chair'], ['rotate', 180]]
final_observation: [{'name': 'bike', 'direction': 'front', 'distance': 'near', 'orientation': 'facing right'}, {'name': 'blue door', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'on front wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [4.0, 4.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q2: View2ActionTextEvaluationTask

Question 3 (ID: 84f10bed2583426f)

✅ Evaluation Results
id: 84f10bed2583426f
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: airplane is at front-left, slightly far, facing right; backpack is at front-slight-left, slightly far, facing left; laptop is at front-left, slightly far, facing forward What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [8, 10]
final_ori: [1, 0]
room_id: [1, 3]
init_pos: [2, 11]
init_ori: [0, 1]
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
minimal_plan: [['rotate', 90], ['jumpto', 'initial_pos']]
final_observation: [{'name': 'airplane', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'backpack', 'direction': 'front-slight-left', 'distance': 'slightly far', 'orientation': 'facing left'}, {'name': 'laptop', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing forward'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: True
final_pos: [12.0, 10.0]
final_ori: [1, 0]
visible_match: False
score: 0
evaluation_mode: prompt_cogmap
Evaluation state
Q3: View2ActionTextEvaluationTask

📊 Task: AlloMappingEvaluationTask

Question 1 (ID: d07c09cead29a5ab)

✅ Evaluation Results
id: d07c09cead29a5ab
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: backpack, books, shoppingcart, rubberduck. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[12, 1], [10, -1], [4, -9], [2, -1]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.809
threshold: 0.900
score: 0.809
evaluation_mode: prompt_cogmap
Evaluation state
Q1: AlloMappingEvaluationTask

Question 2 (ID: c301d9dd74a52e57)

✅ Evaluation Results
id: c301d9dd74a52e57
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: laptop, alarmclock, desklamp, rubberduck, red chair. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[10, 1], [2, 1], [-1, -8], [2, -1], [2, -7]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.827
threshold: 0.900
score: 0.827
evaluation_mode: prompt_cogmap
Evaluation state
Q2: AlloMappingEvaluationTask

Question 3 (ID: 4ade2017a10e41fb)

✅ Evaluation Results
id: 4ade2017a10e41fb
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: books, shoppingcart, laptop, alarmclock, bike. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[10, -1], [4, -9], [10, 1], [2, 1], [2, -5]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.835
threshold: 0.900
score: 0.835
evaluation_mode: prompt_cogmap
Evaluation state
Q3: AlloMappingEvaluationTask

📊 Task: RotEvaluationTask

Question 1 (ID: d41a4cb1c4ae88e8)

✅ Evaluation Results
id: d41a4cb1c4ae88e8
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning counterclockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: desklamp, rubberduck, shoppingcart. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['desklamp', 'shoppingcart', 'rubberduck']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q1: RotEvaluationTask

Question 2 (ID: 35aed4a89a7998dd)

✅ Evaluation Results
id: 35aed4a89a7998dd
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: backpack, pan, rubberduck. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['backpack', 'rubberduck', 'pan']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q2: RotEvaluationTask

Question 3 (ID: 070834d55928c793)

✅ Evaluation Results
id: 070834d55928c793
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: backpack, rubberduck, shoppingcart. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['backpack', 'rubberduck', 'shoppingcart']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q3: RotEvaluationTask

📊 Task: Location2ViewEvaluationTask

Question 1 (ID: 12e5714b691ffd9d)

✅ Evaluation Results
id: 12e5714b691ffd9d
question: Still treat your initial position as origin (0, 0) You move to (4, -2) and face north. What is the egocentric relation of shoes? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front, near
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Evaluation state
Q1: Location2ViewEvaluationTask

Question 2 (ID: 37203b723cb2ed85)

✅ Evaluation Results
id: 37203b723cb2ed85
question: Still treat your initial position as origin (0, 0) You move to (4, 2) and face south. What is the egocentric relation of rubberduck? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-right, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q2: Location2ViewEvaluationTask

Question 3 (ID: 1e0d4b92e81bdd63)

✅ Evaluation Results
id: 1e0d4b92e81bdd63
question: Still treat your initial position as origin (0, 0) You move to (5, 0) and face west. What is the egocentric relation of pan? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-left, slightly far
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Evaluation state
Q3: Location2ViewEvaluationTask

📊 Task: View2LocationTextEvaluationTask

Question 1 (ID: 7eb7d98b38d92500)

✅ Evaluation Results
id: 7eb7d98b38d92500
question: You move to a new location and face north. You observe: shoes is at front, near, facing left; alarmclock is at front-left, mid distance, facing left Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [4, -2]
final_pos: [6, 9]
final_ori: [0, 1]
room_id: 1
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'shoes', 'direction': 'front', 'distance': 'near', 'orientation': 'facing left'}, {'name': 'alarmclock', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing left'}]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.800
raw_score: 0.800
score: 0.800
evaluation_mode: prompt_cogmap
Evaluation state
Q1: View2LocationTextEvaluationTask

Question 2 (ID: 4190e9e80fa5f994)

✅ Evaluation Results
id: 4190e9e80fa5f994
question: You move to a new location and face south. You observe: rubberduck is at front-right, mid distance, facing backward; blue door is at front, slightly far, on front wall; pan is at front-right, slightly far, facing right Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [4, 2]
final_pos: [6, 13]
final_ori: [0, -1]
room_id: 1
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'rubberduck', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'blue door', 'direction': 'front', 'distance': 'slightly far', 'orientation': 'on front wall'}, {'name': 'pan', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing right'}]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.800
raw_score: 0.800
score: 0.800
evaluation_mode: prompt_cogmap
Evaluation state
Q2: View2LocationTextEvaluationTask

Question 3 (ID: e006710c90936706)

✅ Evaluation Results
id: e006710c90936706
question: You move to a new location and face west. You observe: pan is at front-left, slightly far, facing forward; alarmclock is at front-slight-right, mid distance, facing forward; rubberduck is at front-slight-left, mid distance, facing right Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [5, 0]
final_pos: [7, 11]
final_ori: [-1, 0]
room_id: 1
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'pan', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing forward'}, {'name': 'alarmclock', 'direction': 'front-slight-right', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'rubberduck', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing right'}]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q3: View2LocationTextEvaluationTask

📊 Evaluation (use_gt_cogmap)

📊 Task: DirectionEvaluationTask

Question 1 (ID: fd2887aae65af114)

✅ Evaluation Results
id: fd2887aae65af114
question: You return to your starting position and face north. From a Top-Down map, describe where backpack is relative to books. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north east, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: DirectionEvaluationTask

Question 2 (ID: 47e25a4138337ec9)

✅ Evaluation Results
id: 47e25a4138337ec9
question: You return to your starting position and face north. From a Top-Down map, describe where books is relative to pan. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: east, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: DirectionEvaluationTask

Question 3 (ID: 05aacd3852716124)

✅ Evaluation Results
id: 05aacd3852716124
question: You return to your starting position and face north. From a Top-Down map, describe where desklamp is relative to books. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: south west, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: DirectionEvaluationTask

📊 Task: PovEvaluationTask

Question 1 (ID: dd32b518994cc0cd)

✅ Evaluation Results
id: dd32b518994cc0cd
question: Now you jump to rubberduck's direction, facing its direction. Describe where alarmclock is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front, near
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: PovEvaluationTask

Question 2 (ID: b3bd1bdceefdf76a)

✅ Evaluation Results
id: b3bd1bdceefdf76a
question: Now you jump to red chair's direction, facing its direction. Describe where shoppingcart is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: PovEvaluationTask

Question 3 (ID: 9f4070bd1a76781d)

✅ Evaluation Results
id: 9f4070bd1a76781d
question: Now you jump to laptop's direction, facing its direction. Describe where backpack is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front, near
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: PovEvaluationTask

📊 Task: BackwardPovTextEvaluationTask

Question 1 (ID: e0bba9ba020fcf43)

✅ Evaluation Results
id: e0bba9ba020fcf43
question: Now you jump to an object's position, facing its direction. You observe that alarmclock is front, near, facing left. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: rubberduck
final_pos: [4, 10]
final_ori: [0, 1]
final_observation: [{'name': 'alarmclock', 'direction': 'front', 'distance': 'near', 'orientation': 'facing left'}]
object_positions:
shoes: [6.0, 10.0]
shoppingcart: [6.0, 2.0]
laptop: [12.0, 12.0]
rubberduck: [4.0, 10.0]
bike: [4.0, 6.0]
airplane: [14.0, 14.0]
pan: [2.0, 8.0]
alarmclock: [4.0, 12.0]
red chair: [4.0, 4.0]
backpack: [14.0, 12.0]
desklamp: [1.0, 3.0]
books: [12.0, 10.0]
yellow door: [8.0, 10.0]
blue door: [6.0, 7.0]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
room_id: 1
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: BackwardPovTextEvaluationTask

Question 2 (ID: 5ee18081cd74702f)

✅ Evaluation Results
id: 5ee18081cd74702f
question: Now you jump to an object's position, facing its direction. You observe that airplane is front-left, mid distance, facing right; backpack is front, near, facing left. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: laptop
final_pos: [12, 12]
final_ori: [1, 0]
final_observation: [{'name': 'airplane', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'backpack', 'direction': 'front', 'distance': 'near', 'orientation': 'facing left'}]
object_positions:
shoes: [6.0, 10.0]
shoppingcart: [6.0, 2.0]
laptop: [12.0, 12.0]
rubberduck: [4.0, 10.0]
bike: [4.0, 6.0]
airplane: [14.0, 14.0]
pan: [2.0, 8.0]
alarmclock: [4.0, 12.0]
red chair: [4.0, 4.0]
backpack: [14.0, 12.0]
desklamp: [1.0, 3.0]
books: [12.0, 10.0]
yellow door: [8.0, 10.0]
blue door: [6.0, 7.0]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
room_id: 3
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: BackwardPovTextEvaluationTask

Question 3 (ID: 48ef54e966ae6b2b)

✅ Evaluation Results
id: 48ef54e966ae6b2b
question: Now you jump to an object's position, facing its direction. You observe that blue door is front-left, mid distance, on left wall. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: bike
final_pos: [4, 6]
final_ori: [1, 0]
final_observation: [{'name': 'blue door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on left wall'}]
object_positions:
shoes: [6.0, 10.0]
shoppingcart: [6.0, 2.0]
laptop: [12.0, 12.0]
rubberduck: [4.0, 10.0]
bike: [4.0, 6.0]
airplane: [14.0, 14.0]
pan: [2.0, 8.0]
alarmclock: [4.0, 12.0]
red chair: [4.0, 4.0]
backpack: [14.0, 12.0]
desklamp: [1.0, 3.0]
books: [12.0, 10.0]
yellow door: [8.0, 10.0]
blue door: [6.0, 7.0]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
room_id: 2
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: BackwardPovTextEvaluationTask

📊 Task: Action2ViewEvaluationTask

Question 1 (ID: dd857cdb010f8d2c)

✅ Evaluation Results
id: dd857cdb010f8d2c
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(90) 2. Jump to the object at front-right, slightly far. 3. Rotate(-180) 4. Jump to the object at front-left, mid distance. After executing the actions, what is the ego relation of desklamp relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: Action2ViewEvaluationTask

Question 2 (ID: f648d1cc302e836a)

✅ Evaluation Results
id: f648d1cc302e836a
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(90) 2. Jump to the object at front-right, mid distance. 3. Jump to the object at front, mid distance. 4. Rotate(-180) After executing the actions, what is the ego relation of shoes relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front, near
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: Action2ViewEvaluationTask

Question 3 (ID: 746bd0e0e6ba5a86)

✅ Evaluation Results
id: 746bd0e0e6ba5a86
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(90) 2. Jump to the object at front-left, mid distance. 3. Jump to the object at front-right, slightly far. 4. Rotate(-180) After executing the actions, what is the ego relation of shoes relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front, near
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: Action2ViewEvaluationTask

📊 Task: View2ActionTextEvaluationTask

Question 1 (ID: a64c44dde1456896)

✅ Evaluation Results
id: a64c44dde1456896
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: bike is at front-right, slightly far, facing right What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [1, 3]
final_ori: [0, 1]
room_id: 2
init_pos: [2, 11]
init_ori: [0, 1]
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
minimal_plan: [['rotate', 90], ['jumpto', 'blue door'], ['rotate', 180], ['jumpto', 'desklamp'], ['rotate', 90]]
final_observation: [{'name': 'bike', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing right'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [1.0, 3.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: View2ActionTextEvaluationTask

Question 2 (ID: c61a5d692c908db6)

✅ Evaluation Results
id: c61a5d692c908db6
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: bike is at front, near, facing right; blue door is at front-right, mid distance, on front wall What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [4, 4]
final_ori: [0, 1]
room_id: 2
init_pos: [2, 11]
init_ori: [0, 1]
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
minimal_plan: [['rotate', 180], ['jumpto', 'blue door'], ['jumpto', 'red chair'], ['rotate', 180]]
final_observation: [{'name': 'bike', 'direction': 'front', 'distance': 'near', 'orientation': 'facing right'}, {'name': 'blue door', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'on front wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [4.0, 4.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: View2ActionTextEvaluationTask

Question 3 (ID: 84f10bed2583426f)

✅ Evaluation Results
id: 84f10bed2583426f
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: airplane is at front-left, slightly far, facing right; backpack is at front-slight-left, slightly far, facing left; laptop is at front-left, slightly far, facing forward What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [8, 10]
final_ori: [1, 0]
room_id: [1, 3]
init_pos: [2, 11]
init_ori: [0, 1]
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
minimal_plan: [['rotate', 90], ['jumpto', 'initial_pos']]
final_observation: [{'name': 'airplane', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'backpack', 'direction': 'front-slight-left', 'distance': 'slightly far', 'orientation': 'facing left'}, {'name': 'laptop', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing forward'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
error: invalid_format
score: 0
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: View2ActionTextEvaluationTask

📊 Task: AlloMappingEvaluationTask

Question 1 (ID: d07c09cead29a5ab)

✅ Evaluation Results
id: d07c09cead29a5ab
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: backpack, books, shoppingcart, rubberduck. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[12, 1], [10, -1], [4, -9], [2, -1]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
threshold: 0.900
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: AlloMappingEvaluationTask

Question 2 (ID: c301d9dd74a52e57)

✅ Evaluation Results
id: c301d9dd74a52e57
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: laptop, alarmclock, desklamp, rubberduck, red chair. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[10, 1], [2, 1], [-1, -8], [2, -1], [2, -7]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
threshold: 0.900
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: AlloMappingEvaluationTask

Question 3 (ID: 4ade2017a10e41fb)

✅ Evaluation Results
id: 4ade2017a10e41fb
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: books, shoppingcart, laptop, alarmclock, bike. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[10, -1], [4, -9], [10, 1], [2, 1], [2, -5]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
threshold: 0.900
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: AlloMappingEvaluationTask

📊 Task: RotEvaluationTask

Question 1 (ID: d41a4cb1c4ae88e8)

✅ Evaluation Results
id: d41a4cb1c4ae88e8
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning counterclockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: desklamp, rubberduck, shoppingcart. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['desklamp', 'shoppingcart', 'rubberduck']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: RotEvaluationTask

Question 2 (ID: 35aed4a89a7998dd)

✅ Evaluation Results
id: 35aed4a89a7998dd
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: backpack, pan, rubberduck. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['backpack', 'rubberduck', 'pan']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: RotEvaluationTask

Question 3 (ID: 070834d55928c793)

✅ Evaluation Results
id: 070834d55928c793
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: backpack, rubberduck, shoppingcart. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['backpack', 'rubberduck', 'shoppingcart']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: RotEvaluationTask

📊 Task: Location2ViewEvaluationTask

Question 1 (ID: 12e5714b691ffd9d)

✅ Evaluation Results
id: 12e5714b691ffd9d
question: Still treat your initial position as origin (0, 0) You move to (4, -2) and face north. What is the egocentric relation of shoes? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front, near
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: Location2ViewEvaluationTask

Question 2 (ID: 37203b723cb2ed85)

✅ Evaluation Results
id: 37203b723cb2ed85
question: Still treat your initial position as origin (0, 0) You move to (4, 2) and face south. What is the egocentric relation of rubberduck? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-right, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: Location2ViewEvaluationTask

Question 3 (ID: 1e0d4b92e81bdd63)

✅ Evaluation Results
id: 1e0d4b92e81bdd63
question: Still treat your initial position as origin (0, 0) You move to (5, 0) and face west. What is the egocentric relation of pan? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-left, slightly far
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: Location2ViewEvaluationTask

📊 Task: View2LocationTextEvaluationTask

Question 1 (ID: 7eb7d98b38d92500)

✅ Evaluation Results
id: 7eb7d98b38d92500
question: You move to a new location and face north. You observe: shoes is at front, near, facing left; alarmclock is at front-left, mid distance, facing left Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [4, -2]
final_pos: [6, 9]
final_ori: [0, 1]
room_id: 1
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'shoes', 'direction': 'front', 'distance': 'near', 'orientation': 'facing left'}, {'name': 'alarmclock', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing left'}]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: View2LocationTextEvaluationTask

Question 2 (ID: 4190e9e80fa5f994)

✅ Evaluation Results
id: 4190e9e80fa5f994
question: You move to a new location and face south. You observe: rubberduck is at front-right, mid distance, facing backward; blue door is at front, slightly far, on front wall; pan is at front-right, slightly far, facing right Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [4, 2]
final_pos: [6, 13]
final_ori: [0, -1]
room_id: 1
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'rubberduck', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'blue door', 'direction': 'front', 'distance': 'slightly far', 'orientation': 'on front wall'}, {'name': 'pan', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing right'}]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.800
raw_score: 0.800
score: 0.800
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: View2LocationTextEvaluationTask

Question 3 (ID: e006710c90936706)

✅ Evaluation Results
id: e006710c90936706
question: You move to a new location and face west. You observe: pan is at front-left, slightly far, facing forward; alarmclock is at front-slight-right, mid distance, facing forward; rubberduck is at front-slight-left, mid distance, facing right Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [5, 0]
final_pos: [7, 11]
final_ori: [-1, 0]
room_id: 1
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'pan', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing forward'}, {'name': 'alarmclock', 'direction': 'front-slight-right', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'rubberduck', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing right'}]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: View2LocationTextEvaluationTask

📊 Evaluation (use_model_cogmap)

📊 Task: DirectionEvaluationTask

Question 1 (ID: fd2887aae65af114)

✅ Evaluation Results
id: fd2887aae65af114
question: You return to your starting position and face north. From a Top-Down map, describe where backpack is relative to books. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north east, mid distance
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Evaluation state
Q1: DirectionEvaluationTask

Question 2 (ID: 47e25a4138337ec9)

✅ Evaluation Results
id: 47e25a4138337ec9
question: You return to your starting position and face north. From a Top-Down map, describe where books is relative to pan. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: east, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q2: DirectionEvaluationTask

Question 3 (ID: 05aacd3852716124)

✅ Evaluation Results
id: 05aacd3852716124
question: You return to your starting position and face north. From a Top-Down map, describe where desklamp is relative to books. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: south west, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q3: DirectionEvaluationTask

📊 Task: PovEvaluationTask

Question 1 (ID: dd32b518994cc0cd)

✅ Evaluation Results
id: dd32b518994cc0cd
question: Now you jump to rubberduck's direction, facing its direction. Describe where alarmclock is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front, near
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q1: PovEvaluationTask

Question 2 (ID: b3bd1bdceefdf76a)

✅ Evaluation Results
id: b3bd1bdceefdf76a
question: Now you jump to red chair's direction, facing its direction. Describe where shoppingcart is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q2: PovEvaluationTask

Question 3 (ID: 9f4070bd1a76781d)

✅ Evaluation Results
id: 9f4070bd1a76781d
question: Now you jump to laptop's direction, facing its direction. Describe where backpack is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front, near
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Evaluation state
Q3: PovEvaluationTask

📊 Task: BackwardPovTextEvaluationTask

Question 1 (ID: e0bba9ba020fcf43)

✅ Evaluation Results
id: e0bba9ba020fcf43
question: Now you jump to an object's position, facing its direction. You observe that alarmclock is front, near, facing left. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: rubberduck
final_pos: [4, 10]
final_ori: [0, 1]
final_observation: [{'name': 'alarmclock', 'direction': 'front', 'distance': 'near', 'orientation': 'facing left'}]
object_positions:
shoes: [6.0, 10.0]
shoppingcart: [6.0, 2.0]
laptop: [12.0, 12.0]
rubberduck: [4.0, 10.0]
bike: [4.0, 6.0]
airplane: [14.0, 14.0]
pan: [2.0, 8.0]
alarmclock: [4.0, 12.0]
red chair: [4.0, 4.0]
backpack: [14.0, 12.0]
desklamp: [1.0, 3.0]
books: [12.0, 10.0]
yellow door: [8.0, 10.0]
blue door: [6.0, 7.0]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
room_id: 1
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q1: BackwardPovTextEvaluationTask

Question 2 (ID: 5ee18081cd74702f)

✅ Evaluation Results
id: 5ee18081cd74702f
question: Now you jump to an object's position, facing its direction. You observe that airplane is front-left, mid distance, facing right; backpack is front, near, facing left. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: laptop
final_pos: [12, 12]
final_ori: [1, 0]
final_observation: [{'name': 'airplane', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'backpack', 'direction': 'front', 'distance': 'near', 'orientation': 'facing left'}]
object_positions:
shoes: [6.0, 10.0]
shoppingcart: [6.0, 2.0]
laptop: [12.0, 12.0]
rubberduck: [4.0, 10.0]
bike: [4.0, 6.0]
airplane: [14.0, 14.0]
pan: [2.0, 8.0]
alarmclock: [4.0, 12.0]
red chair: [4.0, 4.0]
backpack: [14.0, 12.0]
desklamp: [1.0, 3.0]
books: [12.0, 10.0]
yellow door: [8.0, 10.0]
blue door: [6.0, 7.0]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
room_id: 3
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q2: BackwardPovTextEvaluationTask

Question 3 (ID: 48ef54e966ae6b2b)

✅ Evaluation Results
id: 48ef54e966ae6b2b
question: Now you jump to an object's position, facing its direction. You observe that blue door is front-left, mid distance, on left wall. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: bike
final_pos: [4, 6]
final_ori: [1, 0]
final_observation: [{'name': 'blue door', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'on left wall'}]
object_positions:
shoes: [6.0, 10.0]
shoppingcart: [6.0, 2.0]
laptop: [12.0, 12.0]
rubberduck: [4.0, 10.0]
bike: [4.0, 6.0]
airplane: [14.0, 14.0]
pan: [2.0, 8.0]
alarmclock: [4.0, 12.0]
red chair: [4.0, 4.0]
backpack: [14.0, 12.0]
desklamp: [1.0, 3.0]
books: [12.0, 10.0]
yellow door: [8.0, 10.0]
blue door: [6.0, 7.0]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
room_id: 2
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q3: BackwardPovTextEvaluationTask

📊 Task: Action2ViewEvaluationTask

Question 1 (ID: dd857cdb010f8d2c)

✅ Evaluation Results
id: dd857cdb010f8d2c
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(90) 2. Jump to the object at front-right, slightly far. 3. Rotate(-180) 4. Jump to the object at front-left, mid distance. After executing the actions, what is the ego relation of desklamp relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Evaluation state
Q1: Action2ViewEvaluationTask

Question 2 (ID: f648d1cc302e836a)

✅ Evaluation Results
id: f648d1cc302e836a
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(90) 2. Jump to the object at front-right, mid distance. 3. Jump to the object at front, mid distance. 4. Rotate(-180) After executing the actions, what is the ego relation of shoes relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front, near
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q2: Action2ViewEvaluationTask

Question 3 (ID: 746bd0e0e6ba5a86)

✅ Evaluation Results
id: 746bd0e0e6ba5a86
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(90) 2. Jump to the object at front-left, mid distance. 3. Jump to the object at front-right, slightly far. 4. Rotate(-180) After executing the actions, what is the ego relation of shoes relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front, near
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Evaluation state
Q3: Action2ViewEvaluationTask

📊 Task: View2ActionTextEvaluationTask

Question 1 (ID: a64c44dde1456896)

✅ Evaluation Results
id: a64c44dde1456896
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: bike is at front-right, slightly far, facing right What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [1, 3]
final_ori: [0, 1]
room_id: 2
init_pos: [2, 11]
init_ori: [0, 1]
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
minimal_plan: [['rotate', 90], ['jumpto', 'blue door'], ['rotate', 180], ['jumpto', 'desklamp'], ['rotate', 90]]
final_observation: [{'name': 'bike', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing right'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [1.0, 3.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q1: View2ActionTextEvaluationTask

Question 2 (ID: c61a5d692c908db6)

✅ Evaluation Results
id: c61a5d692c908db6
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: bike is at front, near, facing right; blue door is at front-right, mid distance, on front wall What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [4, 4]
final_ori: [0, 1]
room_id: 2
init_pos: [2, 11]
init_ori: [0, 1]
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
minimal_plan: [['rotate', 180], ['jumpto', 'blue door'], ['jumpto', 'red chair'], ['rotate', 180]]
final_observation: [{'name': 'bike', 'direction': 'front', 'distance': 'near', 'orientation': 'facing right'}, {'name': 'blue door', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'on front wall'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [4.0, 4.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q2: View2ActionTextEvaluationTask

Question 3 (ID: 84f10bed2583426f)

✅ Evaluation Results
id: 84f10bed2583426f
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: airplane is at front-left, slightly far, facing right; backpack is at front-slight-left, slightly far, facing left; laptop is at front-left, slightly far, facing forward What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [8, 10]
final_ori: [1, 0]
room_id: [1, 3]
init_pos: [2, 11]
init_ori: [0, 1]
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
minimal_plan: [['rotate', 90], ['jumpto', 'initial_pos']]
final_observation: [{'name': 'airplane', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'backpack', 'direction': 'front-slight-left', 'distance': 'slightly far', 'orientation': 'facing left'}, {'name': 'laptop', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing forward'}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [8.0, 10.0]
final_ori: [1, 0]
visible_match: True
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q3: View2ActionTextEvaluationTask

📊 Task: AlloMappingEvaluationTask

Question 1 (ID: d07c09cead29a5ab)

✅ Evaluation Results
id: d07c09cead29a5ab
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: backpack, books, shoppingcart, rubberduck. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[12, 1], [10, -1], [4, -9], [2, -1]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.609
threshold: 0.900
score: 0.609
evaluation_mode: use_model_cogmap
Evaluation state
Q1: AlloMappingEvaluationTask

Question 2 (ID: c301d9dd74a52e57)

✅ Evaluation Results
id: c301d9dd74a52e57
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: laptop, alarmclock, desklamp, rubberduck, red chair. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[10, 1], [2, 1], [-1, -8], [2, -1], [2, -7]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.681
threshold: 0.900
score: 0.681
evaluation_mode: use_model_cogmap
Evaluation state
Q2: AlloMappingEvaluationTask

Question 3 (ID: 4ade2017a10e41fb)

✅ Evaluation Results
id: 4ade2017a10e41fb
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: books, shoppingcart, laptop, alarmclock, bike. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[10, -1], [4, -9], [10, 1], [2, 1], [2, -5]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.799
threshold: 0.900
score: 0.799
evaluation_mode: use_model_cogmap
Evaluation state
Q3: AlloMappingEvaluationTask

📊 Task: RotEvaluationTask

Question 1 (ID: d41a4cb1c4ae88e8)

✅ Evaluation Results
id: d41a4cb1c4ae88e8
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning counterclockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: desklamp, rubberduck, shoppingcart. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['desklamp', 'shoppingcart', 'rubberduck']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q1: RotEvaluationTask

Question 2 (ID: 35aed4a89a7998dd)

✅ Evaluation Results
id: 35aed4a89a7998dd
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: backpack, pan, rubberduck. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['backpack', 'rubberduck', 'pan']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q2: RotEvaluationTask

Question 3 (ID: 070834d55928c793)

✅ Evaluation Results
id: 070834d55928c793
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: backpack, rubberduck, shoppingcart. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['backpack', 'rubberduck', 'shoppingcart']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q3: RotEvaluationTask

📊 Task: Location2ViewEvaluationTask

Question 1 (ID: 12e5714b691ffd9d)

✅ Evaluation Results
id: 12e5714b691ffd9d
question: Still treat your initial position as origin (0, 0) You move to (4, -2) and face north. What is the egocentric relation of shoes? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front, near
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: use_model_cogmap
Evaluation state
Q1: Location2ViewEvaluationTask

Question 2 (ID: 37203b723cb2ed85)

✅ Evaluation Results
id: 37203b723cb2ed85
question: Still treat your initial position as origin (0, 0) You move to (4, 2) and face south. What is the egocentric relation of rubberduck? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-right, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Evaluation state
Q2: Location2ViewEvaluationTask

Question 3 (ID: 1e0d4b92e81bdd63)

✅ Evaluation Results
id: 1e0d4b92e81bdd63
question: Still treat your initial position as origin (0, 0) You move to (5, 0) and face west. What is the egocentric relation of pan? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-left, slightly far
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: use_model_cogmap
Evaluation state
Q3: Location2ViewEvaluationTask

📊 Task: View2LocationTextEvaluationTask

Question 1 (ID: 7eb7d98b38d92500)

✅ Evaluation Results
id: 7eb7d98b38d92500
question: You move to a new location and face north. You observe: shoes is at front, near, facing left; alarmclock is at front-left, mid distance, facing left Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [4, -2]
final_pos: [6, 9]
final_ori: [0, 1]
room_id: 1
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'shoes', 'direction': 'front', 'distance': 'near', 'orientation': 'facing left'}, {'name': 'alarmclock', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing left'}]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q1: View2LocationTextEvaluationTask

Question 2 (ID: 4190e9e80fa5f994)

✅ Evaluation Results
id: 4190e9e80fa5f994
question: You move to a new location and face south. You observe: rubberduck is at front-right, mid distance, facing backward; blue door is at front, slightly far, on front wall; pan is at front-right, slightly far, facing right Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [4, 2]
final_pos: [6, 13]
final_ori: [0, -1]
room_id: 1
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'rubberduck', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'blue door', 'direction': 'front', 'distance': 'slightly far', 'orientation': 'on front wall'}, {'name': 'pan', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing right'}]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.800
raw_score: 0.800
score: 0.800
evaluation_mode: use_model_cogmap
Evaluation state
Q2: View2LocationTextEvaluationTask

Question 3 (ID: e006710c90936706)

✅ Evaluation Results
id: e006710c90936706
question: You move to a new location and face west. You observe: pan is at front-left, slightly far, facing forward; alarmclock is at front-slight-right, mid distance, facing forward; rubberduck is at front-slight-left, mid distance, facing right Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [5, 0]
final_pos: [7, 11]
final_ori: [-1, 0]
room_id: 1
object_positions:
shoes: [6, 10]
shoppingcart: [6, 2]
laptop: [12, 12]
rubberduck: [4, 10]
bike: [4, 6]
airplane: [14, 14]
pan: [2, 8]
alarmclock: [4, 12]
red chair: [4, 4]
backpack: [14, 12]
desklamp: [1, 3]
books: [12, 10]
yellow door: [8, 10]
blue door: [6, 7]
object_orientations:
shoes: [-1, 0]
shoppingcart: [0, -1]
laptop: [1, 0]
rubberduck: [0, 1]
bike: [1, 0]
airplane: [0, -1]
pan: [-1, 0]
alarmclock: [-1, 0]
red chair: [0, -1]
backpack: [0, 1]
desklamp: [0, -1]
yellow door: [1, 0]
blue door: [0, 1]
final_observation: [{'name': 'pan', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing forward'}, {'name': 'alarmclock', 'direction': 'front-slight-right', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'rubberduck', 'direction': 'front-slight-left', 'distance': 'mid distance', 'orientation': 'facing right'}]
gate_info:
yellow door:
room_ids: [1, 3]
ori_by_room:
1: [-1, 0]
3: [1, 0]
blue door:
room_ids: [2, 1]
ori_by_room:
2: [0, -1]
1: [0, 1]
object_rooms:
shoes: 1
shoppingcart: 2
laptop: 3
rubberduck: 1
bike: 2
airplane: 3
pan: 1
alarmclock: 1
red chair: 2
backpack: 3
desklamp: 2
books: 3
yellow door: [1, 3]
blue door: [2, 1]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.819
raw_score: 0.819
score: 0.819
evaluation_mode: use_model_cogmap
Evaluation state
Q3: View2LocationTextEvaluationTask