Model: gemini-3-pro-preview

sample_run01

Select Configuration:

📊 Sample Metrics

🔍 Exploration

last_node_coverage: 1
last_edge_coverage: 0.488
n_exploration_steps: 12
action_counts:
move: 4
rotate: 8
return: 0
observe: 11
term: 1
forced_term: 0
query: 0
action_cost: 11
final_information_gain: 0.784
action_fail_ratio: 0
valid_action_ratio: 1
false_belief_steps: 7
false_belief_f1: 1
false_belief_f1_position: 1
false_belief_f1_facing: 1
false_belief_action_cost: 6
false_belief_action_cost_after_seen: 0

✅ Evaluation

overall:
n_total: 27
total_score: 23.110
avg_accuracy: 0.856
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
PovEvaluationTask:
n_total: 3
task_score: 1.500
avg_accuracy: 0.500
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Action2ViewEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
View2ActionTextEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
AlloMappingEvaluationTask:
n_total: 3
task_score: 2.503
avg_accuracy: 0.834
RotEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Location2ViewEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
View2LocationTextEvaluationTask:
n_total: 3
task_score: 2.607
avg_accuracy: 0.869

✅ Evaluation (prompt_cogmap)

overall:
n_total: 27
total_score: 18.984
avg_accuracy: 0.703
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 1.500
avg_accuracy: 0.500
PovEvaluationTask:
n_total: 3
task_score: 1.500
avg_accuracy: 0.500
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Action2ViewEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
View2ActionTextEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
AlloMappingEvaluationTask:
n_total: 3
task_score: 2.484
avg_accuracy: 0.828
RotEvaluationTask:
n_total: 3
task_score: 0
avg_accuracy: 0
Location2ViewEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
View2LocationTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1

✅ Evaluation (use_gt_cogmap)

overall:
n_total: 27
total_score: 26.500
avg_accuracy: 0.981
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
PovEvaluationTask:
n_total: 3
task_score: 2.500
avg_accuracy: 0.833
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Action2ViewEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
View2ActionTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
AlloMappingEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
RotEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Location2ViewEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
View2LocationTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1

✅ Evaluation (use_model_cogmap)

overall:
n_total: 27
total_score: 21.738
avg_accuracy: 0.805
per_task:
DirectionEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
PovEvaluationTask:
n_total: 3
task_score: 1.500
avg_accuracy: 0.500
BackwardPovTextEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Action2ViewEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
View2ActionTextEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
AlloMappingEvaluationTask:
n_total: 3
task_score: 2.535
avg_accuracy: 0.845
RotEvaluationTask:
n_total: 3
task_score: 3
avg_accuracy: 1
Location2ViewEvaluationTask:
n_total: 3
task_score: 2
avg_accuracy: 0.667
View2LocationTextEvaluationTask:
n_total: 3
task_score: 2.702
avg_accuracy: 0.901

🧠 Cognitive Map

exploration:
error:
local_vs_gt_local_avg:
dir: 0.926
facing: 1
pos: 0.868
overall: 0.931
global_vs_gt_global_avg:
dir: 0.786
facing: 0.986
pos: 0.802
overall: 0.858
agent_vs_gt_agent_avg:
dir: 0.833
facing: 1
pos: 0.872
overall: 0.902
newly_observed_vs_gt_local_avg:
dir: 0.938
facing: 1
pos: 0.857
overall: 0.932
correctness:
last_global_vs_gt_full:
dir: 0.733
facing: 1
pos: 0.753
overall: 0.829
consistency:
local_vs_global_avg:
dir: 0.848
facing: 0.889
pos: 0.841
overall: 0.859
position_update_avg: 0.624
facing_update_avg: 0.982
position_stability_avg: 0.624
facing_stability_avg: 0.982
fog_probe:
f1_avg: 0.887
precision_avg: 0.875
recall_avg: 0.944
cogmap_fb:
metrics:
changed:
dir: None
pos: 0.856
facing: 1
overall: None
retention:
dir: None
pos: 0.306
facing: 0
overall: None
unchanged:
dir: 0.742
facing: 0.929
pos: 0.738
overall: 0.803
inertia: 0.000
inertia_list: [0.0005318038676377831, -8.470649232414757e-08]
unchanged_retention:
dir: None
pos: 0.853
facing: 1
overall: None
unchanged_retention_minus_retention:
pos: 0.872
facing: 1
unchanged_exploration:
dir: 0.764
facing: 1
pos: 0.778
overall: 0.847

🧭 False Belief CogMap

inertia: 0.000
changed:
dir: None
pos: 0.856
facing: 1
overall: None
retention:
dir: None
pos: 0.306
facing: 0
overall: None
unchanged:
dir: 0.742
facing: 0.929
pos: 0.738
overall: 0.803
unchanged_retention:
dir: None
pos: 0.853
facing: 1
overall: None
unchanged_retention_minus_retention:
pos: 0.872
facing: 1
unchanged_exploration:
dir: 0.764
facing: 1
pos: 0.778
overall: 0.847

📊 Performance Charts

Cognitive Map (Update)
Global Update Metrics
Cognitive Map (Full)
Global Full Metrics
Cognitive Map (Self-Tracking)
Global Self-Tracking Metrics
FB CogMap (Unchanged)
False Belief CogMap Unchanged per Turn
Fog Probe F1
Fog Probe F1 per Turn
Fog Probe Precision
Fog Probe Precision per Turn
Fog Probe Recall
Fog Probe Recall per Turn
Position Update
Position Update per Turn
Facing Update
Facing Update per Turn
Position Stability
Position Stability per Turn
Facing Stability
Facing Stability per Turn

🌍 Exploration Phase

🔄 Turn 1

💬 Assistant Action
Actions: [Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "north"
  },
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      2,
      4
    ]
  }
}
🎯 Ground Truth (Observed)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Full)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "televison": {
    "position": [
      1,
      8
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      3,
      10
    ],
    "facing": "west"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "wine": {
    "position": [
      1,
      10
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "shoppingcart": {
    "position": [
      5,
      6
    ],
    "facing": "south"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "truck": {
      "position": [
        -2,
        3
      ],
      "facing": "+x"
    },
    "red door": {
      "position": [
        3,
        5
      ]
    }
  }
}
🎯 Ground Truth
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "truck": {
      "position": [
        -2,
        3
      ],
      "facing": "+x"
    },
    "red door": {
      "position": [
        3,
        5
      ]
    }
  }
}
🎯 Ground Truth
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  }
}
🗺️ Symbolic Fog Map
################
#########......#
#########......#
#########......#
#########......#
#########......#
#########......#
#############+##
#......#..EF..##
#......#......##
#......#D.....##
#......#.*....##
#......+.B.C..##
#......#A.....##
################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: A, B, C, D, E, F
🎯 Ground Truth: C, A, B
🧠 Cognitive Map Metrics
Global:
dir: 0.833
facing: 1
pos: 0.835
overall: 0.890
Global (Full):
dir: 0.042
facing: 0.200
pos: 0.167
overall: 0.136
Local:
dir: 1
facing: 1
pos: 0.802
overall: 0.934
Local (Newly):
dir: 1
facing: 1
pos: 0.802
overall: 0.934
Fog Probe:
precision: 0.500
recall: 1
overall: 0.667
📈 Turn Metrics
node_coverage: 0.143
edge_coverage: 0.049
is_action_fail: False
step: 1
action_counts:
move: 0
rotate: 0
return: 0
observe: 1
term: 0
forced_term: 0
query: 0
information_gain: 0.098
reward: -0.100
is_done: False
is_valid_action: True
Current state
State at Turn 1

🔄 Turn 2

💬 Assistant Action
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "east"
  },
  "truck": {
    "position": [
      -2,
      3
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      3,
      5
    ]
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -3
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Observed)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "east"
  }
}
🎯 Ground Truth (Full)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "televison": {
    "position": [
      1,
      8
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      3,
      10
    ],
    "facing": "west"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "wine": {
    "position": [
      1,
      10
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "shoppingcart": {
    "position": [
      5,
      6
    ],
    "facing": "south"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "east"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "backpack": {
      "position": [
        0,
        3
      ],
      "facing": "+x"
    },
    "shoes": {
      "position": [
        3,
        4
      ],
      "facing": "-x"
    },
    "red door": {
      "position": [
        -3,
        4
      ],
      "facing": "+x"
    }
  }
}
🎯 Ground Truth
{
  "shoes": {
    "position": [
      2,
      4
    ],
    "facing": "north"
  },
  "backpack": {
    "position": [
      0,
      3
    ],
    "facing": "south"
  },
  "red door": {
    "position": [
      -4,
      4
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "backpack": {
      "position": [
        0,
        3
      ],
      "facing": "+x"
    },
    "shoes": {
      "position": [
        3,
        4
      ],
      "facing": "-x"
    },
    "red door": {
      "position": [
        -3,
        4
      ],
      "facing": "+x"
    }
  }
}
🎯 Ground Truth
{
  "shoes": {
    "position": [
      2,
      4
    ],
    "facing": "north"
  },
  "backpack": {
    "position": [
      0,
      3
    ],
    "facing": "south"
  }
}
🗺️ Symbolic Fog Map
################
#########......#
#########......#
#########......#
#########......#
#########......#
#########......#
#############+##
#......#.B....##
#......#E.....##
#......#......##
#......#F*....##
#......+......##
#......#AD...C##
################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: A, D, F
🎯 Ground Truth: F, A, D
🧠 Cognitive Map Metrics
Global:
dir: 0.933
facing: 1
pos: 0.856
overall: 0.930
Global (Full):
dir: 0.117
facing: 0.400
pos: 0.285
overall: 0.267
Local:
dir: 0.833
facing: 1
pos: 0.881
overall: 0.905
Local (Newly):
dir: 0.667
facing: 1
pos: 0.896
overall: 0.854
Fog Probe:
precision: 1
recall: 1
overall: 1
📈 Turn Metrics
node_coverage: 0.286
edge_coverage: 0.098
is_action_fail: False
step: 2
action_counts:
move: 0
rotate: 1
return: 0
observe: 2
term: 0
forced_term: 0
query: 0
information_gain: 0.218
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 2
Current state
State at Turn 2

🔄 Turn 3

💬 Assistant Action
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      -2,
      3
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      2,
      6
    ]
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      2,
      -2
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Observed)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "south"
  }
}
🎯 Ground Truth (Full)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "televison": {
    "position": [
      1,
      8
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      3,
      10
    ],
    "facing": "west"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "wine": {
    "position": [
      1,
      10
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "shoppingcart": {
    "position": [
      5,
      6
    ],
    "facing": "south"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "south"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "chair": {
      "position": [
        -2,
        3
      ],
      "facing": "-y"
    }
  }
}
🎯 Ground Truth
{
  "chair": {
    "position": [
      -1,
      2
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "chair": {
      "position": [
        -2,
        3
      ],
      "facing": "-y"
    }
  }
}
🎯 Ground Truth
{
  "chair": {
    "position": [
      -1,
      2
    ],
    "facing": "north"
  }
}
🗺️ Symbolic Fog Map
################
#########......#
#########......#
#########......#
#########......#
#########......#
#########......#
#############+##
#......#......##
#......#..A...##
#......#......##
#......#B*....##
#......+......##
#......#......##
################
🔍 Fog Probe JSONs
📍 Candidates: A, B
🤖 Predicted: B
🎯 Ground Truth: B
🧠 Cognitive Map Metrics
Global:
dir: 0.714
facing: 1
pos: 0.782
overall: 0.832
Global (Full):
dir: 0.125
facing: 0.500
pos: 0.313
overall: 0.313
Local:
dir: 1
facing: 1
pos: 0.802
overall: 0.934
Local (Newly):
dir: 1
facing: 1
pos: 0.802
overall: 0.934
Fog Probe:
precision: 1
recall: 1
overall: 1
📈 Turn Metrics
node_coverage: 0.357
edge_coverage: 0.122
is_action_fail: False
step: 3
action_counts:
move: 0
rotate: 2
return: 0
observe: 3
term: 0
forced_term: 0
query: 0
information_gain: 0.275
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 3
Current state
State at Turn 3

🔄 Turn 4

💬 Assistant Action
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "west"
  },
  "truck": {
    "position": [
      -2,
      3
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      1,
      5
    ]
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -3
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      2,
      -3
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      -2,
      -3
    ]
  }
}
🎯 Ground Truth (Observed)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "west"
  }
}
🎯 Ground Truth (Full)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "televison": {
    "position": [
      1,
      8
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      3,
      10
    ],
    "facing": "west"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "wine": {
    "position": [
      1,
      10
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "shoppingcart": {
    "position": [
      5,
      6
    ],
    "facing": "south"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      0,
      0
    ],
    "facing": "west"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "green door": {
      "position": [
        -2,
        3
      ]
    }
  }
}
🎯 Ground Truth
{
  "green door": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "green door": {
      "position": [
        -2,
        3
      ]
    }
  }
}
🎯 Ground Truth
{
  "green door": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  }
}
🧠 Cognitive Map Metrics
Global:
dir: 0.714
facing: 1
pos: 0.774
overall: 0.829
Global (Full):
dir: 0.167
facing: 0.500
pos: 0.361
overall: 0.343
Local:
dir: 1
facing: 1
pos: 0.802
overall: 0.934
Local (Newly):
dir: 1
facing: 1
pos: 0.802
overall: 0.934
Fog Probe:
(none)
📈 Turn Metrics
node_coverage: 0.429
edge_coverage: 0.146
is_action_fail: False
step: 4
action_counts:
move: 0
rotate: 3
return: 0
observe: 4
term: 0
forced_term: 0
query: 0
information_gain: 0.332
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 4
Current state
State at Turn 4

🔄 Turn 5

💬 Assistant Action
Actions: [JumpTo(green door), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      -3,
      -2
    ],
    "facing": "west"
  },
  "truck": {
    "position": [
      -2,
      4
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      2,
      5
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      4,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      3,
      -3
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      2,
      -3
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      -3,
      -2
    ],
    "facing": "east"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -6,
      -1
    ],
    "facing": "north"
  },
  "table": {
    "position": [
      -6,
      -3
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Observed)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      -2,
      -1
    ],
    "facing": "west"
  }
}
🎯 Ground Truth (Full)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "televison": {
    "position": [
      1,
      8
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      3,
      10
    ],
    "facing": "west"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "wine": {
    "position": [
      1,
      10
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "shoppingcart": {
    "position": [
      5,
      6
    ],
    "facing": "south"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      -2,
      -1
    ],
    "facing": "west"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "scooter": {
      "position": [
        3,
        5
      ],
      "facing": "-y"
    },
    "basket": {
      "position": [
        1,
        3
      ]
    },
    "table": {
      "position": [
        -1,
        3
      ]
    }
  }
}
🎯 Ground Truth
{
  "table": {
    "position": [
      -1,
      3
    ],
    "facing": "east"
  },
  "scooter": {
    "position": [
      3,
      5
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "scooter": {
      "position": [
        3,
        5
      ],
      "facing": "-y"
    },
    "basket": {
      "position": [
        1,
        3
      ]
    },
    "table": {
      "position": [
        -1,
        3
      ]
    }
  }
}
🎯 Ground Truth
{
  "table": {
    "position": [
      -1,
      3
    ],
    "facing": "east"
  },
  "scooter": {
    "position": [
      3,
      5
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      1,
      3
    ],
    "facing": "east"
  }
}
🗺️ Symbolic Fog Map
################
#########......#
#########......#
#########......#
#########......#
#########......#
#########......#
#############+##
#.E..CF#......##
#....D.#......##
#......#......##
#..B...#......##
#......*......##
#...A..#......##
################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: C, F
🎯 Ground Truth: D, F, C
🧠 Cognitive Map Metrics
Global:
dir: 0.873
facing: 1
pos: 0.790
overall: 0.887
Global (Full):
dir: 0.400
facing: 0.600
pos: 0.526
overall: 0.509
Local:
dir: 1
facing: 1
pos: 1
overall: 1
Local (Newly):
dir: 1
facing: 1
pos: 1
overall: 1
Fog Probe:
precision: 1
recall: 0.667
overall: 0.800
📈 Turn Metrics
node_coverage: 0.643
edge_coverage: 0.220
is_action_fail: False
step: 5
action_counts:
move: 1
rotate: 3
return: 0
observe: 5
term: 0
forced_term: 0
query: 0
information_gain: 0.480
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 5
Current state
State at Turn 5

🔄 Turn 6

💬 Assistant Action
Actions: [JumpTo(table), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      -6,
      -1
    ],
    "facing": "west"
  },
  "truck": {
    "position": [
      -2,
      3
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      1,
      5
    ]
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -4
    ],
    "facing": "east"
  },
  "chair": {
    "position": [
      2,
      -3
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      -3,
      0
    ]
  },
  "scooter": {
    "position": [
      -6,
      3
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -6,
      1
    ]
  },
  "table": {
    "position": [
      -6,
      -1
    ]
  }
}
🎯 Ground Truth (Observed)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      -5,
      -2
    ],
    "facing": "west"
  }
}
🎯 Ground Truth (Full)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "televison": {
    "position": [
      1,
      8
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      3,
      10
    ],
    "facing": "west"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "wine": {
    "position": [
      1,
      10
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "shoppingcart": {
    "position": [
      5,
      6
    ],
    "facing": "south"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      -5,
      -2
    ],
    "facing": "west"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {}
}
🎯 Ground Truth
(no data)
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {}
}
🎯 Ground Truth
(no data)
🗺️ Symbolic Fog Map
################
#########......#
#########......#
#########......#
#########......#
#########......#
#########......#
#############+##
#.....F#......##
#.ABE.D#......##
#.....C#......##
#......#......##
#......+......##
#...*..#......##
################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: C, D, E, F
🎯 Ground Truth: C, F, D
🧠 Cognitive Map Metrics
Global:
dir: 0.709
facing: 0.833
pos: 0.771
overall: 0.771
Global (Full):
dir: 0.325
facing: 0.500
pos: 0.514
overall: 0.446
Local:
(none)
Local (Newly):
(none)
Fog Probe:
precision: 0.750
recall: 1
overall: 0.857
📈 Turn Metrics
node_coverage: 0.643
edge_coverage: 0.220
is_action_fail: False
step: 6
action_counts:
move: 2
rotate: 3
return: 0
observe: 6
term: 0
forced_term: 0
query: 0
information_gain: 0.480
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 6
Current state
State at Turn 6

🔄 Turn 7

💬 Assistant Action
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      -6,
      -2
    ],
    "facing": "north"
  },
  "truck": {
    "position": [
      -2,
      2
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      1,
      5
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      3,
      -3
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      2,
      -2
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      -3,
      -1
    ],
    "facing": "west"
  },
  "table": {
    "position": [
      -6,
      -2
    ],
    "facing": "north"
  },
  "scooter": {
    "position": [
      -9,
      2
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -6,
      0
    ],
    "facing": "south"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  }
}
🎯 Ground Truth (Observed)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      -5,
      -2
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Full)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "televison": {
    "position": [
      1,
      8
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      3,
      10
    ],
    "facing": "west"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "wine": {
    "position": [
      1,
      10
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "shoppingcart": {
    "position": [
      5,
      6
    ],
    "facing": "south"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      -5,
      -2
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "basket": {
      "position": [
        0,
        2
      ]
    },
    "scooter": {
      "position": [
        -3,
        5
      ],
      "facing": "+x"
    },
    "bike": {
      "position": [
        3,
        5
      ],
      "facing": "-x"
    }
  }
}
🎯 Ground Truth
{
  "scooter": {
    "position": [
      -2,
      4
    ],
    "facing": "east"
  },
  "bike": {
    "position": [
      2,
      4
    ],
    "facing": "west"
  },
  "basket": {
    "position": [
      0,
      2
    ],
    "facing": "east"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "basket": {
      "position": [
        0,
        2
      ]
    },
    "scooter": {
      "position": [
        -3,
        5
      ],
      "facing": "+x"
    },
    "bike": {
      "position": [
        3,
        5
      ],
      "facing": "-x"
    }
  }
}
🎯 Ground Truth
{
  "bike": {
    "position": [
      2,
      4
    ],
    "facing": "west"
  }
}
🧠 Cognitive Map Metrics
Global:
dir: 0.848
facing: 1
pos: 0.802
overall: 0.884
Global (Full):
dir: 0.467
facing: 0.700
pos: 0.588
overall: 0.585
Local:
dir: 1
facing: 1
pos: 0.835
overall: 0.945
Local (Newly):
dir: 1
facing: 1
pos: 0.802
overall: 0.934
Fog Probe:
(none)
📈 Turn Metrics
node_coverage: 0.714
edge_coverage: 0.293
is_action_fail: False
step: 7
action_counts:
move: 2
rotate: 4
return: 0
observe: 7
term: 0
forced_term: 0
query: 0
information_gain: 0.522
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 7
Current state
State at Turn 7

🔄 Turn 8

💬 Assistant Action
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      -6,
      -2
    ],
    "facing": "east"
  },
  "truck": {
    "position": [
      -1,
      3
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      2,
      5
    ]
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      2,
      -2
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      -3,
      -1
    ]
  },
  "table": {
    "position": [
      -6,
      -2
    ]
  },
  "basket": {
    "position": [
      -6,
      0
    ]
  },
  "scooter": {
    "position": [
      -7,
      3
    ],
    "facing": "east"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  }
}
🎯 Ground Truth (Observed)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  }
}
🎯 Ground Truth (Full)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "televison": {
    "position": [
      1,
      8
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      3,
      10
    ],
    "facing": "west"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "wine": {
    "position": [
      1,
      10
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "shoppingcart": {
    "position": [
      5,
      6
    ],
    "facing": "south"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "green door": {
      "position": [
        -1,
        3
      ],
      "facing": "-y"
    }
  }
}
🎯 Ground Truth
{
  "green door": {
    "position": [
      -1,
      3
    ],
    "facing": "east"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "green door": {
      "position": [
        -1,
        3
      ],
      "facing": "-y"
    }
  }
}
🎯 Ground Truth
(no data)
🧠 Cognitive Map Metrics
Global:
dir: 0.803
facing: 1
pos: 0.850
overall: 0.884
Global (Full):
dir: 0.442
facing: 0.700
pos: 0.623
overall: 0.588
Local:
dir: 1
facing: 1
pos: 1
overall: 1
Local (Newly):
(none)
Fog Probe:
(none)
📈 Turn Metrics
node_coverage: 0.714
edge_coverage: 0.293
is_action_fail: False
step: 8
action_counts:
move: 2
rotate: 5
return: 0
observe: 8
term: 0
forced_term: 0
query: 0
information_gain: 0.522
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 8
Current state
State at Turn 8

🔄 Turn 9

💬 Assistant Action
Actions: [Rotate(90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      -6,
      -2
    ],
    "facing": "south"
  },
  "truck": {
    "position": [
      -2,
      3
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      2,
      5
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -3
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      2,
      -3
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      -3,
      -1
    ],
    "facing": "east"
  },
  "scooter": {
    "position": [
      -7,
      3
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -6,
      0
    ]
  },
  "table": {
    "position": [
      -6,
      -2
    ]
  },
  "bike": {
    "position": [
      -3,
      3
    ],
    "facing": "west"
  }
}
🎯 Ground Truth (Observed)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      -5,
      -2
    ],
    "facing": "south"
  }
}
🎯 Ground Truth (Full)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "televison": {
    "position": [
      1,
      8
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      3,
      10
    ],
    "facing": "west"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "wine": {
    "position": [
      1,
      10
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "shoppingcart": {
    "position": [
      5,
      6
    ],
    "facing": "south"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      -5,
      -2
    ],
    "facing": "south"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {}
}
🎯 Ground Truth
(no data)
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {}
}
🎯 Ground Truth
(no data)
🧠 Cognitive Map Metrics
Global:
dir: 0.818
facing: 1
pos: 0.829
overall: 0.882
Global (Full):
dir: 0.450
facing: 0.700
pos: 0.608
overall: 0.586
Local:
(none)
Local (Newly):
(none)
Fog Probe:
(none)
📈 Turn Metrics
node_coverage: 0.714
edge_coverage: 0.293
is_action_fail: False
step: 9
action_counts:
move: 2
rotate: 6
return: 0
observe: 9
term: 0
forced_term: 0
query: 0
information_gain: 0.522
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 9
Current state
State at Turn 9

🔄 Turn 10

💬 Assistant Action
Actions: [Rotate(-90), JumpTo(green door), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      -3,
      -1
    ],
    "facing": "east"
  },
  "truck": {
    "position": [
      -2,
      2
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      3,
      5
    ],
    "facing": "south"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -4
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      2,
      -2
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      -3,
      -1
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      -6,
      -2
    ]
  },
  "basket": {
    "position": [
      -6,
      0
    ]
  },
  "scooter": {
    "position": [
      -8,
      2
    ],
    "facing": "east"
  },
  "bike": {
    "position": [
      -4,
      3
    ],
    "facing": "west"
  }
}
🎯 Ground Truth (Observed)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  }
}
🎯 Ground Truth (Full)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "televison": {
    "position": [
      1,
      8
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      3,
      10
    ],
    "facing": "west"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "wine": {
    "position": [
      1,
      10
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "shoppingcart": {
    "position": [
      5,
      6
    ],
    "facing": "south"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "chair": {
      "position": [
        1,
        3
      ],
      "facing": "-x"
    },
    "backpack": {
      "position": [
        -2,
        6
      ],
      "facing": "+x"
    },
    "shoes": {
      "position": [
        2,
        7
      ],
      "facing": "-x"
    },
    "red door": {
      "position": [
        -4,
        4
      ],
      "facing": "+x"
    }
  }
}
🎯 Ground Truth
{
  "shoes": {
    "position": [
      1,
      6
    ],
    "facing": "north"
  },
  "backpack": {
    "position": [
      -1,
      5
    ],
    "facing": "south"
  },
  "chair": {
    "position": [
      1,
      3
    ],
    "facing": "north"
  },
  "red door": {
    "position": [
      -5,
      6
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "chair": {
      "position": [
        1,
        3
      ],
      "facing": "-x"
    },
    "backpack": {
      "position": [
        -2,
        6
      ],
      "facing": "+x"
    },
    "shoes": {
      "position": [
        2,
        7
      ],
      "facing": "-x"
    },
    "red door": {
      "position": [
        -4,
        4
      ],
      "facing": "+x"
    }
  }
}
🎯 Ground Truth
(no data)
🧠 Cognitive Map Metrics
Global:
dir: 0.788
facing: 1
pos: 0.834
overall: 0.874
Global (Full):
dir: 0.433
facing: 0.700
pos: 0.611
overall: 0.582
Local:
dir: 0.600
facing: 1
pos: 0.792
overall: 0.797
Local (Newly):
(none)
Fog Probe:
(none)
📈 Turn Metrics
node_coverage: 0.714
edge_coverage: 0.390
is_action_fail: False
step: 10
action_counts:
move: 3
rotate: 7
return: 0
observe: 10
term: 0
forced_term: 0
query: 0
information_gain: 0.613
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 10
Current state
State at Turn 10

🔄 Turn 11

💬 Assistant Action
Actions: [JumpTo(red door), Rotate(-90), Observe()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      2,
      4
    ],
    "facing": "north"
  },
  "truck": {
    "position": [
      -2,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      3,
      -3
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      -2,
      -1
    ]
  },
  "red door": {
    "position": [
      2,
      4
    ]
  },
  "table": {
    "position": [
      -4,
      -3
    ]
  },
  "basket": {
    "position": [
      -4,
      -1
    ]
  },
  "scooter": {
    "position": [
      -5,
      1
    ],
    "facing": "east"
  },
  "bike": {
    "position": [
      -3,
      0
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      7
    ],
    "facing": "south"
  },
  "televison": {
    "position": [
      -1,
      7
    ],
    "facing": "west"
  },
  "wine": {
    "position": [
      -1,
      8
    ]
  },
  "airplane": {
    "position": [
      0,
      8
    ],
    "facing": "west"
  }
}
🎯 Ground Truth (Observed)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "televison": {
    "position": [
      1,
      8
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      3,
      10
    ],
    "facing": "west"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "wine": {
    "position": [
      1,
      10
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "shoppingcart": {
    "position": [
      5,
      6
    ],
    "facing": "south"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Full)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "televison": {
    "position": [
      1,
      8
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      3,
      10
    ],
    "facing": "west"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "wine": {
    "position": [
      1,
      10
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "shoppingcart": {
    "position": [
      5,
      6
    ],
    "facing": "south"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "shoppingcart": {
      "position": [
        2,
        3
      ],
      "facing": "-y"
    },
    "wine": {
      "position": [
        -3,
        6
      ]
    },
    "televison": {
      "position": [
        -3,
        4
      ],
      "facing": "-x"
    },
    "airplane": {
      "position": [
        -1,
        6
      ],
      "facing": "-x"
    }
  }
}
🎯 Ground Truth
{
  "televison": {
    "position": [
      -3,
      4
    ],
    "facing": "west"
  },
  "airplane": {
    "position": [
      -1,
      6
    ],
    "facing": "west"
  },
  "wine": {
    "position": [
      -3,
      6
    ],
    "facing": "east"
  },
  "shoppingcart": {
    "position": [
      1,
      2
    ],
    "facing": "south"
  }
}
📊 Cognitive Map JSONs
🤖 Predicted
{
  "origin": "agent",
  "objects": {
    "shoppingcart": {
      "position": [
        2,
        3
      ],
      "facing": "-y"
    },
    "wine": {
      "position": [
        -3,
        6
      ]
    },
    "televison": {
      "position": [
        -3,
        4
      ],
      "facing": "-x"
    },
    "airplane": {
      "position": [
        -1,
        6
      ],
      "facing": "-x"
    }
  }
}
🎯 Ground Truth
{
  "televison": {
    "position": [
      -3,
      4
    ],
    "facing": "west"
  },
  "airplane": {
    "position": [
      -1,
      6
    ],
    "facing": "west"
  },
  "wine": {
    "position": [
      -3,
      6
    ],
    "facing": "east"
  },
  "shoppingcart": {
    "position": [
      1,
      2
    ],
    "facing": "south"
  }
}
🗺️ Symbolic Fog Map
################
#########......#
#########A.....#
#########......#
#########BD....#
#########.E....#
#########.F.C..#
#############*##
#......#......##
#......#......##
#......#......##
#......#......##
#......+......##
#......#......##
################
🔍 Fog Probe JSONs
📍 Candidates: A, B, C, D, E, F
🤖 Predicted: B, E, F
🎯 Ground Truth: E, B, F
🧠 Cognitive Map Metrics
Global:
dir: 0.658
facing: 1
pos: 0.748
overall: 0.802
Global (Full):
dir: 0.658
facing: 1
pos: 0.748
overall: 0.802
Local:
dir: 0.900
facing: 1
pos: 0.896
overall: 0.932
Local (Newly):
dir: 0.900
facing: 1
pos: 0.896
overall: 0.932
Fog Probe:
precision: 1
recall: 1
overall: 1
📈 Turn Metrics
node_coverage: 1
edge_coverage: 0.488
is_action_fail: False
step: 11
action_counts:
move: 4
rotate: 8
return: 0
observe: 11
term: 0
forced_term: 0
query: 0
information_gain: 0.784
reward: -0.100
is_done: False
is_valid_action: True
Previous state
State before Turn 11
Current state
State at Turn 11

🔄 Turn 12

💬 Assistant Action
Actions: [Term()]
📊 Cognitive Map JSONs
🤖 Predicted
{
  "agent": {
    "position": [
      2,
      5
    ],
    "facing": "north"
  },
  "truck": {
    "position": [
      -2,
      3
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      2,
      5
    ]
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      3,
      -3
    ],
    "facing": "north"
  },
  "chair": {
    "position": [
      2,
      -3
    ],
    "facing": "north"
  },
  "green door": {
    "position": [
      -3,
      -2
    ]
  },
  "scooter": {
    "position": [
      -6,
      2
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -6,
      -1
    ]
  },
  "table": {
    "position": [
      -6,
      -4
    ]
  },
  "bike": {
    "position": [
      -4,
      -1
    ],
    "facing": "west"
  },
  "shoppingcart": {
    "position": [
      4,
      8
    ],
    "facing": "south"
  },
  "wine": {
    "position": [
      0,
      9
    ]
  },
  "televison": {
    "position": [
      0,
      7
    ],
    "facing": "west"
  },
  "airplane": {
    "position": [
      1,
      9
    ],
    "facing": "west"
  }
}
🎯 Ground Truth (Observed)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "televison": {
    "position": [
      1,
      8
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      3,
      10
    ],
    "facing": "west"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "wine": {
    "position": [
      1,
      10
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "shoppingcart": {
    "position": [
      5,
      6
    ],
    "facing": "south"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  }
}
🎯 Ground Truth (Full)
{
  "truck": {
    "position": [
      -1,
      2
    ],
    "facing": "east"
  },
  "televison": {
    "position": [
      1,
      8
    ],
    "facing": "west"
  },
  "bike": {
    "position": [
      -3,
      2
    ],
    "facing": "west"
  },
  "chair": {
    "position": [
      1,
      -2
    ],
    "facing": "north"
  },
  "airplane": {
    "position": [
      3,
      10
    ],
    "facing": "west"
  },
  "scooter": {
    "position": [
      -7,
      2
    ],
    "facing": "east"
  },
  "backpack": {
    "position": [
      3,
      0
    ],
    "facing": "south"
  },
  "shoes": {
    "position": [
      4,
      -2
    ],
    "facing": "north"
  },
  "wine": {
    "position": [
      1,
      10
    ],
    "facing": "east"
  },
  "basket": {
    "position": [
      -5,
      0
    ],
    "facing": "east"
  },
  "shoppingcart": {
    "position": [
      5,
      6
    ],
    "facing": "south"
  },
  "table": {
    "position": [
      -5,
      -2
    ],
    "facing": "east"
  },
  "green door": {
    "position": [
      -2,
      -1
    ],
    "facing": "east"
  },
  "red door": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  },
  "agent": {
    "position": [
      4,
      4
    ],
    "facing": "north"
  }
}
🧠 Cognitive Map Metrics
Global:
dir: 0.733
facing: 1
pos: 0.753
overall: 0.829
Global (Full):
dir: 0.733
facing: 1
pos: 0.753
overall: 0.829
Local:
(none)
Local (Newly):
(none)
Fog Probe:
(none)
📈 Turn Metrics
node_coverage: 1
edge_coverage: 0.488
is_action_fail: False
step: 12
action_counts:
move: 4
rotate: 8
return: 0
observe: 11
term: 1
forced_term: 0
query: 0
information_gain: 0.784
reward: -0.100
is_done: True
is_valid_action: True
Previous state
State before Turn 12
Current state
State at Turn 12

🧭 False Belief Exploration

🔄 FB Turn 13

💬 Assistant Action
Actions: [Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Unchanged (all):
dir: 0.727
facing: 1
pos: 0.802
overall: 0.843
🧾 False Belief Object Sets
all_objects: ['airplane', 'backpack', 'basket', 'bike', 'chair', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'televison', 'truck', 'wine']
all_changed_objects: ['airplane', 'bike', 'chair', 'wine']
newly_observed_changed (this turn): []
unchanged_objects: ['backpack', 'basket', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'televison', 'truck']
pred_changed_keys: []
pred_unchanged_keys: ['airplane', 'backpack', 'basket', 'bike', 'chair', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'televison', 'truck', 'wine']
🧭 False Belief Info
step: 1
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'chair', 'ori': True}, {'name': 'airplane', 'ori': True}, {'name': 'wine', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: []
newly_observed_unchanged_objects: ['red door', 'truck']
Current state
State at Turn 1

🔄 FB Turn 14

💬 Assistant Action
Actions: [Rotate(90), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Changed: bike:
dir: None
pos: 0.856
facing: None
overall: None
Retention: bike:
dir: None
pos: 0.374
facing: None
overall: None
Unchanged (all):
dir: 0.727
facing: 0.714
pos: 0.734
overall: 0.725
🧾 False Belief Object Sets
all_objects: ['airplane', 'backpack', 'basket', 'bike', 'chair', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'televison', 'truck', 'wine']
all_changed_objects: ['airplane', 'bike', 'chair', 'wine']
newly_observed_changed (this turn): ['bike']
unchanged_objects: ['backpack', 'basket', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'televison', 'truck']
pred_changed_keys: ['bike']
pred_unchanged_keys: ['airplane', 'backpack', 'basket', 'bike', 'chair', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'television', 'truck', 'wine']
🧭 False Belief Info
step: 2
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'chair', 'ori': True}, {'name': 'airplane', 'ori': True}, {'name': 'wine', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: ['bike']
newly_observed_unchanged_objects: ['backpack', 'shoes']
Previous state
State before Turn 2
Current state
State at Turn 2

🔄 FB Turn 15

💬 Assistant Action
Actions: [Rotate(90), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Changed: chair:
dir: None
pos: None
facing: 1
overall: None
Retention: chair:
dir: None
pos: None
facing: 0
overall: None
Unchanged (all):
dir: 0.788
facing: 1
pos: 0.743
overall: 0.844
🧾 False Belief Object Sets
all_objects: ['airplane', 'backpack', 'basket', 'bike', 'chair', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'televison', 'truck', 'wine']
all_changed_objects: ['airplane', 'bike', 'chair', 'wine']
newly_observed_changed (this turn): ['chair']
unchanged_objects: ['backpack', 'basket', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'televison', 'truck']
pred_changed_keys: ['chair']
pred_unchanged_keys: ['airplane', 'backpack', 'basket', 'bike', 'chair', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'televison', 'truck', 'wine']
🧭 False Belief Info
step: 3
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'chair', 'ori': True}, {'name': 'airplane', 'ori': True}, {'name': 'wine', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: ['chair']
newly_observed_unchanged_objects: []
Previous state
State before Turn 3
Current state
State at Turn 3

🔄 FB Turn 16

💬 Assistant Action
Actions: [Rotate(90), JumpTo(green door), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Changed: wine:
dir: None
pos: 0.856
facing: None
overall: None
Retention: wine:
dir: None
pos: 0.238
facing: None
overall: None
Unchanged (all):
dir: 0.742
facing: 0.857
pos: 0.704
overall: 0.768
🧾 False Belief Object Sets
all_objects: ['airplane', 'backpack', 'basket', 'bike', 'chair', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'televison', 'truck', 'wine']
all_changed_objects: ['airplane', 'bike', 'chair', 'wine']
newly_observed_changed (this turn): ['wine']
unchanged_objects: ['backpack', 'basket', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'televison', 'truck']
pred_changed_keys: ['wine']
pred_unchanged_keys: ['airplane', 'backpack', 'basket', 'bike', 'chair', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'television', 'truck', 'wine']
🧭 False Belief Info
step: 4
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'chair', 'ori': True}, {'name': 'airplane', 'ori': True}, {'name': 'wine', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: ['wine']
newly_observed_unchanged_objects: ['scooter', 'table', 'basket']
Previous state
State before Turn 4
Current state
State at Turn 4

🔄 FB Turn 17

💬 Assistant Action
Actions: [Rotate(180), JumpTo(red door), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Unchanged (all):
dir: 0.727
facing: 1
pos: 0.743
overall: 0.823
🧾 False Belief Object Sets
all_objects: ['airplane', 'backpack', 'basket', 'bike', 'chair', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'televison', 'truck', 'wine']
all_changed_objects: ['airplane', 'bike', 'chair', 'wine']
newly_observed_changed (this turn): []
unchanged_objects: ['backpack', 'basket', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'televison', 'truck']
pred_changed_keys: []
pred_unchanged_keys: ['airplane', 'backpack', 'basket', 'bike', 'chair', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'televison', 'truck', 'wine']
🧭 False Belief Info
step: 5
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'chair', 'ori': True}, {'name': 'airplane', 'ori': True}, {'name': 'wine', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: []
newly_observed_unchanged_objects: []
Previous state
State before Turn 5
Current state
State at Turn 5

🔄 FB Turn 18

💬 Assistant Action
Actions: [Rotate(-90), Observe()]
📈 Turn Metrics
reward: -0.100
is_done: False
is_valid_action: True
🧭 False Belief Cogmap Metrics
Changed: airplane:
dir: None
pos: None
facing: 1
overall: None
Retention: airplane:
dir: None
pos: None
facing: 0
overall: None
Unchanged (all):
dir: 0.742
facing: 1
pos: 0.699
overall: 0.814
🧾 False Belief Object Sets
all_objects: ['airplane', 'backpack', 'basket', 'bike', 'chair', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'televison', 'truck', 'wine']
all_changed_objects: ['airplane', 'bike', 'chair', 'wine']
newly_observed_changed (this turn): ['airplane']
unchanged_objects: ['backpack', 'basket', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'televison', 'truck']
pred_changed_keys: ['airplane']
pred_unchanged_keys: ['airplane', 'backpack', 'basket', 'bike', 'chair', 'green door', 'red door', 'scooter', 'shoes', 'shoppingcart', 'table', 'televison', 'truck', 'wine']
🧭 False Belief Info
step: 6
correctly_identified_changes: None
f1_overall: None
f1_position: None
f1_facing: None
action_cost: None
action_cost_after_seen: None
ground_truth_changes: [{'name': 'chair', 'ori': True}, {'name': 'airplane', 'ori': True}, {'name': 'wine', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: []
newly_observed_changed_objects: ['airplane']
newly_observed_unchanged_objects: ['televison', 'shoppingcart']
Previous state
State before Turn 6
Current state
State at Turn 6

🔄 FB Turn 19

💬 Assistant Action
Actions: [Term(changes="bike: position, chair: orientation, wine: position, airplane: orientation")]
📈 Turn Metrics
reward: 1
is_done: True
is_valid_action: True
success: 1
🧭 False Belief Info
step: 7
correctly_identified_changes: 1
f1_overall: 1
f1_position: 1
f1_facing: 1
action_cost: 6
action_cost_after_seen: 0
ground_truth_changes: [{'name': 'chair', 'ori': True}, {'name': 'airplane', 'ori': True}, {'name': 'wine', 'pos': True}, {'name': 'bike', 'pos': True}]
reported_changes: [{'name': 'bike', 'pos': True}, {'name': 'chair', 'ori': True}, {'name': 'wine', 'pos': True}, {'name': 'airplane', 'ori': True}]
newly_observed_changed_objects: []
newly_observed_unchanged_objects: []
Previous state
State before Turn 7
Current state
State at Turn 7

📊 Evaluation (default)

📊 Task: DirectionEvaluationTask

Question 1 (ID: a2362e4d12a3effe)

✅ Evaluation Results
id: a2362e4d12a3effe
question: You return to your starting position and face north. From a Top-Down map, describe where wine is relative to table. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north east, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q1: DirectionEvaluationTask

Question 2 (ID: 92c1e7b466fabe79)

✅ Evaluation Results
id: 92c1e7b466fabe79
question: You return to your starting position and face north. From a Top-Down map, describe where televison is relative to chair. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Evaluation state
Q2: DirectionEvaluationTask

Question 3 (ID: 18270900f19204e6)

✅ Evaluation Results
id: 18270900f19204e6
question: You return to your starting position and face north. From a Top-Down map, describe where chair is relative to table. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: east, slightly far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q3: DirectionEvaluationTask

📊 Task: PovEvaluationTask

Question 1 (ID: 3976e12c4008e19b)

✅ Evaluation Results
id: 3976e12c4008e19b
question: Now you jump to shoes's direction, facing its direction. Describe where backpack is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Evaluation state
Q1: PovEvaluationTask

Question 2 (ID: b39109ecaa733d7c)

✅ Evaluation Results
id: b39109ecaa733d7c
question: Now you jump to backpack's direction, facing its direction. Describe where shoes is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Evaluation state
Q2: PovEvaluationTask

Question 3 (ID: 1c4c6cc481b955ac)

✅ Evaluation Results
id: 1c4c6cc481b955ac
question: Now you jump to airplane's direction, facing its direction. Describe where televison is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: default
Evaluation state
Q3: PovEvaluationTask

📊 Task: BackwardPovTextEvaluationTask

Question 1 (ID: 3c2ff1e4e461e1cf)

✅ Evaluation Results
id: 3c2ff1e4e461e1cf
question: Now you jump to an object's position, facing its direction. You observe that scooter is front, mid distance, facing backward; basket is front-left, mid distance. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: bike
final_pos: [6, 5]
final_ori: [-1, 0]
final_observation: [{'name': 'scooter', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'basket', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': None}]
object_positions:
truck: [8.0, 5.0]
televison: [10.0, 11.0]
bike: [6.0, 5.0]
chair: [10.0, 1.0]
airplane: [12.0, 13.0]
scooter: [2.0, 5.0]
backpack: [12.0, 3.0]
shoes: [13.0, 1.0]
wine: [10.0, 13.0]
basket: [4.0, 3.0]
shoppingcart: [14.0, 9.0]
table: [4.0, 1.0]
green door: [7.0, 2.0]
red door: [13.0, 7.0]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
room_id: 3
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: default
Evaluation state
Q1: BackwardPovTextEvaluationTask

Question 2 (ID: 697f480eae6ce726)

✅ Evaluation Results
id: 697f480eae6ce726
question: Now you jump to an object's position, facing its direction. You observe that truck is front-left, slightly far, facing right; backpack is front-right, mid distance, facing backward; red door is front-right, slightly far, on front wall. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: chair
final_pos: [10, 1]
final_ori: [0, 1]
final_observation: [{'name': 'truck', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'backpack', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'red door', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
object_positions:
truck: [8.0, 5.0]
televison: [10.0, 11.0]
bike: [6.0, 5.0]
chair: [10.0, 1.0]
airplane: [12.0, 13.0]
scooter: [2.0, 5.0]
backpack: [12.0, 3.0]
shoes: [13.0, 1.0]
wine: [10.0, 13.0]
basket: [4.0, 3.0]
shoppingcart: [14.0, 9.0]
table: [4.0, 1.0]
green door: [7.0, 2.0]
red door: [13.0, 7.0]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
room_id: 1
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: default
Evaluation state
Q2: BackwardPovTextEvaluationTask

Question 3 (ID: a8c816c15da597f7)

✅ Evaluation Results
id: a8c816c15da597f7
question: Now you jump to an object's position, facing its direction. You observe that bike is front, mid distance, facing backward; basket is front-right, mid distance; green door is front-right, slightly far, on front wall. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: scooter
final_pos: [2, 5]
final_ori: [1, 0]
final_observation: [{'name': 'bike', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'basket', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': None}, {'name': 'green door', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
object_positions:
truck: [8.0, 5.0]
televison: [10.0, 11.0]
bike: [6.0, 5.0]
chair: [10.0, 1.0]
airplane: [12.0, 13.0]
scooter: [2.0, 5.0]
backpack: [12.0, 3.0]
shoes: [13.0, 1.0]
wine: [10.0, 13.0]
basket: [4.0, 3.0]
shoppingcart: [14.0, 9.0]
table: [4.0, 1.0]
green door: [7.0, 2.0]
red door: [13.0, 7.0]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
room_id: 3
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: default
Evaluation state
Q3: BackwardPovTextEvaluationTask

📊 Task: Action2ViewEvaluationTask

Question 1 (ID: ead12aceba22dea5)

✅ Evaluation Results
id: ead12aceba22dea5
question: You return to your starting position and face north. You will execute the following action sequence: 1. Jump to the object at front-right, slightly far. 2. Jump to the object at front-slight-left, slightly far. 3. Rotate(-180) After executing the actions, what is the ego relation of shoppingcart relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q1: Action2ViewEvaluationTask

Question 2 (ID: 329783e3cf710e9c)

✅ Evaluation Results
id: 329783e3cf710e9c
question: You return to your starting position and face north. You will execute the following action sequence: 1. Jump to the object at front-left, mid distance. 2. Rotate(-180) 3. Jump to the object at front-slight-right, mid distance. 4. Rotate(-90) After executing the actions, what is the ego relation of shoes relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-slight-right, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q2: Action2ViewEvaluationTask

Question 3 (ID: 9778d336cf4718dc)

✅ Evaluation Results
id: 9778d336cf4718dc
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(-90) 2. Jump to the object at front-left, mid distance. 3. Rotate(90) 4. Jump to the object at front-slight-left, mid distance. 5. Rotate(-180) After executing the actions, what is the ego relation of basket relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-right, mid distance
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q3: Action2ViewEvaluationTask

📊 Task: View2ActionTextEvaluationTask

Question 1 (ID: d596a6d6b93f9b1f)

✅ Evaluation Results
id: d596a6d6b93f9b1f
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: bike is at front-right, slightly far, facing left; scooter is at front-left, slightly far, facing right; basket is at front, near What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [4, 1]
final_ori: [0, 1]
room_id: 3
init_pos: [9, 3]
init_ori: [0, 1]
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
minimal_plan: [['rotate', -90], ['jumpto', 'green door'], ['jumpto', 'table'], ['rotate', 90]]
final_observation: [{'name': 'bike', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing left'}, {'name': 'scooter', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'basket', 'direction': 'front', 'distance': 'near', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [4.0, 1.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: default
Evaluation state
Q1: View2ActionTextEvaluationTask

Question 2 (ID: bc2218d7bd5246e5)

✅ Evaluation Results
id: bc2218d7bd5246e5
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: televison is at front-left, mid distance, facing forward; wine is at front, near What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [12, 13]
final_ori: [-1, 0]
room_id: 2
init_pos: [9, 3]
init_ori: [0, 1]
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
minimal_plan: [['jumpto', 'red door'], ['jumpto', 'airplane'], ['rotate', -90]]
final_observation: [{'name': 'televison', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'wine', 'direction': 'front', 'distance': 'near', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [12.0, 13.0]
final_ori: [-1, 0]
visible_match: True
score: 1
evaluation_mode: default
Evaluation state
Q2: View2ActionTextEvaluationTask

Question 3 (ID: 2dd6b807f022852a)

✅ Evaluation Results
id: 2dd6b807f022852a
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: airplane is at front-left, slightly far, facing left; wine is at front-left, slightly far What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [14, 9]
final_ori: [0, 1]
room_id: 2
init_pos: [9, 3]
init_ori: [0, 1]
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
minimal_plan: [['jumpto', 'red door'], ['jumpto', 'shoppingcart']]
final_observation: [{'name': 'airplane', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing left'}, {'name': 'wine', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: True
final_pos: [13.0, 7.0]
final_ori: [0, 1]
visible_match: False
score: 0
evaluation_mode: default
Evaluation state
Q3: View2ActionTextEvaluationTask

📊 Task: AlloMappingEvaluationTask

Question 1 (ID: ba2ae1056e95b4f4)

✅ Evaluation Results
id: ba2ae1056e95b4f4
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: wine, table, airplane. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[1, 10], [-5, -2], [3, 10]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.833
threshold: 0.900
score: 0.833
evaluation_mode: default
Evaluation state
Q1: AlloMappingEvaluationTask

Question 2 (ID: ee801e392523d89c)

✅ Evaluation Results
id: ee801e392523d89c
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: televison, chair, shoes, truck, wine. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[1, 8], [1, -2], [4, -2], [-1, 2], [1, 10]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.806
threshold: 0.900
score: 0.806
evaluation_mode: default
Evaluation state
Q2: AlloMappingEvaluationTask

Question 3 (ID: 4f53c659975c6929)

✅ Evaluation Results
id: 4f53c659975c6929
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: truck, shoes, basket. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-1, 2], [4, -2], [-5, 0]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.864
threshold: 0.900
score: 0.864
evaluation_mode: default
Evaluation state
Q3: AlloMappingEvaluationTask

📊 Task: RotEvaluationTask

Question 1 (ID: 8bd34cfbaf041b94)

✅ Evaluation Results
id: 8bd34cfbaf041b94
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: shoes, shoppingcart, wine. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['wine', 'shoppingcart', 'shoes']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q1: RotEvaluationTask

Question 2 (ID: a27ba852e9d2d241)

✅ Evaluation Results
id: a27ba852e9d2d241
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: chair, shoes, shoppingcart, table, televison. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['televison', 'shoppingcart', 'shoes', 'chair', 'table']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q2: RotEvaluationTask

Question 3 (ID: f6d3c113c4f20f24)

✅ Evaluation Results
id: f6d3c113c4f20f24
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: backpack, chair, shoppingcart, wine. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['wine', 'shoppingcart', 'backpack', 'chair']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q3: RotEvaluationTask

📊 Task: RotDualEvaluationTask

Question 1 (ID: 85ffaf62b377f438)

✅ Evaluation Results
id: 85ffaf62b377f438
question: You return to your starting position and face north. You performed a complete 360° rotation in place. Assume all walls are removed (you can see through walls), so every object is visible. During the rotation, these objects appeared directly in front of you in this order: wine, shoppingcart, shoes Based on this sequence, in which direction did you rotate? Answer format: clockwise or counterclockwise Example: clockwise
action: None
answer: clockwise
task_type: RotDualEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q1: RotDualEvaluationTask

Question 2 (ID: c305186d2c81e65b)

✅ Evaluation Results
id: c305186d2c81e65b
question: You return to your starting position and face north. You performed a complete 360° rotation in place. Assume all walls are removed (you can see through walls), so every object is visible. During the rotation, these objects appeared directly in front of you in this order: televison, shoppingcart, shoes, chair, table Based on this sequence, in which direction did you rotate? Answer format: clockwise or counterclockwise Example: clockwise
action: None
answer: clockwise
task_type: RotDualEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q2: RotDualEvaluationTask

Question 3 (ID: e727bfe18d95b6fc)

✅ Evaluation Results
id: e727bfe18d95b6fc
question: You return to your starting position and face north. You performed a complete 360° rotation in place. Assume all walls are removed (you can see through walls), so every object is visible. During the rotation, these objects appeared directly in front of you in this order: wine, shoppingcart, backpack, chair Based on this sequence, in which direction did you rotate? Answer format: clockwise or counterclockwise Example: clockwise
action: None
answer: clockwise
task_type: RotDualEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q3: RotDualEvaluationTask

📊 Task: Location2ViewEvaluationTask

Question 1 (ID: 59a7cc1d9f9a2684)

✅ Evaluation Results
id: 59a7cc1d9f9a2684
question: Still treat your initial position as origin (0, 0) You move to (2, 2) and face west. What is the egocentric relation of truck? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front, mid distance
task_type: Action2LocationEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q1: Location2ViewEvaluationTask

Question 2 (ID: e28473b0be6f4b18)

✅ Evaluation Results
id: e28473b0be6f4b18
question: Still treat your initial position as origin (0, 0) You move to (0, 2) and face south. What is the egocentric relation of chair? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-slight-left, slightly far
task_type: Action2LocationEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q2: Location2ViewEvaluationTask

Question 3 (ID: 6e766b24aeabe349)

✅ Evaluation Results
id: 6e766b24aeabe349
question: Still treat your initial position as origin (0, 0) You move to (0, 0) and face east. What is the egocentric relation of backpack? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front, mid distance
task_type: Action2LocationEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: default
Evaluation state
Q3: Location2ViewEvaluationTask

📊 Task: View2LocationTextEvaluationTask

Question 1 (ID: 837814331d62f020)

✅ Evaluation Results
id: 837814331d62f020
question: You move to a new location and face west. You observe: truck is at front, mid distance, facing backward; green door is at front-left, slightly far, on front wall Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [2, 2]
final_pos: [11, 5]
final_ori: [-1, 0]
room_id: 1
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
final_observation: [{'name': 'truck', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'green door', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'on front wall'}]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: Location2ActionTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.607
raw_score: 0.607
score: 0.607
evaluation_mode: default
Evaluation state
Q1: View2LocationTextEvaluationTask

Question 2 (ID: f60f38aad8e949fb)

✅ Evaluation Results
id: f60f38aad8e949fb
question: You move to a new location and face south. You observe: chair is at front-slight-left, slightly far, facing backward; shoes is at front-left, slightly far, facing backward; green door is at front-right, mid distance, on right wall Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [0, 2]
final_pos: [9, 5]
final_ori: [0, -1]
room_id: 1
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
final_observation: [{'name': 'chair', 'direction': 'front-slight-left', 'distance': 'slightly far', 'orientation': 'facing backward'}, {'name': 'shoes', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing backward'}, {'name': 'green door', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'on right wall'}]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: Location2ActionTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: default
Evaluation state
Q2: View2LocationTextEvaluationTask

Question 3 (ID: 9bffb5402a1f2564)

✅ Evaluation Results
id: 9bffb5402a1f2564
question: You move to a new location and face east. You observe: backpack is at front, mid distance, facing right; shoes is at front-right, slightly far, facing left; red door is at front-left, slightly far, on left wall Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [0, 0]
final_pos: [9, 3]
final_ori: [1, 0]
room_id: 1
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
final_observation: [{'name': 'backpack', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'shoes', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing left'}, {'name': 'red door', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'on left wall'}]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: Location2ActionTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: default
Evaluation state
Q3: View2LocationTextEvaluationTask

📊 Evaluation (prompt_cogmap)

📊 Task: DirectionEvaluationTask

Question 1 (ID: a2362e4d12a3effe)

✅ Evaluation Results
id: a2362e4d12a3effe
question: You return to your starting position and face north. From a Top-Down map, describe where wine is relative to table. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north east, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Evaluation state
Q1: DirectionEvaluationTask

Question 2 (ID: 92c1e7b466fabe79)

✅ Evaluation Results
id: 92c1e7b466fabe79
question: You return to your starting position and face north. From a Top-Down map, describe where televison is relative to chair. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Evaluation state
Q2: DirectionEvaluationTask

Question 3 (ID: 18270900f19204e6)

✅ Evaluation Results
id: 18270900f19204e6
question: You return to your starting position and face north. From a Top-Down map, describe where chair is relative to table. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: east, slightly far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Evaluation state
Q3: DirectionEvaluationTask

📊 Task: PovEvaluationTask

Question 1 (ID: 3976e12c4008e19b)

✅ Evaluation Results
id: 3976e12c4008e19b
question: Now you jump to shoes's direction, facing its direction. Describe where backpack is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Evaluation state
Q1: PovEvaluationTask

Question 2 (ID: b39109ecaa733d7c)

✅ Evaluation Results
id: b39109ecaa733d7c
question: Now you jump to backpack's direction, facing its direction. Describe where shoes is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Evaluation state
Q2: PovEvaluationTask

Question 3 (ID: 1c4c6cc481b955ac)

✅ Evaluation Results
id: 1c4c6cc481b955ac
question: Now you jump to airplane's direction, facing its direction. Describe where televison is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Evaluation state
Q3: PovEvaluationTask

📊 Task: BackwardPovTextEvaluationTask

Question 1 (ID: 3c2ff1e4e461e1cf)

✅ Evaluation Results
id: 3c2ff1e4e461e1cf
question: Now you jump to an object's position, facing its direction. You observe that scooter is front, mid distance, facing backward; basket is front-left, mid distance. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: bike
final_pos: [6, 5]
final_ori: [-1, 0]
final_observation: [{'name': 'scooter', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'basket', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': None}]
object_positions:
truck: [8.0, 5.0]
televison: [10.0, 11.0]
bike: [6.0, 5.0]
chair: [10.0, 1.0]
airplane: [12.0, 13.0]
scooter: [2.0, 5.0]
backpack: [12.0, 3.0]
shoes: [13.0, 1.0]
wine: [10.0, 13.0]
basket: [4.0, 3.0]
shoppingcart: [14.0, 9.0]
table: [4.0, 1.0]
green door: [7.0, 2.0]
red door: [13.0, 7.0]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
room_id: 3
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q1: BackwardPovTextEvaluationTask

Question 2 (ID: 697f480eae6ce726)

✅ Evaluation Results
id: 697f480eae6ce726
question: Now you jump to an object's position, facing its direction. You observe that truck is front-left, slightly far, facing right; backpack is front-right, mid distance, facing backward; red door is front-right, slightly far, on front wall. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: chair
final_pos: [10, 1]
final_ori: [0, 1]
final_observation: [{'name': 'truck', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'backpack', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'red door', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
object_positions:
truck: [8.0, 5.0]
televison: [10.0, 11.0]
bike: [6.0, 5.0]
chair: [10.0, 1.0]
airplane: [12.0, 13.0]
scooter: [2.0, 5.0]
backpack: [12.0, 3.0]
shoes: [13.0, 1.0]
wine: [10.0, 13.0]
basket: [4.0, 3.0]
shoppingcart: [14.0, 9.0]
table: [4.0, 1.0]
green door: [7.0, 2.0]
red door: [13.0, 7.0]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
room_id: 1
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q2: BackwardPovTextEvaluationTask

Question 3 (ID: a8c816c15da597f7)

✅ Evaluation Results
id: a8c816c15da597f7
question: Now you jump to an object's position, facing its direction. You observe that bike is front, mid distance, facing backward; basket is front-right, mid distance; green door is front-right, slightly far, on front wall. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: scooter
final_pos: [2, 5]
final_ori: [1, 0]
final_observation: [{'name': 'bike', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'basket', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': None}, {'name': 'green door', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
object_positions:
truck: [8.0, 5.0]
televison: [10.0, 11.0]
bike: [6.0, 5.0]
chair: [10.0, 1.0]
airplane: [12.0, 13.0]
scooter: [2.0, 5.0]
backpack: [12.0, 3.0]
shoes: [13.0, 1.0]
wine: [10.0, 13.0]
basket: [4.0, 3.0]
shoppingcart: [14.0, 9.0]
table: [4.0, 1.0]
green door: [7.0, 2.0]
red door: [13.0, 7.0]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
room_id: 3
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q3: BackwardPovTextEvaluationTask

📊 Task: Action2ViewEvaluationTask

Question 1 (ID: ead12aceba22dea5)

✅ Evaluation Results
id: ead12aceba22dea5
question: You return to your starting position and face north. You will execute the following action sequence: 1. Jump to the object at front-right, slightly far. 2. Jump to the object at front-slight-left, slightly far. 3. Rotate(-180) After executing the actions, what is the ego relation of shoppingcart relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q1: Action2ViewEvaluationTask

Question 2 (ID: 329783e3cf710e9c)

✅ Evaluation Results
id: 329783e3cf710e9c
question: You return to your starting position and face north. You will execute the following action sequence: 1. Jump to the object at front-left, mid distance. 2. Rotate(-180) 3. Jump to the object at front-slight-right, mid distance. 4. Rotate(-90) After executing the actions, what is the ego relation of shoes relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-slight-right, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q2: Action2ViewEvaluationTask

Question 3 (ID: 9778d336cf4718dc)

✅ Evaluation Results
id: 9778d336cf4718dc
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(-90) 2. Jump to the object at front-left, mid distance. 3. Rotate(90) 4. Jump to the object at front-slight-left, mid distance. 5. Rotate(-180) After executing the actions, what is the ego relation of basket relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-right, mid distance
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q3: Action2ViewEvaluationTask

📊 Task: View2ActionTextEvaluationTask

Question 1 (ID: d596a6d6b93f9b1f)

✅ Evaluation Results
id: d596a6d6b93f9b1f
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: bike is at front-right, slightly far, facing left; scooter is at front-left, slightly far, facing right; basket is at front, near What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [4, 1]
final_ori: [0, 1]
room_id: 3
init_pos: [9, 3]
init_ori: [0, 1]
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
minimal_plan: [['rotate', -90], ['jumpto', 'green door'], ['jumpto', 'table'], ['rotate', 90]]
final_observation: [{'name': 'bike', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing left'}, {'name': 'scooter', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'basket', 'direction': 'front', 'distance': 'near', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [4.0, 1.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q1: View2ActionTextEvaluationTask

Question 2 (ID: bc2218d7bd5246e5)

✅ Evaluation Results
id: bc2218d7bd5246e5
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: televison is at front-left, mid distance, facing forward; wine is at front, near What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [12, 13]
final_ori: [-1, 0]
room_id: 2
init_pos: [9, 3]
init_ori: [0, 1]
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
minimal_plan: [['jumpto', 'red door'], ['jumpto', 'airplane'], ['rotate', -90]]
final_observation: [{'name': 'televison', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'wine', 'direction': 'front', 'distance': 'near', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [12.0, 13.0]
final_ori: [-1, 0]
visible_match: True
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q2: View2ActionTextEvaluationTask

Question 3 (ID: 2dd6b807f022852a)

✅ Evaluation Results
id: 2dd6b807f022852a
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: airplane is at front-left, slightly far, facing left; wine is at front-left, slightly far What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [14, 9]
final_ori: [0, 1]
room_id: 2
init_pos: [9, 3]
init_ori: [0, 1]
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
minimal_plan: [['jumpto', 'red door'], ['jumpto', 'shoppingcart']]
final_observation: [{'name': 'airplane', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing left'}, {'name': 'wine', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: True
final_pos: [13.0, 7.0]
final_ori: [0, 1]
visible_match: False
score: 0
evaluation_mode: prompt_cogmap
Evaluation state
Q3: View2ActionTextEvaluationTask

📊 Task: AlloMappingEvaluationTask

Question 1 (ID: ba2ae1056e95b4f4)

✅ Evaluation Results
id: ba2ae1056e95b4f4
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: wine, table, airplane. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[1, 10], [-5, -2], [3, 10]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.860
threshold: 0.900
score: 0.860
evaluation_mode: prompt_cogmap
Evaluation state
Q1: AlloMappingEvaluationTask

Question 2 (ID: ee801e392523d89c)

✅ Evaluation Results
id: ee801e392523d89c
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: televison, chair, shoes, truck, wine. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[1, 8], [1, -2], [4, -2], [-1, 2], [1, 10]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.747
threshold: 0.900
score: 0.747
evaluation_mode: prompt_cogmap
Evaluation state
Q2: AlloMappingEvaluationTask

Question 3 (ID: 4f53c659975c6929)

✅ Evaluation Results
id: 4f53c659975c6929
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: truck, shoes, basket. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-1, 2], [4, -2], [-5, 0]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.877
threshold: 0.900
score: 0.877
evaluation_mode: prompt_cogmap
Evaluation state
Q3: AlloMappingEvaluationTask

📊 Task: RotEvaluationTask

Question 1 (ID: 8bd34cfbaf041b94)

✅ Evaluation Results
id: 8bd34cfbaf041b94
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: shoes, shoppingcart, wine. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['wine', 'shoppingcart', 'shoes']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: prompt_cogmap
Evaluation state
Q1: RotEvaluationTask

Question 2 (ID: a27ba852e9d2d241)

✅ Evaluation Results
id: a27ba852e9d2d241
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: chair, shoes, shoppingcart, table, televison. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['televison', 'shoppingcart', 'shoes', 'chair', 'table']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: prompt_cogmap
Evaluation state
Q2: RotEvaluationTask

Question 3 (ID: f6d3c113c4f20f24)

✅ Evaluation Results
id: f6d3c113c4f20f24
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: backpack, chair, shoppingcart, wine. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['wine', 'shoppingcart', 'backpack', 'chair']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: prompt_cogmap
Evaluation state
Q3: RotEvaluationTask

📊 Task: Location2ViewEvaluationTask

Question 1 (ID: 59a7cc1d9f9a2684)

✅ Evaluation Results
id: 59a7cc1d9f9a2684
question: Still treat your initial position as origin (0, 0) You move to (2, 2) and face west. What is the egocentric relation of truck? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q1: Location2ViewEvaluationTask

Question 2 (ID: e28473b0be6f4b18)

✅ Evaluation Results
id: e28473b0be6f4b18
question: Still treat your initial position as origin (0, 0) You move to (0, 2) and face south. What is the egocentric relation of chair? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-slight-left, slightly far
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: prompt_cogmap
Evaluation state
Q2: Location2ViewEvaluationTask

Question 3 (ID: 6e766b24aeabe349)

✅ Evaluation Results
id: 6e766b24aeabe349
question: Still treat your initial position as origin (0, 0) You move to (0, 0) and face east. What is the egocentric relation of backpack? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q3: Location2ViewEvaluationTask

📊 Task: View2LocationTextEvaluationTask

Question 1 (ID: 837814331d62f020)

✅ Evaluation Results
id: 837814331d62f020
question: You move to a new location and face west. You observe: truck is at front, mid distance, facing backward; green door is at front-left, slightly far, on front wall Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [2, 2]
final_pos: [11, 5]
final_ori: [-1, 0]
room_id: 1
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
final_observation: [{'name': 'truck', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'green door', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'on front wall'}]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q1: View2LocationTextEvaluationTask

Question 2 (ID: f60f38aad8e949fb)

✅ Evaluation Results
id: f60f38aad8e949fb
question: You move to a new location and face south. You observe: chair is at front-slight-left, slightly far, facing backward; shoes is at front-left, slightly far, facing backward; green door is at front-right, mid distance, on right wall Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [0, 2]
final_pos: [9, 5]
final_ori: [0, -1]
room_id: 1
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
final_observation: [{'name': 'chair', 'direction': 'front-slight-left', 'distance': 'slightly far', 'orientation': 'facing backward'}, {'name': 'shoes', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing backward'}, {'name': 'green door', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'on right wall'}]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q2: View2LocationTextEvaluationTask

Question 3 (ID: 9bffb5402a1f2564)

✅ Evaluation Results
id: 9bffb5402a1f2564
question: You move to a new location and face east. You observe: backpack is at front, mid distance, facing right; shoes is at front-right, slightly far, facing left; red door is at front-left, slightly far, on left wall Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [0, 0]
final_pos: [9, 3]
final_ori: [1, 0]
room_id: 1
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
final_observation: [{'name': 'backpack', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'shoes', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing left'}, {'name': 'red door', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'on left wall'}]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: prompt_cogmap
Evaluation state
Q3: View2LocationTextEvaluationTask

📊 Evaluation (use_gt_cogmap)

📊 Task: DirectionEvaluationTask

Question 1 (ID: a2362e4d12a3effe)

✅ Evaluation Results
id: a2362e4d12a3effe
question: You return to your starting position and face north. From a Top-Down map, describe where wine is relative to table. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north east, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: DirectionEvaluationTask

Question 2 (ID: 92c1e7b466fabe79)

✅ Evaluation Results
id: 92c1e7b466fabe79
question: You return to your starting position and face north. From a Top-Down map, describe where televison is relative to chair. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: DirectionEvaluationTask

Question 3 (ID: 18270900f19204e6)

✅ Evaluation Results
id: 18270900f19204e6
question: You return to your starting position and face north. From a Top-Down map, describe where chair is relative to table. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: east, slightly far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: DirectionEvaluationTask

📊 Task: PovEvaluationTask

Question 1 (ID: 3976e12c4008e19b)

✅ Evaluation Results
id: 3976e12c4008e19b
question: Now you jump to shoes's direction, facing its direction. Describe where backpack is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: PovEvaluationTask

Question 2 (ID: b39109ecaa733d7c)

✅ Evaluation Results
id: b39109ecaa733d7c
question: Now you jump to backpack's direction, facing its direction. Describe where shoes is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: PovEvaluationTask

Question 3 (ID: 1c4c6cc481b955ac)

✅ Evaluation Results
id: 1c4c6cc481b955ac
question: Now you jump to airplane's direction, facing its direction. Describe where televison is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: PovEvaluationTask

📊 Task: BackwardPovTextEvaluationTask

Question 1 (ID: 3c2ff1e4e461e1cf)

✅ Evaluation Results
id: 3c2ff1e4e461e1cf
question: Now you jump to an object's position, facing its direction. You observe that scooter is front, mid distance, facing backward; basket is front-left, mid distance. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: bike
final_pos: [6, 5]
final_ori: [-1, 0]
final_observation: [{'name': 'scooter', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'basket', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': None}]
object_positions:
truck: [8.0, 5.0]
televison: [10.0, 11.0]
bike: [6.0, 5.0]
chair: [10.0, 1.0]
airplane: [12.0, 13.0]
scooter: [2.0, 5.0]
backpack: [12.0, 3.0]
shoes: [13.0, 1.0]
wine: [10.0, 13.0]
basket: [4.0, 3.0]
shoppingcart: [14.0, 9.0]
table: [4.0, 1.0]
green door: [7.0, 2.0]
red door: [13.0, 7.0]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
room_id: 3
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: BackwardPovTextEvaluationTask

Question 2 (ID: 697f480eae6ce726)

✅ Evaluation Results
id: 697f480eae6ce726
question: Now you jump to an object's position, facing its direction. You observe that truck is front-left, slightly far, facing right; backpack is front-right, mid distance, facing backward; red door is front-right, slightly far, on front wall. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: chair
final_pos: [10, 1]
final_ori: [0, 1]
final_observation: [{'name': 'truck', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'backpack', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'red door', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
object_positions:
truck: [8.0, 5.0]
televison: [10.0, 11.0]
bike: [6.0, 5.0]
chair: [10.0, 1.0]
airplane: [12.0, 13.0]
scooter: [2.0, 5.0]
backpack: [12.0, 3.0]
shoes: [13.0, 1.0]
wine: [10.0, 13.0]
basket: [4.0, 3.0]
shoppingcart: [14.0, 9.0]
table: [4.0, 1.0]
green door: [7.0, 2.0]
red door: [13.0, 7.0]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
room_id: 1
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: BackwardPovTextEvaluationTask

Question 3 (ID: a8c816c15da597f7)

✅ Evaluation Results
id: a8c816c15da597f7
question: Now you jump to an object's position, facing its direction. You observe that bike is front, mid distance, facing backward; basket is front-right, mid distance; green door is front-right, slightly far, on front wall. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: scooter
final_pos: [2, 5]
final_ori: [1, 0]
final_observation: [{'name': 'bike', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'basket', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': None}, {'name': 'green door', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
object_positions:
truck: [8.0, 5.0]
televison: [10.0, 11.0]
bike: [6.0, 5.0]
chair: [10.0, 1.0]
airplane: [12.0, 13.0]
scooter: [2.0, 5.0]
backpack: [12.0, 3.0]
shoes: [13.0, 1.0]
wine: [10.0, 13.0]
basket: [4.0, 3.0]
shoppingcart: [14.0, 9.0]
table: [4.0, 1.0]
green door: [7.0, 2.0]
red door: [13.0, 7.0]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
room_id: 3
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: BackwardPovTextEvaluationTask

📊 Task: Action2ViewEvaluationTask

Question 1 (ID: ead12aceba22dea5)

✅ Evaluation Results
id: ead12aceba22dea5
question: You return to your starting position and face north. You will execute the following action sequence: 1. Jump to the object at front-right, slightly far. 2. Jump to the object at front-slight-left, slightly far. 3. Rotate(-180) After executing the actions, what is the ego relation of shoppingcart relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: Action2ViewEvaluationTask

Question 2 (ID: 329783e3cf710e9c)

✅ Evaluation Results
id: 329783e3cf710e9c
question: You return to your starting position and face north. You will execute the following action sequence: 1. Jump to the object at front-left, mid distance. 2. Rotate(-180) 3. Jump to the object at front-slight-right, mid distance. 4. Rotate(-90) After executing the actions, what is the ego relation of shoes relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-slight-right, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: Action2ViewEvaluationTask

Question 3 (ID: 9778d336cf4718dc)

✅ Evaluation Results
id: 9778d336cf4718dc
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(-90) 2. Jump to the object at front-left, mid distance. 3. Rotate(90) 4. Jump to the object at front-slight-left, mid distance. 5. Rotate(-180) After executing the actions, what is the ego relation of basket relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-right, mid distance
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: Action2ViewEvaluationTask

📊 Task: View2ActionTextEvaluationTask

Question 1 (ID: d596a6d6b93f9b1f)

✅ Evaluation Results
id: d596a6d6b93f9b1f
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: bike is at front-right, slightly far, facing left; scooter is at front-left, slightly far, facing right; basket is at front, near What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [4, 1]
final_ori: [0, 1]
room_id: 3
init_pos: [9, 3]
init_ori: [0, 1]
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
minimal_plan: [['rotate', -90], ['jumpto', 'green door'], ['jumpto', 'table'], ['rotate', 90]]
final_observation: [{'name': 'bike', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing left'}, {'name': 'scooter', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'basket', 'direction': 'front', 'distance': 'near', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [4.0, 1.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: View2ActionTextEvaluationTask

Question 2 (ID: bc2218d7bd5246e5)

✅ Evaluation Results
id: bc2218d7bd5246e5
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: televison is at front-left, mid distance, facing forward; wine is at front, near What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [12, 13]
final_ori: [-1, 0]
room_id: 2
init_pos: [9, 3]
init_ori: [0, 1]
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
minimal_plan: [['jumpto', 'red door'], ['jumpto', 'airplane'], ['rotate', -90]]
final_observation: [{'name': 'televison', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'wine', 'direction': 'front', 'distance': 'near', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [12.0, 13.0]
final_ori: [-1, 0]
visible_match: True
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: View2ActionTextEvaluationTask

Question 3 (ID: 2dd6b807f022852a)

✅ Evaluation Results
id: 2dd6b807f022852a
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: airplane is at front-left, slightly far, facing left; wine is at front-left, slightly far What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [14, 9]
final_ori: [0, 1]
room_id: 2
init_pos: [9, 3]
init_ori: [0, 1]
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
minimal_plan: [['jumpto', 'red door'], ['jumpto', 'shoppingcart']]
final_observation: [{'name': 'airplane', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing left'}, {'name': 'wine', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [14.0, 9.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: View2ActionTextEvaluationTask

📊 Task: AlloMappingEvaluationTask

Question 1 (ID: ba2ae1056e95b4f4)

✅ Evaluation Results
id: ba2ae1056e95b4f4
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: wine, table, airplane. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[1, 10], [-5, -2], [3, 10]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
threshold: 0.900
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: AlloMappingEvaluationTask

Question 2 (ID: ee801e392523d89c)

✅ Evaluation Results
id: ee801e392523d89c
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: televison, chair, shoes, truck, wine. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[1, 8], [1, -2], [4, -2], [-1, 2], [1, 10]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
threshold: 0.900
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: AlloMappingEvaluationTask

Question 3 (ID: 4f53c659975c6929)

✅ Evaluation Results
id: 4f53c659975c6929
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: truck, shoes, basket. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-1, 2], [4, -2], [-5, 0]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
threshold: 0.900
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: AlloMappingEvaluationTask

📊 Task: RotEvaluationTask

Question 1 (ID: 8bd34cfbaf041b94)

✅ Evaluation Results
id: 8bd34cfbaf041b94
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: shoes, shoppingcart, wine. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['wine', 'shoppingcart', 'shoes']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: RotEvaluationTask

Question 2 (ID: a27ba852e9d2d241)

✅ Evaluation Results
id: a27ba852e9d2d241
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: chair, shoes, shoppingcart, table, televison. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['televison', 'shoppingcart', 'shoes', 'chair', 'table']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: RotEvaluationTask

Question 3 (ID: f6d3c113c4f20f24)

✅ Evaluation Results
id: f6d3c113c4f20f24
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: backpack, chair, shoppingcart, wine. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['wine', 'shoppingcart', 'backpack', 'chair']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: RotEvaluationTask

📊 Task: Location2ViewEvaluationTask

Question 1 (ID: 59a7cc1d9f9a2684)

✅ Evaluation Results
id: 59a7cc1d9f9a2684
question: Still treat your initial position as origin (0, 0) You move to (2, 2) and face west. What is the egocentric relation of truck? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: Location2ViewEvaluationTask

Question 2 (ID: e28473b0be6f4b18)

✅ Evaluation Results
id: e28473b0be6f4b18
question: Still treat your initial position as origin (0, 0) You move to (0, 2) and face south. What is the egocentric relation of chair? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-slight-left, slightly far
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: Location2ViewEvaluationTask

Question 3 (ID: 6e766b24aeabe349)

✅ Evaluation Results
id: 6e766b24aeabe349
question: Still treat your initial position as origin (0, 0) You move to (0, 0) and face east. What is the egocentric relation of backpack? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: Location2ViewEvaluationTask

📊 Task: View2LocationTextEvaluationTask

Question 1 (ID: 837814331d62f020)

✅ Evaluation Results
id: 837814331d62f020
question: You move to a new location and face west. You observe: truck is at front, mid distance, facing backward; green door is at front-left, slightly far, on front wall Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [2, 2]
final_pos: [11, 5]
final_ori: [-1, 0]
room_id: 1
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
final_observation: [{'name': 'truck', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'green door', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'on front wall'}]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q1: View2LocationTextEvaluationTask

Question 2 (ID: f60f38aad8e949fb)

✅ Evaluation Results
id: f60f38aad8e949fb
question: You move to a new location and face south. You observe: chair is at front-slight-left, slightly far, facing backward; shoes is at front-left, slightly far, facing backward; green door is at front-right, mid distance, on right wall Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [0, 2]
final_pos: [9, 5]
final_ori: [0, -1]
room_id: 1
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
final_observation: [{'name': 'chair', 'direction': 'front-slight-left', 'distance': 'slightly far', 'orientation': 'facing backward'}, {'name': 'shoes', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing backward'}, {'name': 'green door', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'on right wall'}]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q2: View2LocationTextEvaluationTask

Question 3 (ID: 9bffb5402a1f2564)

✅ Evaluation Results
id: 9bffb5402a1f2564
question: You move to a new location and face east. You observe: backpack is at front, mid distance, facing right; shoes is at front-right, slightly far, facing left; red door is at front-left, slightly far, on left wall Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [0, 0]
final_pos: [9, 3]
final_ori: [1, 0]
room_id: 1
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
final_observation: [{'name': 'backpack', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'shoes', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing left'}, {'name': 'red door', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'on left wall'}]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_gt_cogmap
Evaluation state
Q3: View2LocationTextEvaluationTask

📊 Evaluation (use_model_cogmap)

📊 Task: DirectionEvaluationTask

Question 1 (ID: a2362e4d12a3effe)

✅ Evaluation Results
id: a2362e4d12a3effe
question: You return to your starting position and face north. From a Top-Down map, describe where wine is relative to table. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north east, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q1: DirectionEvaluationTask

Question 2 (ID: 92c1e7b466fabe79)

✅ Evaluation Results
id: 92c1e7b466fabe79
question: You return to your starting position and face north. From a Top-Down map, describe where televison is relative to chair. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: north, far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Evaluation state
Q2: DirectionEvaluationTask

Question 3 (ID: 18270900f19204e6)

✅ Evaluation Results
id: 18270900f19204e6
question: You return to your starting position and face north. From a Top-Down map, describe where chair is relative to table. Answer format: <cardinal direction>, <distance> Example: north-west, near
action: None
answer: east, slightly far
task_type: DirectionEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Evaluation state
Q3: DirectionEvaluationTask

📊 Task: PovEvaluationTask

Question 1 (ID: 3976e12c4008e19b)

✅ Evaluation Results
id: 3976e12c4008e19b
question: Now you jump to shoes's direction, facing its direction. Describe where backpack is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Evaluation state
Q1: PovEvaluationTask

Question 2 (ID: b39109ecaa733d7c)

✅ Evaluation Results
id: b39109ecaa733d7c
question: Now you jump to backpack's direction, facing its direction. Describe where shoes is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Evaluation state
Q2: PovEvaluationTask

Question 3 (ID: 1c4c6cc481b955ac)

✅ Evaluation Results
id: 1c4c6cc481b955ac
question: Now you jump to airplane's direction, facing its direction. Describe where televison is relative to you. Answer format: <ego direction>, <distance> Example: front-left, near
action: None
answer: front-left, mid distance
task_type: PovEvaluationTask
choices: []
kwargs:
(none)
score: 0.500
evaluation_mode: use_model_cogmap
Evaluation state
Q3: PovEvaluationTask

📊 Task: BackwardPovTextEvaluationTask

Question 1 (ID: 3c2ff1e4e461e1cf)

✅ Evaluation Results
id: 3c2ff1e4e461e1cf
question: Now you jump to an object's position, facing its direction. You observe that scooter is front, mid distance, facing backward; basket is front-left, mid distance. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: bike
final_pos: [6, 5]
final_ori: [-1, 0]
final_observation: [{'name': 'scooter', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'basket', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': None}]
object_positions:
truck: [8.0, 5.0]
televison: [10.0, 11.0]
bike: [6.0, 5.0]
chair: [10.0, 1.0]
airplane: [12.0, 13.0]
scooter: [2.0, 5.0]
backpack: [12.0, 3.0]
shoes: [13.0, 1.0]
wine: [10.0, 13.0]
basket: [4.0, 3.0]
shoppingcart: [14.0, 9.0]
table: [4.0, 1.0]
green door: [7.0, 2.0]
red door: [13.0, 7.0]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
room_id: 3
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q1: BackwardPovTextEvaluationTask

Question 2 (ID: 697f480eae6ce726)

✅ Evaluation Results
id: 697f480eae6ce726
question: Now you jump to an object's position, facing its direction. You observe that truck is front-left, slightly far, facing right; backpack is front-right, mid distance, facing backward; red door is front-right, slightly far, on front wall. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: chair
final_pos: [10, 1]
final_ori: [0, 1]
final_observation: [{'name': 'truck', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'backpack', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'red door', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
object_positions:
truck: [8.0, 5.0]
televison: [10.0, 11.0]
bike: [6.0, 5.0]
chair: [10.0, 1.0]
airplane: [12.0, 13.0]
scooter: [2.0, 5.0]
backpack: [12.0, 3.0]
shoes: [13.0, 1.0]
wine: [10.0, 13.0]
basket: [4.0, 3.0]
shoppingcart: [14.0, 9.0]
table: [4.0, 1.0]
green door: [7.0, 2.0]
red door: [13.0, 7.0]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
room_id: 1
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q2: BackwardPovTextEvaluationTask

Question 3 (ID: a8c816c15da597f7)

✅ Evaluation Results
id: a8c816c15da597f7
question: Now you jump to an object's position, facing its direction. You observe that bike is front, mid distance, facing backward; basket is front-right, mid distance; green door is front-right, slightly far, on front wall. Which object are you standing at? Answer format: <object_name> Example: lamp
action: None
answer:
answer: scooter
final_pos: [2, 5]
final_ori: [1, 0]
final_observation: [{'name': 'bike', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'basket', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': None}, {'name': 'green door', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'on front wall'}]
object_positions:
truck: [8.0, 5.0]
televison: [10.0, 11.0]
bike: [6.0, 5.0]
chair: [10.0, 1.0]
airplane: [12.0, 13.0]
scooter: [2.0, 5.0]
backpack: [12.0, 3.0]
shoes: [13.0, 1.0]
wine: [10.0, 13.0]
basket: [4.0, 3.0]
shoppingcart: [14.0, 9.0]
table: [4.0, 1.0]
green door: [7.0, 2.0]
red door: [13.0, 7.0]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
room_id: 3
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: BackwardPovTextEvaluationTask
choices: []
kwargs:
(none)
name_match: True
view_match: True
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q3: BackwardPovTextEvaluationTask

📊 Task: Action2ViewEvaluationTask

Question 1 (ID: ead12aceba22dea5)

✅ Evaluation Results
id: ead12aceba22dea5
question: You return to your starting position and face north. You will execute the following action sequence: 1. Jump to the object at front-right, slightly far. 2. Jump to the object at front-slight-left, slightly far. 3. Rotate(-180) After executing the actions, what is the ego relation of shoppingcart relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-left, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q1: Action2ViewEvaluationTask

Question 2 (ID: 329783e3cf710e9c)

✅ Evaluation Results
id: 329783e3cf710e9c
question: You return to your starting position and face north. You will execute the following action sequence: 1. Jump to the object at front-left, mid distance. 2. Rotate(-180) 3. Jump to the object at front-slight-right, mid distance. 4. Rotate(-90) After executing the actions, what is the ego relation of shoes relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-slight-right, slightly far
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q2: Action2ViewEvaluationTask

Question 3 (ID: 9778d336cf4718dc)

✅ Evaluation Results
id: 9778d336cf4718dc
question: You return to your starting position and face north. You will execute the following action sequence: 1. Rotate(-90) 2. Jump to the object at front-left, mid distance. 3. Rotate(90) 4. Jump to the object at front-slight-left, mid distance. 5. Rotate(-180) After executing the actions, what is the ego relation of basket relative to you? Answer format: <ego direction>, <distance> Example: front, near
action: None
answer: front-right, mid distance
task_type: Action2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q3: Action2ViewEvaluationTask

📊 Task: View2ActionTextEvaluationTask

Question 1 (ID: d596a6d6b93f9b1f)

✅ Evaluation Results
id: d596a6d6b93f9b1f
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: bike is at front-right, slightly far, facing left; scooter is at front-left, slightly far, facing right; basket is at front, near What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [4, 1]
final_ori: [0, 1]
room_id: 3
init_pos: [9, 3]
init_ori: [0, 1]
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
minimal_plan: [['rotate', -90], ['jumpto', 'green door'], ['jumpto', 'table'], ['rotate', 90]]
final_observation: [{'name': 'bike', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing left'}, {'name': 'scooter', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing right'}, {'name': 'basket', 'direction': 'front', 'distance': 'near', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [4.0, 1.0]
final_ori: [0, 1]
visible_match: True
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q1: View2ActionTextEvaluationTask

Question 2 (ID: bc2218d7bd5246e5)

✅ Evaluation Results
id: bc2218d7bd5246e5
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: televison is at front-left, mid distance, facing forward; wine is at front, near What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [12, 13]
final_ori: [-1, 0]
room_id: 2
init_pos: [9, 3]
init_ori: [0, 1]
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
minimal_plan: [['jumpto', 'red door'], ['jumpto', 'airplane'], ['rotate', -90]]
final_observation: [{'name': 'televison', 'direction': 'front-left', 'distance': 'mid distance', 'orientation': 'facing forward'}, {'name': 'wine', 'direction': 'front', 'distance': 'near', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: True
ori_match: True
final_pos: [12.0, 13.0]
final_ori: [-1, 0]
visible_match: True
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q2: View2ActionTextEvaluationTask

Question 3 (ID: 2dd6b807f022852a)

✅ Evaluation Results
id: 2dd6b807f022852a
question: You return to your starting position and face north. Then you have executed an action sequence and changed to a new location and facing direction. You observe the following: airplane is at front-left, slightly far, facing left; wine is at front-left, slightly far What action sequence led to this final view? The action sequence must be valid and only contain move actions. Answer format: <sequence of move actions> Example: JumpTo(lamp), Rotate(90)
action: None
answer:
final_pos: [14, 9]
final_ori: [0, 1]
room_id: 2
init_pos: [9, 3]
init_ori: [0, 1]
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
minimal_plan: [['jumpto', 'red door'], ['jumpto', 'shoppingcart']]
final_observation: [{'name': 'airplane', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing left'}, {'name': 'wine', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': None}]
task_type: View2ActionTextEvaluationTask
choices: []
kwargs:
(none)
pos_match: False
ori_match: True
final_pos: [13.0, 7.0]
final_ori: [0, 1]
visible_match: False
score: 0
evaluation_mode: use_model_cogmap
Evaluation state
Q3: View2ActionTextEvaluationTask

📊 Task: AlloMappingEvaluationTask

Question 1 (ID: ba2ae1056e95b4f4)

✅ Evaluation Results
id: ba2ae1056e95b4f4
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: wine, table, airplane. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[1, 10], [-5, -2], [3, 10]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.816
threshold: 0.900
score: 0.816
evaluation_mode: use_model_cogmap
Evaluation state
Q1: AlloMappingEvaluationTask

Question 2 (ID: ee801e392523d89c)

✅ Evaluation Results
id: ee801e392523d89c
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: televison, chair, shoes, truck, wine. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[1, 8], [1, -2], [4, -2], [-1, 2], [1, 10]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.866
threshold: 0.900
score: 0.866
evaluation_mode: use_model_cogmap
Evaluation state
Q2: AlloMappingEvaluationTask

Question 3 (ID: 4f53c659975c6929)

✅ Evaluation Results
id: 4f53c659975c6929
question: Treat your starting position as the origin (0, 0) while facing north. Report allocentric coordinates using (x right/east, y up/north). Objects: truck, shoes, basket. Answer format: (x0, y0); (x1, y1); ... in the same order. Example: (1, 0); (-2, 3); (0, -1)
action: None
answer: [[-1, 2], [4, -2], [-5, 0]]
task_type: AlloMappingEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.854
threshold: 0.900
score: 0.854
evaluation_mode: use_model_cogmap
Evaluation state
Q3: AlloMappingEvaluationTask

📊 Task: RotEvaluationTask

Question 1 (ID: 8bd34cfbaf041b94)

✅ Evaluation Results
id: 8bd34cfbaf041b94
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: shoes, shoppingcart, wine. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['wine', 'shoppingcart', 'shoes']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q1: RotEvaluationTask

Question 2 (ID: a27ba852e9d2d241)

✅ Evaluation Results
id: a27ba852e9d2d241
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: chair, shoes, shoppingcart, table, televison. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['televison', 'shoppingcart', 'shoes', 'chair', 'table']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q2: RotEvaluationTask

Question 3 (ID: f6d3c113c4f20f24)

✅ Evaluation Results
id: f6d3c113c4f20f24
question: You return to your starting position and face north. You will perform a full 360-degree rotation by continuously turning clockwise in place. Assume all walls are removed (you can see through walls), so every object is visible. Focus on this set of objects: backpack, chair, shoppingcart, wine. List them in the exact order they appear directly ahead while you rotate. If two objects share a bearing, place the nearer one first. Answer format: <object_name1>, <object_name2>, ... Example: mug, sofa, plant
action: None
answer: ['wine', 'shoppingcart', 'backpack', 'chair']
task_type: RotEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q3: RotEvaluationTask

📊 Task: Location2ViewEvaluationTask

Question 1 (ID: 59a7cc1d9f9a2684)

✅ Evaluation Results
id: 59a7cc1d9f9a2684
question: Still treat your initial position as origin (0, 0) You move to (2, 2) and face west. What is the egocentric relation of truck? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 0
evaluation_mode: use_model_cogmap
Evaluation state
Q1: Location2ViewEvaluationTask

Question 2 (ID: e28473b0be6f4b18)

✅ Evaluation Results
id: e28473b0be6f4b18
question: Still treat your initial position as origin (0, 0) You move to (0, 2) and face south. What is the egocentric relation of chair? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front-slight-left, slightly far
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q2: Location2ViewEvaluationTask

Question 3 (ID: 6e766b24aeabe349)

✅ Evaluation Results
id: 6e766b24aeabe349
question: Still treat your initial position as origin (0, 0) You move to (0, 0) and face east. What is the egocentric relation of backpack? Answer format: <direction>, <distance> Example: front, near
action: None
answer: front, mid distance
task_type: Location2ViewEvaluationTask
choices: []
kwargs:
(none)
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q3: Location2ViewEvaluationTask

📊 Task: View2LocationTextEvaluationTask

Question 1 (ID: 837814331d62f020)

✅ Evaluation Results
id: 837814331d62f020
question: You move to a new location and face west. You observe: truck is at front, mid distance, facing backward; green door is at front-left, slightly far, on front wall Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [2, 2]
final_pos: [11, 5]
final_ori: [-1, 0]
room_id: 1
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
final_observation: [{'name': 'truck', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing backward'}, {'name': 'green door', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'on front wall'}]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 0.702
raw_score: 0.702
score: 0.702
evaluation_mode: use_model_cogmap
Evaluation state
Q1: View2LocationTextEvaluationTask

Question 2 (ID: f60f38aad8e949fb)

✅ Evaluation Results
id: f60f38aad8e949fb
question: You move to a new location and face south. You observe: chair is at front-slight-left, slightly far, facing backward; shoes is at front-left, slightly far, facing backward; green door is at front-right, mid distance, on right wall Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [0, 2]
final_pos: [9, 5]
final_ori: [0, -1]
room_id: 1
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
final_observation: [{'name': 'chair', 'direction': 'front-slight-left', 'distance': 'slightly far', 'orientation': 'facing backward'}, {'name': 'shoes', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'facing backward'}, {'name': 'green door', 'direction': 'front-right', 'distance': 'mid distance', 'orientation': 'on right wall'}]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q2: View2LocationTextEvaluationTask

Question 3 (ID: 9bffb5402a1f2564)

✅ Evaluation Results
id: 9bffb5402a1f2564
question: You move to a new location and face east. You observe: backpack is at front, mid distance, facing right; shoes is at front-right, slightly far, facing left; red door is at front-left, slightly far, on left wall Still treat your initial position as origin (0, 0) What is your new 2D coordinate (x, y)? Answer format: (x, y) Example: (2, -1)
action: None
answer:
coord: [0, 0]
final_pos: [9, 3]
final_ori: [1, 0]
room_id: 1
object_positions:
truck: [8, 5]
televison: [10, 11]
bike: [6, 5]
chair: [10, 1]
airplane: [12, 13]
scooter: [2, 5]
backpack: [12, 3]
shoes: [13, 1]
wine: [10, 13]
basket: [4, 3]
shoppingcart: [14, 9]
table: [4, 1]
green door: [7, 2]
red door: [13, 7]
object_orientations:
truck: [1, 0]
televison: [-1, 0]
bike: [-1, 0]
chair: [0, 1]
airplane: [-1, 0]
scooter: [1, 0]
backpack: [0, -1]
shoes: [0, 1]
shoppingcart: [0, -1]
green door: [1, 0]
red door: [0, 1]
final_observation: [{'name': 'backpack', 'direction': 'front', 'distance': 'mid distance', 'orientation': 'facing right'}, {'name': 'shoes', 'direction': 'front-right', 'distance': 'slightly far', 'orientation': 'facing left'}, {'name': 'red door', 'direction': 'front-left', 'distance': 'slightly far', 'orientation': 'on left wall'}]
gate_info:
green door:
room_ids: [3, 1]
ori_by_room:
3: [-1, 0]
1: [1, 0]
red door:
room_ids: [1, 2]
ori_by_room:
1: [0, -1]
2: [0, 1]
object_rooms:
truck: 1
televison: 2
bike: 3
chair: 1
airplane: 2
scooter: 3
backpack: 1
shoes: 1
wine: 2
basket: 3
shoppingcart: 2
table: 3
green door: [3, 1]
red door: [1, 2]
task_type: View2LocationTextEvaluationTask
choices: []
kwargs:
(none)
similarity: 1
raw_score: 1
score: 1
evaluation_mode: use_model_cogmap
Evaluation state
Q3: View2LocationTextEvaluationTask