Once the agent has explored the environment and detects the target object, the agent is asked to approach the object and stop as close as possible in front of it; this pose is named destination pose.
We first process the depth channel of the last observation to estimate the position of the detected target in the environment. The depth crop of the object detection is converted to 3D points with the camera pose, where the \(\lbrace x, y\rbrace \)  position of the closest point to the camera is used to approximate the target position.
Then we generate the destination pose by selecting the subset of admissible poses that can see the target position, according to the observation model, and finding the closest to the target position.
We use the Dijkstra algorithm [1]} to compute the shortest path between the current pose (reached using POMCP) and the estimated destination pose.
In order to be robust against the detector imperfections, we further introduce a path replanning scheme after new observations. At every time step in the approaching phase the agent observes the scene. In case the target object is detected, we recompute the object location in the environment, the destination pose and re-plan the optimal path to reach it. If instead the object is not detected, the agent continues with the planned path.
The effect of the path replanning against following the originally planned path can be seen in Table REF , where we have an average improvement of 10% in success rate.
