In this paper, we address the problem of controlling a mobile stereo camera under image quantization noise. Assuming that a pair of images of a set of targets is available, the camera moves through a sequence of Next-Best-Views (NBVs), i.e., a sequence of views that minimize the trace of the targets' cumulative state covariance, constructed using a realistic model of the stereo rig that captures image quantization noise and a Kalman Filter (KF) that fuses the observation history with new information. The proposed algorithm decomposes control into two stages: first the NBV is computed in the camera relative coordinates, and then the camera moves to realize this view in the fixed global coordinate frame. This decomposition allows the camera to drive to a new pose that effectively realizes the NBV in camera coordinates while satisfying Field-of-View constraints in global coordinates, a task that is particularly challenging using complex sensing models. We provide simulations and real experiments that illustrate the ability of the proposed mobile camera system to accurately localize sets of targets. We also propose a novel data-driven technique to characterize unmodeled uncertainty, such as calibration errors, at the pixel level and show that this method ensures stability of the KF.
In this paper, we consider the problem of precisely localizing a group of stationary targets using a single stereo camera mounted on a mobile robot. In particular, assuming that at least one pair of stereo images of the targets is available, we seek to determine where to move the stereo camera so that the localization uncertainty of the targets is minimized. We call this problem the Next-Best-View problem. The advantage of using a stereo camera is that, using triangulation, the two simultaneous images can yield range and bearing measurements of the targets, as well as their uncertainty. We use a Kalman filter to fuse location and uncertainty estimates as more measurements are acquired. Our solution to the Next-Best-View problem is to iteratively minimize the fused uncertainty of the targets' locations subject to field-of-view constraints. We capture these objectives by appropriate artificial potentials on the camera's relative frame and the global frame, respectively. In particular, with every new observation, the mobile stereo camera computes the new next best view on the relative frame and subsequently realizes this view in the global frame via gradient descent on the space of robot positions and orientations, until a new observation is made. Integration of next best view with motion planning results in a hybrid system, which we illustrate in computer simulations. In this paper, we control image collection for a mobile stereo camera that is actively localizing a group of mobile targets. In particular, assuming that at least one pair of stereo images of the targets is available, we propose a novel approach to control the rotation and translation of the stereo camera so that the next observation of the targets will minimize their localization uncertainty. We call this problem the Next-Best-View problem for mobile targets (mNBV). The advantage of using a stereo camera is that, using triangulation, the two simultaneous images taken by the robot during a single observation can yield range and bearing measurements of the targets, as well as their uncertainty. A Kalman filter fuses the full state history and covariance estimates, as more measurements are acquired. Our solution to the mNBV problem determines the relative transformations between camera and targets that will minimize the fused uncertainty of the targets' locations. We determine a motion plan that realizes the mNBV while respecting field of view constraints. In particular, with every new observation, we compute a new mNBV in the frame relative to the camera and subsequently realize this view in global coordinates via a gradient descent algorithm that also respects field of view constraints. Integration of mNBV with motion planning results in a hybrid system, which we illustrate in computer simulations.