This paper presents a data-driven approach to learning vision-based collective behavior from a simple flocking algorithm. We simulate a swarm of quadrotor drones and formulate the controller as a regression problem in which we generate 3D velocity commands directly from raw camera images. The dataset is created by simultaneously acquiring omnidirectional images and computing the corresponding control command from the flocking algorithm. We show that a convolutional neural network trained on the visual inputs of the drone can learn not only robust collision avoidance but also coherence of the flock in a sample-efficient manner. The neural controller effectively learns to localize other agents in the visual input, which we show by visualizing the regions with the most influence on the motion of an agent. This weakly supervised saliency map can be computed efficiently and may be used as a prior for subsequent detection and relative localization of other agents. We remove the dependence on sharing positions among flock members by taking only local visual information into account for control. Our work can therefore be seen as the first step towards a fully decentralized, vision-based flock without the need for communication or visual markers to aid detection of other agents.
We present a fast and precise vision-based software intended for multiple robot localization. The core component of the software is a novel and efficient algorithm for black and white pattern detection. The method is robust to variable lighting conditions, achieves sub-pixel precision and its computational complexity is independent of the processed image size. With off-the-shelf computational equipment and low-cost cameras, the core algorithm is able to process hundreds of images per second while tracking hundreds of objects with millimeter precision. In addition, we present the method's mathematical model, which allows to estimate the expected localization precision, area of coverage, and processing speed from the camera's intrinsic parameters and hardware's processing capacity. The correctness of the presented model and performance of the algorithm in real-world conditions is verified in several experiments. Apart from the method description, we also make its source code public at http: purl.org robotics whycon ; so, it can be used as an enabling technology for various mobile robotic problems. In this paper, we present a small, light-weight, low-cost, fast and reliable system designed to satisfy requirements of relative localization within a swarm of micro aerial vehicles. The core of the proposed solution is based on off-the-shelf components consisting of the Caspa camera module and Gumstix Overo board accompanied by a developed efficient image processing method for detecting black and white circular patterns. Although the idea of the roundel recognition is simple, the developed system exhibits reliable and fast estimation of the relative position of the pattern up to 30 fps using the full resolution of the Caspa camera. Thus, the system is suited to meet requirements for a vision based stabilization of the robotic swarm. The intent of this paper is to present the developed system as an enabling technology for various robotic tasks. 