Automating the navigation of unmanned aerial vehicles (UAVs) in diverse scenarios has gained much attention in recent years. However, teaching UAVs to fly in challenging environments remains an unsolved problem, mainly due to the lack of training data. In this paper, we train a deep neural network to predict UAV controls from raw image data for the task of autonomous UAV racing in a photo-realistic simulation. Training is done through imitation learning with data augmentation to allow for the correction of navigation mistakes. Extensive experiments demonstrate that our trained network (when sufficient data augmentation is used) outperforms state-of-the-art methods and flies more consistently than many human pilots. Additionally, we show that our optimized network architecture can run in real-time on embedded hardware, allowing for efficient on-board processing critical for real-world deployment. From a broader perspective, our results underline the importance of extensive data augmentation techniques to improve robustness in end-to-end learning setups.
@cite_0 trained a network on autonomous car datasets and then deployed it to control a drone. For this, they used full supervision by providing image and measured steering angle pairs from pre-collected datasets, and collecting their own dataset containing image and binary obstacle indication pairs. While they demonstrate an ability to transfer successfully to other environments, their approach does not model and exploit the full six degrees of freedom available. It also focuses on slow and safe navigation, rather than optimizing for speed as is the case for racing. Finally, with their network being fairly complex, they report an inference speed of 20fps (CPU) for remote processing, which is more than three times lower than the estimated frame rate for our proposed method when running on-board processing, and more than 27 times lower compared to our method running remotely on GPU.
@cite_0
2293598046
We describe a vision-based obstacle avoidance system for off-road mobile robots. The system is trained from end to end to map raw input images to steering angles. It is trained in supervised mode to predict the steering angles provided by a human driver during training runs collected in a wide variety of terrains, weather conditions, lighting conditions, and obstacle types. The robot is a 50cm off-road truck, with two forward-pointing wireless color cameras. A remote computer processes the video and controls the robot via radio. The learning system is a large 6-layer convolutional network whose input is a single left right pair of unprocessed low-resolution images. The robot exhibits an excellent ability to detect obstacles and navigate around them in real time at speeds of 2 m s.