This work explores the feasibility of steering a drone with a (recurrent) neural network, based on input from a forward looking camera, in the context of a high-level navigation task. We set up a generic framework for training a network to perform navigation tasks based on imitation learning. It can be applied to both aerial and land vehicles. As a proof of concept we apply it to a UAV (Unmanned Aerial Vehicle) in a simulated environment, learning to cross a room containing a number of obstacles. So far only feedforward neural networks (FNNs) have been used to train UAV control. To cope with more complex tasks, we propose the use of recurrent neural networks (RNN) instead and successfully train an LSTM (Long-Short Term Memory) network for controlling UAVs. Vision based control is a sequential prediction problem, known for its highly correlated input data. The correlation makes training a network hard, especially an RNN. To overcome this issue, we investigate an alternative sampling method during training, namely window-wise truncated backpropagation through time (WW-TBPTT). Further, end-to-end training requires a lot of data which often is not available. Therefore, we compare the performance of retraining only the Fully Connected (FC) and LSTM control layers with networks which are trained end-to-end. Performing the relatively simple task of crossing a room already reveals important guidelines and good practices for training neural control networks. Different visualizations help to explain the behavior learned.
Training a control network with solely RGB images as input has already been demonstrated in 1990, by @cite_8 . In that work, an FNN was trained online from a set of shifted and rotated images. This important work showed that networks are capable of performing a restrictive task like following a road. Also, it showed the need for recovery data in the training set. The network contained only 5 hidden units and 30 discrete output units. The computational power of today allows us to work with more complex networks and continuous control.
@cite_8
2763968347
We evaluated how deep convolutional neural networks (DCNN) could assist in the labor-intensive process of human visual searches for objects of interest in high-resolution imagery over large areas of the Earth’s surface. Various DCNN were trained and tested using fewer than 100 positive training examples (China only) from a worldwide surface-to-air-missile (SAM) site dataset. A ResNet-101 DCNN achieved a 98.2 average accuracy for the China SAM site data. The ResNet-101 DCNN was used to process ∼19.6 M image chips over a large study area in southeastern China. DCNN chip detections (∼9300) were postprocessed with a spatial clustering algorithm to produce a ranked list of ∼2100 candidate SAM site locations. The combination of DCNN processing and spatial clustering effectively reduced the search area by ∼660X (0.15 of the DCNN-processed land area). An efficient web interface was used to facilitate a rapid serial human review of the candidate SAM sites in the China study area. Four novice imagery analysts with no prior imagery analysis experience were able to complete a DCNN-assisted SAM site search in an average time of ∼42 min. This search was ∼81X faster than a traditional visual search over an equivalent land area of ∼88,640 km2 while achieving nearly identical statistical accuracy (∼90 F1).