This paper addresses the problem of building a speech recognition system attuned to the control of unmanned aerial vehicles (UAVs). Even though UAVs are becoming widespread, the task of creating voice interfaces for them is largely unaddressed. To this end, we introduce a multi-modal evaluation dataset for UAV control, consisting of spoken commands and associated images, which represent the visual context of what the UAV "sees" when the pilot utters the command. We provide baseline results and address two research directions: (i) how robust the language models are, given an incomplete list of commands at train time; (ii) how to incorporate visual information in the language model. We find that recurrent neural networks (RNNs) are a solution to both tasks: they can be successfully adapted using a small number of commands and they can be extended to use visual cues. Our results show that the image-based RNN outperforms its text-only counterpart even if the command-image training associations are automatically generated and inherently imperfect. The dataset and our code are available at this http URL.
 This project presents a speech-based control system for DRONE using Support Vector Machines (SVM). The set of controlling speeches consists of BACKWARD, FORWARD, HOLD ON, LANDING, MOVE UP, MOVE DOWN, TAKE OFF, TURN LEFT and TURN RIGHT are trained the SVM. The feature extraction of speech used in this study comprises of “fundamental frequency”, “Energy”, and Mel Frequency Cepstral Coefficient”. For performance evaluation, a set of features are used to test the SVM-based system developed by MATLAB. The results show that the average percentage of accuracy of the controlling speeches are 22.22, 46.67, 97.78 and 95.56 for fundamental frequency, energy, Mel frequency cepstral coefficient and all features, respectively. In addition, the interface of SVM-based system and DRONE is developed in practical use. Unmanned aerial vehicle (UAV) control stations feature multiple menu pages with systems accessed by keyboard presses. Use of speech-based input may enable operators to navigate through menus and select options more quickly. This experiment examined the utility of conventional manual input versus speech input for tasks performed by operators of a UAV control station simulator at two levels of mission difficulty. Pilots performed a continuous flight navigation control task while completing eight different data entry task types with each input modality. Results showed that speech input was significantly better than manual input in terms of task completion time, task accuracy, flight navigation measures, and pilot ratings. Across tasks, data entry time was reduced by approximately 40 with speech input. Additional research is warranted to confirm that this head-up, hands-free control is still beneficial in operational UAV control station auditory environments and does not conflict with intercom operations and...