The uncertainty and variability of underwater environment propose the request to control underwater robots in real time and dynamically, especially in the scenarios where human and robots need to work collaboratively in the field. However, the underwater environment imposes harsh restrictions on the application of typical control and communication methods. Considering that gestures are a natural and efficient interactive way for human, we, utilizing convolution neural network, implement a real-time gesture-based recognition system, who can recognize 50 kinds of gestures from images captured by one normal monocular camera, and apply this recognition system in human and underwater robot interaction. We design A Flexible and Extendable Interaction Scheme (AFEIS) through which underwater robots can be programmed in situ underwater by human operators using customized gesture-based sign language. This paper elaborates the design of gesture recognition system and AFEIS, and presents our field trial results when applying this system and scheme on underwater robots.
Besides, in @cite_10 , the authors define a set of syntax in the form similar to natural language as the way to translate gestures to commands accepted by robots. An effort to interact with robots using natural sentences expressed by gestures are made in the paper. However, in practice, we find that the syntax similar to programming language is more intuitive and acceptable by operators. Therefore, in AFEIS, we define a set of syntax similar to programming language but with simpler form. Besides commands the robot to make action through hand gestures, the syntax of AFEIS also supports function definition and variable setting, which are convenient for doing repetitive tasks.
@cite_10
1518898410
Underwater environment is characterized by harsh conditions and is difficult to monitor. The CADDY project deals with the development of a companion robot devoted to support and to monitor human operations and activities during the dive. In this scenario the communication and correct reception of messages between the diver and the robot are essential for success of the dive goals. However, the underwater environment poses a set of technical constraints hardly limiting the communication possibilities. For such reasons the solution proposed is to develop a communication language based on the consolidated and standardized diver gestures, commonly employed during professional and recreational dives, thus leading to the definition of a CADDY language, called CADDIAN, and a communication protocol. This article focuses on the creation of the language providing alphabet, syntax and semantics: future work will explain the part of recognition of gestures that is still in progress.