Nonlinear control systems - A brief overview of historical and recent advances <s> Challenges <s> Covers in a progressive fashion a number of analysis tools and design techniques directly applicable to nonlinear control problems in high performance systems (in aerospace, robotics and automotive areas). <s> BIB001 </s> Nonlinear control systems - A brief overview of historical and recent advances <s> Challenges <s> Safety is very important for physical human-robot interaction. Compliance control can solve an important aspect of the safety problem by dealing with impact and other forces arising during close contact between humans and robots. <s> BIB002 </s> Nonlinear control systems - A brief overview of historical and recent advances <s> Challenges <s> Abstract This paper provides an overview of the reinforcement learning and optimal adaptive control literature and its application to robotics. Reinforcement learning is bridging the gap between traditional optimal control, adaptive control and bio-inspired learning techniques borrowed from animals. This work is highlighting some of the key techniques presented by well known researchers from the combined areas of reinforcement learning and optimal control theory. At the end, an example of an implementation of a novel model-free Q-learning based discrete optimal adaptive controller for a humanoid robot arm is presented. The controller uses a novel adaptive dynamic programming (ADP) reinforcement learning (RL) approach to develop an optimal policy on-line. The RL joint space tracking controller was implemented for two links (shoulder flexion and elbow flexion joints) of the arm of the humanoid Bristol-Elumotion-Robotic-Torso II (BERT II) torso. The constrained case (joint limits) of the RL scheme was tested for a single link (elbow flexion) of the BERT II arm by modifying the cost function to deal with the extra nonlinearity due to the joint constraints. <s> BIB003
Nature is nonlinear and the best way will be to deal with it with nonlinear approaches. Although, linear control has been successfully employed for decades. The problem with linear system is that it may not be able to cater for the modern and advanced technology such as multi-degree of freedom robots (with under actuation or constraints), UAVS, submarines and missile control in which more complex system operations are involved and greater application ranges are required. The decision to choose linear or nonlinear control for a particular application may be a difficult one. Linear control is well tested and the people in industry have confidence in it. There are plenty of excellent analysis tools available for linear systems such as Bode plot, root locus, Nyquist stability criteria, Laplace transform, Z-transform and Fourier transform etc. In contrast, nonlinear system needs complex mathematical analysis e.g. Lypunov stability criterion, Popov criterion and singular perturbation methods. Mathematical modelling may also be cumbersome for nonlinear systems. Nonlinear system may suffer from limit cycle, chaos and bifurcation. Most of the schemes can only ensure local stability while global stabilities cannot be guaranteed BIB001 . In addition, computational power may be one of the main limitations to implement advanced nonlinear control approaches due to high initial investment involved. The cost of nonlinear control prototyping hardware is still very high. To commercialize these advanced techniques, no cheap, portable and capable hardware is available yet. More efficient programming may be required to reduce computational overhead. For instance, in our previous work BIB003 , reinforcement learning based optimal adaptive control for humanoid robots was implemented for a single DOF on dSPACE 1006 employing a sampling time of 1 msec. To extend it to two DOF, MATLAB's embedded functions had to be replaced with more efficient C/C++ S-functions. Although actuator saturations usually incur in linear control too, it may, however, be more problematic and catastrophic in case of nonlinear control since the control signal here is very high and aggressive. Hence, it easily leads to windup of the system. In the work by Khan et al. BIB002 , anti-windup compensator was incorporated to deal with actuator saturation issues in the model reference adaptive compliance control of a humanoid robotic arm. However, anti-windup compensators usually degrade the performance of the nominal control scheme. Nonlinear control schemes may need more expensive actuators to survive longer. Most of the nonlinear controllers pose greater risk to damage expensive systems if not implemented with care. Also, complex and exhaustive tuning process may be involved. Due to the above mentioned reasons, people in industry are still quite wary of the nonlinear control and there is a big gap between nonlinear control researchers and industrial control engineers. Industries still rely mainly on simple traditional linear schemes. More work is needed to reduce complexity of these advanced nonlinear strategies. Significant efforts are needed to convince the control practitioners to adapt these modern nonlinear techniques.