The use of robotics in controlled environments has flourished over the last several decades and training robots to perform tasks using control strategies developed from dynamical models of their hardware have proven very effective. However, in many real-world settings, the uncertainties of the environment, the safety requirements and generalized capabilities that are expected of robots make rigid industrial robots unsuitable. This created great research interest into developing control strategies for flexible robot hardware for which building dynamical models are challenging. In this paper, inspired by the success of deep reinforcement learning (DRL) in other areas, we systematically study the efficacy of policy search methods using DRL in training flexible robots. Our results indicate that DRL is successfully able to learn efficient and robust policies for complex tasks at various degrees of flexibility. We also note that DRL using Deep Deterministic Policy Gradients can be sensitive to the choice of sensors and adding more informative sensors does not necessarily make the task easier to learn.
In work where flexibility is taken into consideration, learning is still based either on building a more complex model @cite_14 @cite_7 @cite_5 , an approximate model @cite_4 or plugging in a learned-model component into a model-based controller. Recently, work involving end-to-end model-free methods using deep reinforcement learning have been demonstrated successfully in rigid real robots @cite_26 @cite_13 @cite_1 . @cite_21 have shown that learning directly in hardware is possible with policy search, they rightly point out that even in simple tasks, factors such as joint slackness etc. make it very difficult to train. While they and @cite_10 have shown that this is possible using policy search, this paper systematically studies how policy search methods perform with flexible hardware.
@cite_14 @cite_4 @cite_26 @cite_7 @cite_10 @cite_21 @cite_1 @cite_5 @cite_13
2090108740 1968559739 2964161785 2173248099 2888235455
Over the last few decades, extensive use of flexible manipulators in various robotic applications has made it as one of the research interests for many scholars over the world. Recent studies on the modeling, sensor systems and controllers for the applications of flexible robotic manipulators are reviewed in order to complement the previous literature surveyed by Benosman & Vey (Robotica 22:533---545, 2004) and Dwivedy & Eberhard (Mech. Mach. Theory 41:749---777, 2006) . A brief introduction of the essential modeling techniques is first presented, followed by a review of the practical alternatives of sensor systems that can help scientists or engineers to choose the appropriate sensors for their applications. It followed by the main goal of this paper with a comprehensive review of the control strategies for the flexible manipulators and flexible joints that were studied in recent literatures. The issues for controlling flexible manipulators are highlighted. Most of the noteworthy control techniques that were not covered in the recent surveys in references (Benosman & Vey Robotica 22:533---545, 2004; Dwivedy & Eberhard Mech. Mach. Theory 41:749---777, 2006) are then reviewed. It concludes by providing some possible issues for future research works. Model based control schemes use the inverse dynamics of the robot arm to produce the main torque component necessary for trajectory tracking. For model-based controller one is required to know the model parameters accurately. This is a very difficult task especially if the manipulator is flexible. So a reduced model based controller has been developed, which requires only the information of space robot base velocity and link parameters. The flexible link is modeled as Euler Bernoulli beam. To simplify the analysis we have considered Jacobian of rigid manipulator. Bond graph modeling is used to model the dynamics of the system and to devise the control strategy. The scheme has been verified using simulation for two links flexible space manipulator. Policy search methods can allow robots to learn control policies for a wide range of tasks, but practical applications of policy search often require hand-engineered components for perception, state estimation, and low-level control. In this paper, we aim to answer the following question: does training the perception and control systems jointly end-to-end provide better performance than training each component separately? To this end, we develop a method that can be used to learn policies that map raw image observations directly to torques at the robot's motors. The policies are represented by deep convolutional neural networks (CNNs) with 92,000 parameters, and are trained using a guided policy search method, which transforms policy search into supervised learning, with supervision provided by a simple trajectory-centric reinforcement learning method. We evaluate our method on a range of real-world manipulation tasks that require close coordination between vision and control, such as screwing a cap onto a bottle, and present simulated comparisons to a range of prior policy search methods. We adapt the ideas underlying the success of Deep Q-Learning to the continuous action domain. We present an actor-critic, model-free algorithm based on the deterministic policy gradient that can operate over continuous action spaces. Using the same learning algorithm, network architecture and hyper-parameters, our algorithm robustly solves more than 20 simulated physics tasks, including classic problems such as cartpole swing-up, dexterous manipulation, legged locomotion and car driving. Our algorithm is able to find policies whose performance is competitive with those found by a planning algorithm with full access to the dynamics of the domain and its derivatives. We further demonstrate that for many of the tasks the algorithm can learn policies end-to-end: directly from raw pixel inputs. In this paper, we present an automated learning environment for developing control policies directly on the hardware of a modular legged robot. This environment facilitates the reinforcement learning process by computing the rewards using a vision-based tracking system and relocating the robot to the initial position using a resetting mechanism. We employ two state-of-the-art deep reinforcement learning (DRL) algorithms, Trust Region Policy Optimization (TRPO) and Deep Deterministic Policy Gradient (DDPG), to train neural network policies for simple rowing and crawling motions. Using the developed environment, we demonstrate both learning algorithms can effectively learn policies for simple locomotion skills on highly stochastic hardware and environments. We further expedite learning by transferring policies learned on a single legged configuration to multi-legged ones.