Graphical abstractDisplay Omitted HighlightsAn algorithm which is able to learn controllers with embedded preprocessing for mobile robotics is presented.Quantified Fuzzy Propositions, a model able to summarize the low-level input data, are used.The algorithm was tested with the wall-following behavior both in simulated and real environments.Results show a better and statistically significant performance of our proposal.The approach was also successfully tested in three real world behaviors. The automatic design of controllers for mobile robots usually requires two stages. In the first stage, sensorial data are preprocessed or transformed into high level and meaningful values of variables which are usually defined from expert knowledge. In the second stage, a machine learning technique is applied to obtain a controller that maps these high level variables to the control commands that are actually sent to the robot. This paper describes an algorithm that is able to embed the preprocessing stage into the learning stage in order to get controllers directly starting from sensorial raw data with no expert knowledge involved. Due to the high dimensionality of the sensorial data, this approach uses Quantified Fuzzy Rules (QFRs), that are able to transform low-level input variables into high-level input variables, reducing the dimensionality through summarization. The proposed learning algorithm, called Iterative Quantified Fuzzy Rule Learning (IQFRL), is based on genetic programming. IQFRL is able to learn rules with different structures, and can manage linguistic variables with multiple granularities. The algorithm has been tested with the implementation of the wall-following behavior both in several realistic simulated environments with different complexity and on a Pioneer 3-AT robot in two real environments. Results have been compared with several well-known learning algorithms combined with different data preprocessing techniques, showing that IQFRL exhibits a better and statistically significant performance. Moreover, three real world applications for which IQFRL plays a central role are also presented: path and object tracking with static and moving obstacles avoidance.
The learning of controllers for autonomous robots has been dealt with by using different machine learning techniques. Among the most popular approaches can be found evolutionary algorithms @cite_10 @cite_38 , neural networks @cite_34 and reinforcement learning @cite_30 @cite_41 . Also hibridations of them, like evolutionary neural networks @cite_32 , reinforcement learning with evolutionary algorithms @cite_17 @cite_13 , the widely used genetic fuzzy systems @cite_9 @cite_5 @cite_23 @cite_12 @cite_29 @cite_31 @cite_22 , or even more uncommon combinations like ant colony optimization with reinforcement learning @cite_24 or differential evolution @cite_4 or evolutionary group based particle swarm optimization @cite_0 have been successfully applied. Furthermore, over the last few years, mobile robotic controllers have been getting some attention as a test case for the automatic design of type-2 fuzzy logic controllers @cite_35 @cite_38 @cite_18 .
@cite_30 @cite_35 @cite_22 @cite_41 @cite_29 @cite_5 @cite_10 @cite_38 @cite_18 @cite_4 @cite_23 @cite_17 @cite_32 @cite_34 @cite_12 @cite_9 @cite_24 @cite_0 @cite_31 @cite_13
1561996023 2233677881 1963565134 2124958386 1989329927 1540106380 2028249667 2168640251 2072881998 2079690958 1995515218 2132586633 2164301484 2111616957 2111065664 2024327854 2035397042
The successful application of Reinforcement Learning (RL) techniques to robot control is limited by the fact that, in most robotic tasks, the state and action spaces are continuous, multidimensional, and in essence, too large for conventional RL algorithms to work. The well known curse of dimensionality makes infeasible using a tabular representation of the value function, which is the classical approach that provides convergence guarantees. When a function approximation technique is used to generalize among similar states, the convergence of the algorithm is compromised, since updates unavoidably affect an extended region of the domain, that is, some situations are modified in a way that has not been really experienced, and the update may degrade the approximation. We propose a RL algorithm that uses a probability density estimation in the joint space of states, actions and Q-values as a means of function approximation. This allows us to devise an updating approach that, taking into account the local sampling density, avoids an excessive modification of the approximation far from the observed sample. In recent years, the autonomous mobile robot has found diverse applications such as home health care system, surveillance system in civil and military applications and exhibition robot. For surveillance tasks such as moving target pursuit or following and patrol in a region using mobile robot, this paper presents a fuzzy Q-learning, as an intelligent control for cost-based navigation, for autonomous learning of suitable behaviors without the supervision or external human command. The Q-learning is used to select the appropriate rule of interval type-2 fuzzy rule base. The initial testing of the intelligent control is demonstrated by simulation as well as experiment of a simple wall-following based patrolling task of autonomous mobile robot. The design of fuzzy controllers for the implementation of behaviors in mobile robotics is a complex and highly time-consuming task. The use of machine learning techniques such as evolutionary algorithms or artificial neural networks for the learning of these controllers allows to automate the design process. In this paper, the automated design of a fuzzy controller using genetic algorithms for the implementation of the wall-following behavior in a mobile robot is described. The algorithm is based on the iterative rule learning approach, and is characterized by three main points. First, learning has no restrictions neither in the number of membership functions, nor in their values. In the second place, the training set is composed of a set of examples uniformly distributed along the universe of discourse of the variables. This warrantees that the quality of the learned behavior does not depend on the environment, and also that the robot will be capable to face different situations. Finally, the trade off between the number of rules and the quality accuracy of the controller can be adjusted selecting the value of a parameter. Once the knowledge base has been learned, a process for its reduction and tuning is applied, increasing the cooperation between rules and reducing its number. The problem of an effective behavior learning of autonomous robots is one of the most important tasks of the modern robotics. In fact, it is well known that the learning to optimize actions of autonomous agents in a dynamic environment is one of the most complex challenges of the intelligent system design. In this paper, we propose a hybrid approach integrating fuzzy logic system with genetic algorithm for high-level skills learning of robots within the RoboCup simulation soccer domain. Through the experiments, we found that the proposed method has good property of computation efficiency and also has a good advantage applied to the environment of RoboCup. The design of fuzzy controllers for the implementation of behaviors in mobile robotics is a complex and highly time-consuming task. The use of machine learning techniques, such as evolutionary algorithms or artificial neural networks for the learning of these controllers allows to automate the design process. In this paper, the automated design of a fuzzy controller using genetic algorithms for the implementation of the wall-following behavior in a mobile robot is described. The algorithm is based on the Iterative Rule Learning (IRL) approach, and a parameter (@d) is defined with the aim of selecting the relation between the number of rules and the quality and accuracy of the controller. The designer has to define the universe of discourse and the precision of each variable, and also the scoring function. No restrictions are placed neither in the number of linguistic labels nor in the values that define the membership functions. Genetic programming provides an automated design strategy to evolve complex controllers based on evolution in nature. In this contribution we use genetic programming to automatically evolve efficient robot controllers for a corridor following task. Based on tests executed in a simulation environment we show that very robust and efficient controllers can be obtained. Also, we stress that it is important to provide sufficiently diverse fitness cases, offering a sound basis for learning more complex behaviour. The evolved controller is successfully applied to real environments as well. Finally, controller and sensor morphology are co-evolved, clearly resulting in an improved sensor configuration. This paper proposes evolutionary wall-following control of a mobile robot using an interval type-2 fuzzy controller (IT2FC) with species-differential-evolution-activated continuous ant colony optimization (SDE-CACO). Both the position and speed of a mobile robot are controlled by using two IT2FCs to improve noise resistance ability. A new cost function is defined to accurately evaluate the wall-following performance of an evolutionary IT2FC. A two-stage training approach is proposed that learns a position IT2FC followed by a speed IT2FC to optimize both the wall-following accuracy and the moving speed. The proposed learning approach avoids the time consuming task of the exhaustive collection of supervised input-output training pairs. All fuzzy rules are generated online using a clustering-based approach during the evolutionary learning process. All of the free parameters in an online-generated IT2FC are optimized using SDE-CACO, in which an SDE mutation operation is incorporated within a continuous ACO to improve its explorative ability. The proposed SDE-CACO is compared with various population-based optimization algorithms to demonstrate its efficiency and effectiveness in the wall-following control problem. This study also includes experiments that demonstrate wall-following control utilizing a real mobile robot. A methodology for learning behaviors in mobile robotics has been developed. It consists of a technique to automatically generate input–output data plus a genetic fuzzy system that obtains cooperative weighted rules. The advantages of our methodology over other approaches are that the designer has to choose the values of only a few parameters, the obtained controllers are general (the quality of the controller does not depend on the environment), and the learning process takes place in simulation, but the controllers work also on the real robot with good performance. The methodology has been used to learn the wall-following behavior, and the obtained controller has been tested using a Nomad 200 robot in both simulated and real environments. © 2009 Wiley Periodicals, Inc. Conventional fuzzy logic controller is applicable when there are only two fuzzy inputs with usually one output. Complexity increases when there are more than one inputs and outputs making the system unrealizable. The ordinal structure model of fuzzy reasoning has an advantage of managing high-dimensional problem with multiple input and output variables ensuring the interpretability of the rule set. This is achieved by giving an associated weight to each rule in the defuzzification process. In this work, a methodology to design an ordinal fuzzy logic controller with application for obstacle avoidance of Khepera mobile robot is presented. The implementation will show that ordinal structure fuzzy is easier to design with highly interpretable rules compared to conventional fuzzy controller. In order to achieve high accuracy, a specially tailored Genetic Algorithm (GA) approach for reinforcement learning has been proposed to optimize the ordinal structure fuzzy controller. Simulation results demonstrated improved obstacle avoidance performance in comparison with conventional fuzzy controllers. Comparison of direct and incremental GA for optimization of the controller is also presented. Evolutionary Robotics (ER) is one of promising approaches to design robot controllers which essentially have complicated and or complex properties. In most ER research, the sensory-motor mappings of robots are represented as artificial neural networks, and their connection weights (and sometimes the structure of the networks) can be optimized in the parameter spaces by using evolutionary computation. However, generally, the evolved neural controllers could be fragile in unexperienced environments, especially in real worlds, because the evolutionary optimization processes would be executed in idealized simulators. This is known as the gap problem between the simulated and real worlds. To overcome this, the author focused on evolving an on-line learning ability instead of weight parameters in a simulated environment. According to recent biological findings, actually, the kinds of on-line adaptation abilities can be found in real nervous systems of insects and crustaceans, and it is also known that a variety of neuromodulators (NMs) play crucial roles to regulate the network characteristics (i.e. activating blocking changing of synaptic connections). Based on this, a neuromodulatory neural network model was proposed and it was utilized as a mobile robot controller. In the paper, the detail behavior analysis of the evolved neuromodulatory neural network is also discussed. Robots have played an important role in the automation of computer aided manufacturing. The classical robot control implementation involves an expensive key step of model-based programming. An intuitive way to reduce this expensive exercise is to replace programming with machine learning of robot actions from demonstration where a (learner) robot learns an action by observing a demonstrator robot performing the same. To achieve this learning from demonstration (LFD) different machine learning techniques such as Artificial Neural Networks (ANN), Genetic Algorithms, Hidden Markov Models, Support Vector Machines, etc. can be used. This piece of work focuses exclusively on ANNs. Since ANNs have many standard architectural variations divided into two basic computational categories namely the recurrent networks and feed-forward networks, representative networks from each have been selected for study, i.e. Feed Forward Multilayer Perceptron (FF) network for feed-forward networks category and Elman (EL), and Nonlinear Autoregressive Exogenous Model (NARX) networks for the recurrent networks category. The main objective of this work is to identify the most suitable neural architecture for application of LFD in learning different robot actions. The sensor and actuator streams of demonstrated action are used as training data for ANN learning. Consequently, the learning capability is measured by comparing the error between demonstrator and corresponding learner streams. To achieve fairness in comparison three steps have been taken. First, Dynamic Time Warping is used to measure the error between demonstrator and learner streams, which gives resilience against translation in time. Second, comparison statistics are drawn between the best, instead of weight-equal, configurations of competing architectures so that learning capability of any architecture is not forced handicap. Third, each configuration's error is calculated as the average of ten trials of all possible learning sequences with random weight initialization so that the error value is independent of a particular sequence of learning or a particular set of initial weights. Six experiments are conducted to get a performance pattern of each architecture. In each experiment, a total of nine different robot actions were tested. Error statistics thus obtained have shown that NARX architecture is most suitable for this learning problem whereas Elman architecture has shown the worst suitability. Interestingly the computationally lesser MLP gives much lower and slightly higher error statistics compared to the computationally superior Elman and NARX neural architectures, respectively. Service robots will play an increasing and more important role in the society in the next years. One of the main challenges is to endow robots with enough autonomy to operate on real environments. To reach that goal, the design of controllers to solve simple tasks must be automatized. Engineers look for learning algorithms that are general, robust, require low expertise knowledge, and generate controllers that can run on the real robot without any tuning stage. In this paper, a framework to learn behaviors (controllers) in mobile robotics, fulfilling the previous requirements, has been used. The framework is based on two modules: dataset generation and a data-driven evolutionary-based learning algorithm to obtain fuzzy controllers. Nevertheless, the design of a fuzzy controller still requires the selection of the type of learning algorithm, and also to choose the value of some design parameters. In this paper we present an exhaustive study on a set of evolutionary-based data-driven learning algorithms, for learning fuzzy controllers in mobile robotics, that cover a wide range of the accuracy interpretability trade-off. The study has also evaluated the influence of the values of all the design parameters over accuracy and interpretability. The objective is to analyze the performance of the different algorithms for the design of behaviors in mobile robotics, and to extract some general rules that can help in the process to design new behaviors. The analysis comprises two different behaviors (wall-following and moving object following) and more than 450 tests, both in simulation and on a Pioneer II AT robot. Results have shown very good performances in complex and realistic conditions for the different combinations of algorithms and parameters. An Autonomous Mobile Robot (AMR) is a machine able to extract information from its environment and use knowledge about its world to move safely in a meaningful and purposeful manner. Robot Navigation and Obstacle Avoidance are from the most important problems in mobile robots, especially in unknown environments. It must be able to interact with other objects safely. Several techniques such as Fuzzy logic, Reinforcement learning, Neural Networks and Genetic Algorithms, have applied to AMR in order to improve their performance. During the past several years Hybrid Genetic-fuzzy method has emerged as one of the most active and fruitful areas for research in the application of intelligent system design. The objective of this work is to provide a Hybrid method by which an improved set of rules governing the actions and behavior of a simple navigating and obstacle avoiding AMR. Genes are in the form of distances and angles labels. The chromosomes are represented as a rule written in a Boolean algebraic form. The method used to enhance the performance employs a simulation model designed by using Visual Basic software. This paper proposes a reinforcement ant optimized fuzzy controller (FC) design method, called RAOFC, and applies it to wheeled-mobile-robot wall-following control under reinforcement learning environments. The inputs to the designed FC are range-finding sonar sensors, and the controller output is a robot steering angle. The antecedent part in each fuzzy rule uses interval type-2 fuzzy sets in order to increase FC robustness. No a priori assignment of fuzzy rules is necessary in RAOFC. An online aligned interval type-2 fuzzy clustering (AIT2FC) method is proposed to generate rules automatically. The AIT2FC not only flexibly partitions the input space but also reduces the number of fuzzy sets in each input dimension, which improves controller interpretability. The consequent part of each fuzzy rule is designed using Q-value aided ant colony optimization (QACO). The QACO approach selects the consequent part from a set of candidate actions according to ant pheromone trails and Q-values, both of whose values are updated using reinforcement signals. Simulations and experiments on mobile-robot wall-following control show the effectiveness and efficiency of the proposed RAOFC. This paper proposes an evolutionary-group-based particle-swarm-optimization (EGPSO) algorithm for fuzzy-controller (FC) design. The EGPSO uses a group-based framework to incorporate crossover and mutation operations into particle-swarm optimization. The EGPSO dynamically forms different groups to select parents in crossover operations, particle updates, and replacements. An adaptive velocity-mutated operation (AVMO) is incorporated to improve search ability. The EGPSO is applied to design all of the free parameters in a zero-order Takagi-Sugeno-Kang (TSK)-type FC. The objective of EGPSO is to improve fuzzy-control accuracy and design efficiency. Comparisons with different population-based optimizations of fuzzy-control problems demonstrate the superiority of EGPSO performance. In particular, the EGPSO-designed FC is applied to mobile-robot navigation in unknown environments. In this application, the robot learns to follow object boundaries through an EGPSO-designed FC. A simple learning environment is created to build this behavior without an exhaustive collection of input-output training pairs in advance. A behavior supervisor is proposed to combine the boundary-following behavior and the target-seeking behavior for navigation, and the problem of dead cycles is considered. Successful mobile-robot navigation in simulation and real environments verifies the EGPSO-designed FC-navigation approach. In view of many applications, in recent years, there has been increasing interest in robot's control. Two intelligent controllers based on fuzzy logic and neural network are developed to trace the desired trajectory for a robot. A variety of evolutionary algorithms, have been proposed to approximately solve problems of common engineering applications. Increasingly common applications involve automatic learning of nonlinear mappings that govern the behavior of control systems. In many cases where robot control is of primary concern, the systems used to demonstrate the effectiveness of evolutionary algorithms often do not represent practical robotic systems. In this paper, genetic algorithms (GA) are the evolutionary strategy of interest. This procedure and the manner in which fuzzy controllers are codified into chromosomes is described. It is applied to learn fuzzy control rules for a practical autonomous vehicle steering control problem, namely, path tracking. GA handles the simultaneous evolution of membership functions and rule bases for the fuzzy path tracker. Simulation results show that the proposed fuzzy controller whose all parameters have been tuned simultaneously using GAs, offers advantages over existing controllers and has improved performance. Genetic Network Programming (GNP) has been proposed as one of the evolutionary algorithms and extended with reinforcement learning (GNP-RL). The combination of evolution and learning can efficiently evolve programs and the fitness improvement has been confirmed in the simulations of tileworld problems, elevator group supervisory control systems, stock trading models and wall following behavior of Khepera robot. However, its robustness in testing environments has not been analyzed in detail yet. In this paper, the learning mechanism in the testing environment is introduced and it is confirmed that GNP-RL can show the robustness using a robot simulator WEBOTS, especially when unexperienced sensor troubles suddenly occur. The simulation results show that GNP-RL works well in the testing even if wrong sensor information is given because GNP-RL has a function to change programs using alternative actions automatically. In addition, the analysis on the effects of the parameters of GNP-RL is carried out in both training and testing simulations.