Definition of an accurate system model for Automated Planner (AP) is often impractical, especially for real-world problems. Conversely, off-the-shelf planners fail to scale up and are domain dependent. These drawbacks are inherited from conventional transition systems such as Finite State Machines (FSMs) that describes the action-plan execution generated by the AP. On the other hand, Behavior Trees (BTs) represent a valid alternative to FSMs presenting many advantages in terms of modularity, reactiveness, scalability and domain-independence. In this paper, we propose a model-free AP framework using Genetic Programming (GP) to derive an optimal BT for an autonomous agent to achieve a given goal in unknown (but fully observable) environments. We illustrate the proposed framework using experiments conducted with an open source benchmark Mario AI for automated generation of BTs that can play the game character Mario to complete a certain level at various levels of difficulty to include enemies and obstacles.
Scheper et. al @cite_30 applied evolutionary learning to BTs for a real-world robotic (Micro Air Vehicle) application. It appears as the first real-world robotic application of evolving BTs. They used a (sub-optimal) manually crafted BT as an initial BT in the evolutionary learning process, and conducted experiments with a flying robot, while the BT that controls the robot is learning itself in every experiment. Finally, they demonstrated significant improvement in the performances of the evolved final BT comparing to the initial user-defined BT. While we take inspirations from this work, the downside is that this work require an initial BT for it to work, which goes against our model-free objective.
@cite_30
1910406352 2028249667 2502984779 1965675101
Evolutionary Robotics allows robots with limited sensors and processing to tackle complex tasks by means of sensory-motor coordination. In this article we show the first application of the Behavior Tree framework on a real robotic platform using the evolutionary robotics methodology. This framework is used to improve the intelligibility of the emergent robotic behavior over that of the traditional neural network formulation. As a result, the behavior is easier to comprehend and manually adapt when crossing the reality gap from simulation to reality. This functionality is shown by performing real-world flight tests with the 20-g DelFly Explorer flapping wing micro air vehicle equipped with a 4-g onboard stereo vision system. The experiments show that the DelFly can fully autonomously search for and fly through a window with only its onboard sensors and processing. The success rate of the optimized behavior in simulation is 88 , and the corresponding real-world performance is 54 after user adaptation. Although this leaves room for improvement, it is higher than the 46 success rate from a tuned user-defined controller. This paper proposes evolutionary wall-following control of a mobile robot using an interval type-2 fuzzy controller (IT2FC) with species-differential-evolution-activated continuous ant colony optimization (SDE-CACO). Both the position and speed of a mobile robot are controlled by using two IT2FCs to improve noise resistance ability. A new cost function is defined to accurately evaluate the wall-following performance of an evolutionary IT2FC. A two-stage training approach is proposed that learns a position IT2FC followed by a speed IT2FC to optimize both the wall-following accuracy and the moving speed. The proposed learning approach avoids the time consuming task of the exhaustive collection of supervised input-output training pairs. All fuzzy rules are generated online using a clustering-based approach during the evolutionary learning process. All of the free parameters in an online-generated IT2FC are optimized using SDE-CACO, in which an SDE mutation operation is incorporated within a continuous ACO to improve its explorative ability. The proposed SDE-CACO is compared with various population-based optimization algorithms to demonstrate its efficiency and effectiveness in the wall-following control problem. This study also includes experiments that demonstrate wall-following control utilizing a real mobile robot. The use of evolutionary robotics in robots with complex means of locomotion has, so far, mainly been limited to gait evolution. Increasing the number of degrees of freedom available to a controller significantly enlarges the search space, which in turn makes the evolution of solutions for a given task more challenging. In this paper, we present Evolutionary Repertoire-based Control (EvoRBC), an approach that enables the evolution of control for robots with arbitrary locomotion complexity. EvoRBC separates the synthesis of control into two levels: the generation of a repertoire of behavior primitives through the application of Quality Diversity techniques; and the evolution of a behavior arbitrator that uses the repertoire's primitives to solve a particular task. We evaluate EvoRBC in simulated robots with different numbers of degrees of freedom in two tasks, navigation and foraging. Our results show that while standard evolutionary approaches are highly affected by the locomotion complexity of the robot, EvoRBC is consistently able to evolve high-performing solutions. We also show that EvoRBC allows for the evolution of general controllers, that can be successfully used in robots different than those with which they were evolved. Learning in robotics typically involves choosing a simple goal (e.g. walking) and assessing the performance of each controller with regard to this task (e.g. walking speed). However, learning advanced, input-driven controllers (e.g. walking in each direction) requires testing each controller on a large sample of the possible input signals. This costly process makes difficult to learn useful low-level controllers in robotics. Here we introduce BR-Evolution, a new evolutionary learning technique that generates a behavioral repertoire by taking advantage of the candidate solutions that are usually discarded. Instead of evolving a single, general controller, BR-evolution thus evolves a collection of simple controllers, one for each variant of the target behavior; to distinguish similar controllers, it uses a performance objective that allows it to produce a collection of diverse but high-performing behaviors. We evaluated this new technique by evolving gait controllers for a simulated hexapod robot. Results show that a single run of the EA quickly finds a collection of controllers that allows the robot to reach each point of the reachable space. Overall, BR-Evolution opens a new kind of learning algorithm that simultaneously optimizes all the achievable behaviors of a robot.