Machine learning algorithms for controlling devices will need to learn quickly, with few trials. Such a goal can be attained with concepts borrowed from continental philosophy and formalized using tools from the mathematical theory of categories. Illustrations of this approach are presented on a cyberphysical system: the slot car game, and also on Atari 2600 games. There is a growing need for algorithms that control cyberphysical systems to learn with very little data how to operate quickly in a partially-known environment. Many reinforcement-learning (RL) solutions using neural networks (NN) have proved to work well with emulators, for instance with the Atari 1 2600 games BID17 , or with real systems such as robots BID11 . However, these state-of-the-art approaches need a lot of training data, which may not be obtainable within the allowed time frame or budget. This work thus started as an alternative approach to teach computers to learn quickly to perform as efficiently as the existing solution with approximately one percent of the training data, time, and computing resources.We first review reinforcement learning methods for Markov Decision Processes (MDP) and Partially Observable MDP (POMDP). We then explain the motivation behind our continental-philosophyinspired approach. We describe the two classes of problems on which we focus: the bijective case, which may lead to playing by imitating, and the category-based approach, which should lead to a more innovative behavior of the control algorithms. Both approaches rely on knowledge accumulated during previous experiences, as in Lifelong Machine Learning BID6 .These two approaches are illustrated by results from both a commercial slot car game controlled by an 8-bit Arduino system, and from Atari 2600 video games running within the Arcade Learning Environment (ALE, see BID1 ). Continental philosophy lead us to formalize a mathematical concept to control an agent evolving in a world, whether it is simulated or real. The power of this framework was illustrated by the theoretical example of the slot car on unknown circuits. Results from experiments with a real slot car, using real analog signals confirmed our expectations, even though it only used a basic survival approach. Moreover, the same basic survival strategy was applied to two Atari 2600 games and showed the same trend: even though not as skilled as, for instance, DQN-based agents trained with two hundred million frames, our AI reached in less than ten thousand frames scores that DQN met after learning with a few million frames.The next steps are to apply the transposition properties to the Atari games, as we did for the slot car, which should further decrease the learning time when playing a new game. Moreover, going beyond the basic survival strategy will be mandatory to reach higher scores: approaches based on Monte-Carlo Tree Search will be investigated.
Continental-philosophy-inspired approach to learn with few data.
{"src": "scitldr-AIC_train", "paper_id": "SyF7Erp6W"}