The Challenge of Non-Technical Loss Detection using Artificial Intelligence: A Survey <s> B. Machine learning <s> We present an application of back-propagation networks to handwritten digit recognition. Minimal preprocessing of the data was required, but architecture of the network was highly constrained and specifically designed for the task. The input of the network consists of normalized images of isolated digits. The method has 1% error rate and about a 9% reject rate on zipcode digits provided by the U.S. Postal Service. <s> BIB001 </s> The Challenge of Non-Technical Loss Detection using Artificial Intelligence: A Survey <s> B. Machine learning <s> The amount of readily available on-line text has reached hundreds of billions of words and continues to grow. Yet for most core natural language tasks, algorithms continue to be optimized, tested and compared after training on corpora consisting of only one million words or less. In this paper, we evaluate the performance of different learning methods on a prototypical natural language disambiguation task, confusion set disambiguation, when trained on orders of magnitude more labeled data than has previously been used. We are fortunate that for this particular application, correctly labeled training data is free. Since this will often not be the case, we examine methods for effectively exploiting very large corpora when labeled data comes at a cost. <s> BIB002 </s> The Challenge of Non-Technical Loss Detection using Artificial Intelligence: A Survey <s> B. Machine learning <s> In this paper, the cooperation of two feature families for handwritten digit recognition using SVM (support vector machine) classifiers is examined. We investigate the advantages and weaknesses of various decision fusion schemes using statistical and rule-based reasoning. The obtained results show that it is difficult to exceed the recognition rate of a single classifier applied straightforwardly on both feature families as one set by rule based reasoning applied on the individual classifier decisions. However, the rule-based cooperation schemes enable an easy and efficient implementation of various rejection criteria. On the other hand, the statistical cooperation schemes offer a better possibility for fine tuning of the recognition versus the reliability tradeoff, which leads to recognition systems with high reliability that also keep high recognition rates. <s> BIB003 </s> The Challenge of Non-Technical Loss Detection using Artificial Intelligence: A Survey <s> B. Machine learning <s> Reinforcement learning offers one of the most general framework to take traditional robotics towards true autonomy and versatility. However, applying reinforcement learning to high dimensional movement systems like humanoid robots remains an unsolved problem. In this paper, we discuss different approaches of reinforcement learning in terms of their applicability in humanoid robotics. Methods can be coarsely classified into three different categories, i.e., greedy methods, ‘vanilla’ policy gradient methods, and natural gradient methods. We discuss that greedy methods are not likely to scale into the domain humanoid robotics as they are problematic when used with function approximation. ‘Vanilla’ policy gradient methods on the other hand have been successfully applied on real-world robots including at least one humanoid robot [3]. We demonstrate that these methods can be significantly improved using the natural policy gradient instead of the regular policy gradient. A derivation of the natural policy gradient is provided, proving that the average policy gradient of Kakade [10] is indeed the true natural gradient. A general algorithm for estimating the natural gradient, the Natural Actor-Critic algorithm, is introduced. This algorithm converges to the nearest local minimum of the cost function with respect to the Fisher information metric under suitable conditions. The algorithm outperforms non-natural policy gradients by far in a cart-pole balancing evaluation, and for learning nonlinear dynamic motor primitives for humanoid robot control. It offers a promising route for the development of reinforcement learning for truly high-dimensionally continuous state-action systems. <s> BIB004 </s> The Challenge of Non-Technical Loss Detection using Artificial Intelligence: A Survey <s> B. Machine learning <s> Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning. <s> BIB005 </s> The Challenge of Non-Technical Loss Detection using Artificial Intelligence: A Survey <s> B. Machine learning <s> Automated analysis of human affective behavior has attracted increasing attention from researchers in psychology, computer science, linguistics, neuroscience, and related disciplines. Promising approaches have been reported, including automatic methods for facial and vocal affect recognition. However, the existing methods typically handle only deliberately displayed and exaggerated expressions of prototypical emotions-despite the fact that deliberate behavior differs in visual and audio expressions from spontaneously occurring behavior. Recently efforts to develop algorithms that can process naturally occurring human affective behavior have emerged. This paper surveys these efforts. We first discuss human emotion perception from a psychological perspective. Next, we examine the available approaches to solving the problem of machine understanding of human affective behavior occurring in real-world settings. We finally outline some scientific and engineering challenges for advancing human affect sensing technology. <s> BIB006 </s> The Challenge of Non-Technical Loss Detection using Artificial Intelligence: A Survey <s> B. Machine learning <s> The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves. Here we introduce a new approach to computer Go that uses ‘value networks’ to evaluate board positions and ‘policy networks’ to select moves. These deep neural networks are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-play. Without any lookahead search, the neural networks play Go at the level of stateof-the-art Monte Carlo tree search programs that simulate thousands of random games of self-play. We also introduce a new search algorithm that combines Monte Carlo simulation with value and policy networks. Using this search algorithm, our program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0. This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away. <s> BIB007
To avoid the shortcomings of expert systems, a diametrically opposed approach is to learn patterns from data rather than hand-crafting rules. This branch of AI is called machine learning or pattern recognition. Both approaches have their justification and neither is generally better or worse than the other in artificial intelligence BIB003 . Machine learning gives computers the ability to learn from data without being explicitly programmed [50] . This property has allowed to significantly improve AI in various applications, such as in handwritten digit recognition BIB001 , facial expression recognition BIB006 or speech recognition  . Machine learning consists of three pillars: supervised, unsupervised and reinforcement learning. The term data mining is strongly related to machine learning, but has a wider scope that includes data cleaning, data preprocessing and concrete applications. 1) Supervised learning: Supervised learning algorithms learn patterns from labeled training examples (x (i) , y (i) ), in which x (i) is a training data point and y (i) is a corresponding label. This is also called function induction and typical applications include regression or classification  . This pillar is best understood at present time and there are a wide variety of available learning algorithms. The choice of which learning algorithm to apply to a concrete problem is challenging and often requires comparative experiments. However, having a lot of representative data is considered sometimes to be more relevant than the actual algorithm BIB002 . 2) Unsupervised learning: Unsupervised learning uses only unlabeled data points x (i) in order to find hidden structure in the data  . Applications include dimensionality reduction methods such as the Principal Component Analysis (PCA) or t-sne  and clustering algorithms such as K-means. 3) Reinforcement learning: In many learning problems, there is no intuitively correct supervision. Reinforcement learning is a reward-based learning technique for actions in order to get to a goal BIB005 . It has for example successfully been applied to humanoid robotics BIB004 , autonomous helicopter flying  and playing the game of Go at super-human performance BIB007 .