An Overview on Concept Drift Learning <s> I. INTRODUCTION <s> Abstract A neon circuit which consists of nonvolatile metal-ferroelectric-semiconductor field effect transistors (MFSFETs) and a uni-junction transistor (UJT) has been proposed. In the proposed circuit MFSFETs act as analog memories to store the synaptic weights which can be changed by the adaptive-learning process during the operations. In this paper, we first simulate the operation of the ferroelectric neuron circuit using a circuit simulator, SPICE. It is shown that the output frequency of the proposed neuron circuit can be changed after it processes a certain number of input pulses. Then, we report the fabrication of UJTs and UJT pulse oscillation circuits using silicon-on-insulator (SOI) substrates. It is found that the output frequency increases with decreasing the charging time of the capacitor in the circuit and that the operation at higher frequencies is possible for integrated UJT oscillation circuits. Finally, we demonstrate the memory and learning properties of n-channel ferroelectric-gate FETs using (Pb,La)(Zr,Ti)O3 (PLZT) films. It is shown that the drain current of the PLZT/SrTiO3/Si FETs can be controlled by a “write” pulse before the measurements. <s> BIB001 </s> An Overview on Concept Drift Learning <s> I. INTRODUCTION <s> Most of the work in machine learning assume that examples are generated at random according to some stationary probability distribution. In this work we study the problem of learning when the distribution that generate the examples changes over time. We present a method for detection of changes in the probability distribution of examples. The idea behind the drift detection method is to control the online error-rate of the algorithm. The training examples are presented in sequence. When a new training example is available, it is classified using the actual model. Statistical theory guarantees that while the distribution is stationary, the error will decrease. When the distribution changes, the error will increase. The method controls the trace of the online error of the algorithm. For the actual context we define a warning level, and a drift level. A new context is declared, if in a sequence of examples, the error increases reaching the warning level at example k w , and the drift level at example k d . This is an indication of a change in the distribution of the examples. The algorithm learns a new model using only the examples since k w . The method was tested with a set of eight artificial datasets and a real world dataset. We used three learning algorithms: a perceptron, a neural network and a decision tree. The experimental results show a good performance detecting drift and with learning the new concept. We also observe that the method is independent of the learning algorithm. <s> BIB002 </s> An Overview on Concept Drift Learning <s> I. INTRODUCTION <s> On-line learning in domains where the target concept depends on some hidden context poses serious problems. A changing context can induce changes in the target concepts, producing what is known as concept drift. We describe a family of learning algorithms that flexibly react to concept drift and can take advantage of situations where contexts reappear. The general approach underlying all these algorithms consists of (1) keeping only a window of currently trusted examples and hypotheses; (2) storing concept descriptions and re-using them when a previous context re-appears; and (3) controlling both of these functions by a heuristic that constantly monitors the system's behavior. The paper reports on experiments that test the systems' performance under various conditions such as different levels of noise and different extent and rate of concept drift. <s> BIB003 </s> An Overview on Concept Drift Learning <s> I. INTRODUCTION <s> We present an ensemble method for concept drift that dynamically creates and removes weighted experts in response to changes in performance. The method, dynamic weighted majority (*DWM*), uses four mechanisms to cope with concept drift: It trains online learners of the ensemble, it weights those learners based on their performance, it removes them, also based on their performance, and it adds new experts based on the global performance of the ensemble. After an extensive evaluation---consisting of five experiments, eight learners, and thirty data sets that varied in type of target concept, size, presence of noise, and the like---we concluded that *DWM* outperformed other learners that only incrementally learn concept descriptions, that maintain and use previously encountered examples, and that employ an unweighted, fixed-size ensemble of experts. <s> BIB004 </s> An Overview on Concept Drift Learning <s> I. INTRODUCTION <s> Concept drift refers to a non stationary learning problem over time. The training and the application data often mismatch in real life problems. In this report we present a context of concept drift problem 1. We focus on the issues relevant to adaptive training set formation. We present the framework and terminology, and formulate a global picture of concept drift learners design. We start with formalizing the framework for the concept drifting data in Section 1. In Section 2 we discuss the adaptivity mechanisms of the concept drift learners. In Section 3 we overview the principle mechanisms of concept drift learners. In this chapter we give a general picture of the available algorithms and categorize them based on their properties. Section 5 discusses the related research fields and Section 5 groups and presents major concept drift applications. This report is intended to give a bird's view of concept drift research field, provide a context of the research and position it within broad spectrum of research fields and applications. <s> BIB005 </s> An Overview on Concept Drift Learning <s> I. INTRODUCTION <s> It is challenging to use traditional data mining techniques to deal with real-time data stream classifications. Existing mining classifiers need to be updated frequently to adapt to the changes in data streams. To address this issue, in this paper we propose an adaptive ensemble approach for classification and novel class detection in concept drifting data streams. The proposed approach uses traditional mining classifiers and updates the ensemble model automatically so that it represents the most recent concepts in data streams. For novel class detection we consider the idea that data points belonging to the same class should be closer to each other and should be far apart from the data points belonging to other classes. If a data point is well separated from the existing data clusters, it is identified as a novel class instance. We tested the performance of this proposed stream classification model against that of existing mining algorithms using real benchmark datasets from UCI (University of California, Irvine) machine learning repository. The experimental results prove that our approach shows great flexibility and robustness in novel class detection in concept drifting and outperforms traditional classification models in challenging real-life data stream applications. <s> BIB006 </s> An Overview on Concept Drift Learning <s> I. INTRODUCTION <s> Concept drift primarily refers to an online supervised learning scenario when the relation between the input data and the target variable changes over time. Assuming a general knowledge of supervised learning in this article, we characterize adaptive learning processes; categorize existing strategies for handling concept drift; overview the most representative, distinct, and popular techniques and algorithms; discuss evaluation methodology of adaptive algorithms; and present a set of illustrative applications. The survey covers the different facets of concept drift in an integrated way to reflect on the existing scattered state of the art. Thus, it aims at providing a comprehensive introduction to the concept drift adaptation for researchers, industry analysts, and practitioners. <s> BIB007 </s> An Overview on Concept Drift Learning <s> I. INTRODUCTION <s> The major problem of online learning or incremental learning is that, target function is frequently changing over time. This problem is commonly known as concept drift. Concept drift can be is further complicated if the dataset is class- imbalanced. There are different learning methods presented so far to handle concept drift like rule-based systems, decision trees, Naive Bayes, support vector machines, instance based learning, ensemble of classifiers, etc. This learning method requires further to combined with methods of drift detection in order to constantly monitor the performance of concept drift, however online changes detection was failed. In literature there are many methods presented for learning from data streams and drift detection, but most of methods failed to achieve speed and accuracy due to data inconsistency. In this project our goal is to present efficient method for online and non-parametric drift detection. This proposed method is based on recently presented Hoeffdings Bounds and HDDM. It handles concept drift regardless of the learning model to monitor the performance metrics measured during the learning process, to trigger drift signals when a significant variation has been detected. The existing system however as Naive Bayes classifier are having limitations, there is no scope to improve accuracy of HDDM. The Propose system will be efficiently provide drift detection method for data stream mining to improve accuracy. <s> BIB008
Traditional learning algorithms generally consider being in a static environment. However, data in the real-world environment may have a dynamic behavior, and the concept can change BIB002 . The term Concept Drift address the problem of learning in dynamic environments BIB005 . Therefore, concept drift occurs when the training and testing data diverge. In many areas, the dilemma of concept drift can be found including monitoring, planning, personal assistance, applications to handle diverse environments BIB007 , among others. Additionally, many other environments may contain hidden concept drift as well BIB001 -  . The literature proposes several methods for dealing with the concept drift. Widmer and Kubat BIB003 presented the FLORA (FLOating Rough Approximation) framework: a family of methods that maintains a set of descriptors, employing dynamic size window to select the descriptors that correctly assimilate the current concept. Klinkenberg and Joachims  employed Support Vector Machines (SVM) to deal with concept drift, preserving training instances located within a window upon data and discarding irrelevant instances that minimize the generalization error. The authors employed the following data management approaches: ''no memory,'' ''full memory,'' ''window of fixed size,'' and ''window of adaptive size.'' Stanley  proposed the Concept Drift Committee, a committee of hypotheses whose weighted vote make decisions on the current classification. A new member replaces an older member of the committee if the voting record of the older member falls below a threshold. Later on, Kolter and Maloof BIB004 proposed an ensemble algorithm to detect concept drift. Weighted experts are added and removed dynamically as changes in performance are noticed. With dynamic weighted majority vote for classification phase, the ensemble learners are weighted according to their performance, and the method deletes or creates new learners based on the global ensemble performance. Finally, Farid et al. BIB006 proposed an ensemble algorithm based on decision trees able to detect new classes. Since concept drift is an important area that has gained the attention of the last years, this work presents a compilation of several works to foster the research in the area of knowledge. The main contribution of this survey is to study different techniques to detect and deal with concept drift, as well as to study public synthetic and real datasets in this area. The remainder of this paper is organized as follows. Section II presents the main theoretical background regarding concept drift, types of algorithms to deal with (Subsection II-A), and techniques in the literature that handles the concept drift issue (Section III). Section IV presents a summary of the FIGURE 1. Types of concept drift, extracted from BIB008 : (a) sudden drift, (b) incremental drift, (c) gradual drift, (d) recurring drift, (e) outlier, and (f) noise. articles studied, and Section V states conclusions and future directions of the area.