Machine learning for sequential data: A review <s> The Sliding Window Method <s> This paper describes NETtalk, a class of massively-parallel network systems that learn to convert English text to speech. The memory representations for pronunciations are learned by practice and are shared among many processing units. The performance of NETtalk has some similarities with observed human performance. (i) The learning follows a power law. (ii) The more words the network learns, the better it is at generalizing and correctly pronouncing new words, (iii) The performance of the network degrades very slowly as connections in the network are damaged: no single link or processing unit is essential. (iv) Relearning after damage is much faster than learning during the original training. (v) Distributed or spaced practice is more effective for long-term retention than massed practice. Network models can be constructed that have the same performance and learning characteristics on a particular task, but differ completely at the levels of synaptic strengths and single-unit responses. However, hierarchical clustering techniques applied to NETtalk reveal that these different networks have similar internal representations of letter-to-sound correspondences within groups of processing units. This suggests that invariant internal representations may be found in assemblies of neurons intermediate in size between highly localized and completely distributed representations. <s> BIB001 </s> Machine learning for sequential data: A review <s> The Sliding Window Method <s> We present a new method for predicting the secondary structure of globular proteins based on non-linear neural network models. Network models learn from existing protein structures how to predict the secondary structure of local sequences of amino acids. The average success rate of our method on a testing set of proteins non-homologous with the corresponding training set was 64.3% on three types of secondary structure (alpha-helix, beta-sheet, and coil), with correlation coefficients of C alpha = 0.41, C beta = 0.31 and Ccoil = 0.41. These quality indices are all higher than those of previous methods. The prediction accuracy for the first 25 residues of the N-terminal sequence was significantly better. We conclude from computational experiments on real and artificial structures that no method based solely on local information in the protein sequence is likely to produce significantly better results for non-homologous proteins. The performance of our method of homologous proteins is much better than for non-homologous proteins, but is not as good as simply assuming that homologous sequences have identical structures. <s> BIB002 </s> Machine learning for sequential data: A review <s> The Sliding Window Method <s> One method for detecting fraud is to check for suspicious changes in user behavior. This paper describes the automatic design of user profiling methods for the purpose of fraud detection, using a series of data mining techniques. Specifically, we use a rule-learning program to uncover indicators of fraudulent behavior from a large database of customer transactions. Then the indicators are used to create a set of monitors, which profile legitimate customer behavior and indicate anomalies. Finally, the outputs of the monitors are used as features in a system that learns to combine evidence to generate high-confidence alarms. The system has been applied to the problem of detecting cellular cloning fraud based on a database of call records. Experiments indicate that this automatic approach performs better than hand-crafted methods for detecting fraud. Furthermore, this approach can adapt to the changing conditions typical of fraud detection environments. <s> BIB003
The sliding window method converts the sequential supervised learning problem into the classical supervised learning problem. It constructs a window classifier h w that maps an input window of width w into an individual output value y. Specifically, let d = (w − 1)/2 be the "half-width" of the window. Then h w predicts y i,t using the window In effect, the input sequence x i is padded on each end by d "null" values and then converted into N i separate examples. The window classifier h w is trained by converting each sequential training example (x i , y i ) into windows and then applying a standard supervised learning algorithm. A new sequence x is classified by converting it to windows, applying h w to predict each y t and then concatenating the y t 's to form the predicted sequence y. The obvious advantage of this sliding window method is that permits any classical supervised learning algorithm to be applied. Sejnowski and Rosenberg BIB001 applied the backpropagation neural network algorithm with a 7-letter sliding window to the task of pronouncing English words. A similar approach (but with a 15-letter window) was employed by Qian and Sejnowski BIB002 to predict protein secondary structure from the protein's sequence of amino acid residues. Provost and Fawcett BIB003 addressed the problem of cellular telephone cloning by applying the RL rule learning system to day-long windows from telephone calling logs. Although the sliding window method gives adequate performance in many applications, it does not take advantage of correlations between nearby y t values. To be more precise, the only relationships between nearby y t values that are captured are those that are predictable from nearby x t values. If there are correlations among the y t values that are independent of the x t values, then these are not captured.