We present a method for training a deep neural network containing sinusoidal activation functions to fit to time-series data. Weights are initialized using a fast Fourier transform, then trained with regularization to improve generalization. A simple dynamic parameter tuning method is employed to adjust both the learning rate and regularization term, such that stability and efficient training are both achieved. We show how deeper layers can be utilized to model the observed sequence using a sparser set of sinusoid units, and how non-uniform regularization can improve generalization by promoting the shifting of weight toward simpler units. The method is demonstrated with time-series problems to show that it leads to effective extrapolation of nonlinear trends.
 A new technique, wavelet network, is introduced to predict chaotic time series. By using this technique, firstly, we make accurate short-term predictions of the time series from chaotic attractors. Secondly, we make accurate predictions of the values and bifurcation structures of the time series from dynamical systems whose parameter values are changing with time. Finally we predict chaotic attractors by making long-term predictions based on remarkably few data points, where the correlation dimensions of predicted attractors are calculated and are found to be almost identical to those of actual attractors. The effectiveness of a multiscale neural net architecture for time series prediction of nonlinear dynamic systems is investigated. The prediction task is simplified by decomposing different scales of past windows into different scales of wavelets, and predicting the coefficients of each scale of wavelets by means of a separate multilayer perceptron. The short-term history is decomposed into the lower scales of wavelet coefficients, which are utilized for detailed analysis and prediction, while the long-term history is decomposed into higher scales of wavelet coefficients that are used for the analysis and prediction of slow trends in the time series. These coordinated scales of time and frequency provide an interpretation of the series structures, and more information about the history of the series, using fewer coefficients than other methods. Results concerning scales of time and frequencies are combined by another expert perceptron, which learns the weight of each scale in the goal-prediction of the original time series. Each network is trained by backpropagation. The weights and biases are initialized by a clustering algorithm of the temporal patterns of the time series, which improves the prediction results as compared to random initialization. The suggested multiscale architecture outperforms the corresponding single-scale architectures. The employment of improved learning methods for each of the ScaleNet networks can further improve the prediction results.  Since EEG is one of the most important sources of information in therapy of epilepsy, several researchers tried to address the issue of decision support for such a data. In this paper, we introduce two fundamentally different approaches for designing classification models (classifiers); the traditional statistical method based on logistic regression and the emerging computationally powerful techniques based on artificial neural networks (ANNs). Logistic regression as well as feedforward error backpropagation artificial neural networks (FEBANN) and wavelet neural networks (WNN) based classifiers were developed and compared in relation to their accuracy in classification of EEG signals. In these methods we used FFT and autoregressive (AR) model by using maximum likelihood estimation (MLE) of EEG signals as an input to classification system with two discrete outputs: epileptic seizure or nonepileptic seizure. By identifying features in the signal we want to provide an automatic system that will support a physician in the diagnosing process. By applying AR with MLE in connection with WNN, we obtained novel and reliable classifier architecture. The network is constructed by the error backpropagation neural network using Morlet mother wavelet basic function as node activation function. The comparisons between the developed classifiers were primarily based on analysis of the receiver operating characteristic (ROC) curves as well as a number of scalar performance measures pertaining to the classification. The WNN-based classifier outperformed the FEBANN and logistic regression based counterpart. Within the same group, the WNN-based classifier was more accurate than the FEBANN-based classifier, and the logistic regression-based classifier. Support vector machine (SVM) is a very specific type of learning algorithms characterized by the capacity control of the decision function, the use of the kernel functions and the sparsity of the solution. In this paper, we investigate the predictability of financial movement direction with SVM by forecasting the weekly movement direction of NIKKEI 225 index. To evaluate the forecasting ability of SVM, we compare its performance with those of Linear Discriminant Analysis, Quadratic Discriminant Analysis and Elman Backpropagation Neural Networks. The experiment results show that SVM outperforms the other classification methods. Further, we propose a combining model by integrating SVM with the other classification methods. The combining model performs best among all the forecasting methods.  A local linear wavelet neural network (LLWNN) is presented in this paper. The difference of the network with conventional wavelet neural network (WNN) is that the connection weights between the hidden layer and output layer of conventional WNN are replaced by a local linear model. A hybrid training algorithm of particle swarm optimization (PSO) with diversity learning and gradient descent method is introduced for training the LLWNN. Simulation results for the prediction of time-series show the feasibility and effectiveness of the proposed method. Time series prediction techniques have been used in many real-world applications such as financial market prediction, electric utility load forecasting , weather and environmental state prediction, and reliability forecasting. The underlying system models and time series data generating processes are generally complex for these applications and the models for these systems are usually not known a priori. Accurate and unbiased estimation of the time series data produced by these systems cannot always be achieved using well known linear techniques, and thus the estimation process requires more advanced time series prediction algorithms. This paper provides a survey of time series prediction applications using a novel machine learning approach: support vector machines (SVM). The underlying motivation for using SVMs is the ability of this methodology to accurately forecast time series data when the underlying system processes are typically nonlinear, non-stationary and not defined a-priori. SVMs have also been proven to outperform other non-linear techniques including neural-network based non-linear prediction techniques such as multi-layer perceptrons.The ultimate goal is to provide the reader with insight into the applications using SVM for time series prediction, to give a brief tutorial on SVMs for time series prediction, to outline some of the advantages and challenges in using SVMs for time series prediction, and to provide a source for the reader to locate books, technical journals, and other online SVM research resources.