Nonlinear System Identification Using Type-2 Fuzzy Recurrent Wavelet Neural Network

In this paper, the integration of Type-2 fuzzy set theory and recurrent wavelet neural network(WNN) is proposed to allow managing of non-uniform uncertainties for identifying non-linear dynamic system. The proposed Type-2 fuzzy WNN is inherently a recurrent multilayered network which constructed based on a set of Type-2 fuzzy rules and recurrent connections in the second layer of the FWNN. Each rule comprises a wavelet function in the consequent part. The structure has both advantages of recurrent and wavelet neural network which expand the basic ability of fuzzy neural network to deal with temporal problems. Both antecedent and consequent parameters update rules are derived based on the gradient descent method. The structure is applied in the identification of dynamic plants which is commonly used in the literature. Simulation result from the identification of a second-order non-linear plant confirms the better performance and effectiveness of the proposed structure.


Introduction
The use of conventional modeling approaches in the study of nonlinear dynamical system identification suffers from some deficiencies, including lack of precise, strongly nonlinear and time-varying behavior, formal knowledge about the system and a high degree of uncertainty. Under such conditions, modelfree approaches such as artificial neural networks, fuzzy logic systems and fusion of them which are universal approximators, have been developed to compensate for the effects of nonlinearities and system uncertainties without significant prior knowledge of system dynamics [1,2].
Type-2 fuzzy logic systems as an extension of its type-1 have better ability to deal with and model uncertainties and measurement noise [3]. Different optimization methods are applied to estimate the parameters of neural network and fuzzy neural systems from numerical data. The methods can be categories into two types: derivative-based (such as gradient descent, least square) and derivative-free optimization methods(such as particle swarm optimization and genetic algorithm).
Back propagation error method is mainly used for training feed-forward neural networks. Because of some deficiencies of BP algorithm like slow convergence rate, being able to trap to local optima and dependent to the initial condition, some differ-1 ent methods based on BP algorithm are developed such as Momentum, adaptive learning rate. These modifications accelerate the convergence of network training, but they have not the ability to avoid areas of local optima [4].
Wavelet function as the activation function can enhance the advantages of neural networks for faster learning ability and wavelet decomposition for identification purposes. In literature, wavelet neural network (WNN) and its integration with fuzzy logic (FWNN) to determine an optimal definition of the premise and consequent part of fuzzy rules, are applied in the identification and control of nonlinear dynamical systems [5][6][7][8].
To enhance the approximation accuracy of the network and capture the dynamic behavior of the system, different kinds of feedback loops in the topology of the network are embedded. The combination of RNN and fuzzy logic (RFNN) are applied to various practical application [9][10][11], and different methods for their parameters tuning are discussed.
In this study, a type-2 fuzzy recurrent wavelet neural network(T2FRWNN) is presented which integrates FWNN and recurrent neural network to benefits the advantages of both. The proposed dynamic identification structure is equipped the networks to process temporal information and learn sequences with using local internal feedback loops by feeding the lower and upper of type-2 membership function back to itself. Besides, network response in handling the uncertainties and imprecision can be achieved by utilization of type-2 FNN and wavelet activation function. For the parameter adaptation of the T2FRWNN, gradient descent is used.
The main body of this paper is organized as follows. In Section 2, the structure of T2FRWNN is introduced. In Section 3, the parameters update rules based on gradient descent are derived. In Section 4, simulation results are shown and discussed.
Finally, concluding remarks are given in Section 5.

Network structure
The T2FRWNN structure benefits from recurrent type-2 membership function in the premise part and crisp wavelet numbers for the consequent part. In other words, the structure is the recurrent version of the so-called A2-C0 fuzzy system [12] which implements a recurrent wavelet fuzzy model. Each fuzzy if-then rule base with I input variables considered here is as follows: where K is the number of membership function and f r is the output of rth rule (r = {1, 2..., N }).Ã ik is the kth type-2 fuzzy membership function related to ith input variable. ρ is the weight coefficient between the input and the hidden layer. Consequent part of above rule involved a wavelet function of input variables. Wavelets are defined by a family of functions a and b (a > 0, b ϵ R) as dilation and translation respectively. In this manuscript, among several families of wavelets, Mexican Hat is considered as mother wavelet function. where where, z ri = xi−bri ari . The structure of T2FRWNN proposed in this paper includes seven layers. In the first layer, the number of nodes is equal to the number of input signals. These 2 nodes are used for distributing input signals. In the second layer, each node corresponds to one linguistic term. For each input signal entering into the system, type-2 membership functions are used. The type-2 MFs have uncertain standard deviation and fixed center and the membership degree µ ik (x i ) and µ ik (x i ) are calculated according to (4) and (5).
where c ik is center of type-2 MF, σ ik and σ ik are the upper and lower standard deviation of the kth type-2 MF of ith input. Moreover, ξ ik is the recurrent parameter defined as (6) which store the past information of the network. It is to be noted that the feed back weights of nodes in layer 2 are interval values.
where θ ik and θ ik are considered as feed back weights of the nodes in this layer. Nodes in the third layer represent one fuzzy rule and perform a fuzzy meet operation on inputs from layer 2 using an algebraic product operation to obtain upper and lower firing strength w r and w r which are calculated as follows: Layer 4 determines the normalized values of the lower and the upper firing strength corresponding to each node in layer 3: Layer 5 is consequent layer. Nodes in this layer compute the product of normalized firing strengthw r ,w r and wavelet function of input variables. Layer 6 consists of two summation blocks, one for summation of upper and the other for summation of lower outputs of the previous layer. Layer 7 calculates the output of the network using (10) where q is the design parameter which enables to adjust the lower or the upper portions, depending on the level of certainty of the system [13]. Consequently, the adaptation laws for the parameters and the proof of the stability of the learning process are given using a time-varying q.

Parameter Update Rules For Learning
In order to obtain the learning algorithm for the parameters gradient descent method with adaptive learning rate is applied. The use of adaptive learning rate give an assurance of convergence and speeds up the learning of the network. Moreover, a momentum is used to accelerate the learning process. The parameters which should be updated are the parameters of premise and consequent part of fuzzy rules, including c ik , σ ik , σ ik , θ ik and θ ik as premise part of kth type-2 MF of ith input and a r , b r , and ρ r as the consequent part of the rth rule. The error function E(k) of T2FRWNN is defined by: where K is the number of input signals of the network. The parameters mentioned above are adjusted as follows: and for the coefficient q: where α is the learning rate, λ is the momentum, i is the number of input signals of the network (input neurons), r is the number of rules (r = {1, ..., N }) and k is the number of MFs (k = {1, ..., m}). The values of derivatives in (12) can be calculated by the following formulas (15)- (17): the coefficient q is updated as follows: The derivatives in (12) are determined by the following formulas (20)-(24): Figure 1: where,

Simulation Example
In this section, one simulation studies of nonlinear system identifications are considered regarding the proposed model and learning algorithm to evaluate the performance of proposed T2FRWNN. Fig.1 depicts the structure of the identification system. The inputs of the model are delayed values from the nonlinear plant output and delayed input signals from the plant. Here, the prob-lem is to find such values of parameters of proposed T2FRWNN by using them in the system for all input values the difference between plant output y(t) and the T2FRWNN output y N (t) will be minimum. As an example, identification of a second-order non-linear plant that has been used in [14] is considered. The process is described by the following difference equation: where y(k−1), y(k−2), y(k−3) are one-, two-and threestep delayed outputs of the plant, respectively, u(k) and u(k − 1) are current and one-step delayed inputs of the plant. In order to have a comparison, the identification of the same plant by using different models are considered in [14] and [15]. For the simulation studies, the input signal in this manuscript has the following expression:

Conclusion
This paper has proposed an T2FRWNN structure for identification of dynamic plants. According to the design aspect of the proposed structure, the Type-2 Tagaki-Sugeno fuzzy logic is integrated with wavelet neural network which makes a fuzzy input space into different wavelet-based subspaces. Besides, the recurrent part of the structure gives the abilities to attract dynamics and store temporary information. In the training aspect, the parameter update rules of the structure are derived based on the gradient descent algorithm. In the simulation example of system identification, the proposed T2FRWNN has a better performance in comparison with other models, despite smaller number of parameters.