Neural Network-Based Fading Channel Prediction: A Comprehensive Overview <s> III. RECURRENT NEURAL NETWORK <s> We propose a robust learning algorithm and apply it to recurrent neural networks. This algorithm is based on filtering outliers from the data and then estimating parameters from the filtered data. The filtering removes outliers from both the target function and the inputs of the neural network. The filtering is soft in that some outliers are neither completely rejected nor accepted. To show the need for robust recurrent networks, we compare the predictive ability of least squares estimated recurrent networks on synthetic data and on the Puget Power Electric Demand time series. These investigations result in a class of recurrent neural networks, NARMA(p,q), which show advantages over feedforward neural networks for time series with a moving average component. Conventional least squares methods of fitting NARMA(p,q) neural network models are shown to suffer a lack of robustness towards outliers. This sensitivity to outliers is demonstrated on both the synthetic and real data sets. Filtering the Puget Power Electric Demand time series is shown to automatically remove the outliers due to holidays. Neural networks trained on filtered data are then shown to give better predictions than neural networks trained on unfiltered time series. > <s> BIB001 </s> Neural Network-Based Fading Channel Prediction: A Comprehensive Overview <s> III. RECURRENT NEURAL NETWORK <s> Neural network applications in adaptive multiuser detection (MUD) schemes are suggested here in the context of space division multiple access–orthogonal frequency division multiplexing system. In this paper, various neural network (NN) models like feed forward network (FFN), recurrent neural network (RNN) and radial basis function network (RBFN) are adopted for MUD. MUD using NN models outperforms other existing schemes like genetic algorithm--assisted minimum bit error rate (MBER) and minimum mean square error MUDs in terms of BER performance and convergence speed. Among these NN models, the FNN MUD performs efficiently as RNN in full load scenario, where the number of users is equal to number of receiving antennas. In overload scenario, where the number of users is more than the number of receiving antennas, the FNN MUD performs better than RNN MUD. Further, the RBFN MUD provides a significant enhancement in performance over FNN and RNN MUDs under both overload and full load scenarios because of its better classification ability due to Gaussian nonlinearity. Extensive simulation analysis considering Stanford University Interim channel models applied for fixed wireless applications shows improvement in convergence speed and BER performance of the proposed NN-based MUD algorithms. <s> BIB002
Recurrent neural network is a class of machine learning that has shown great potential in the field of time-series prediction BIB001 . Unlike a feed-forward network that only learns from training data, a recurrent neural network can also use its memory of past states to process sequences of inputs. RNN has several variants, among which a Jordan network is currently used to build a channel predictor, as illustrated in Fig. 1 . Basically, a simple network consists of three layers: an input layer with N i neurons, a hidden layer with N h neurons, and a layer having N o outputs. Each connection between the activation of a neuron in the predecessor layer and the input of a neuron in the successor layer is assigned a weight. Let w ln denote the weight connecting the n th input and the l th hidden neuron, while v ol is the weight for hidden neuron l and output o, where 1≤n≤N i , 1≤l≤N h , and 1≤o≤N o . Constructing a N h ×N i weight matrix W as and denoting the activation vector of the input layer and the recurrent component (feedback) at time step t as x(t) = x 1 (t), ..., tively, the input for the hidden layer is expressed in matrix form by where denotes the vector of biases in the hidden layer. Using a matrix F to represent the mapping from the output at the previous time step, i.e., y(t−1)= y 1 (t−1), ...  The behaviour of a neural network depends on activation functions, typically falling into the following categories: linear, rectified linear, threshold, sigmoid, and tangent. In general, a sigmoid function is employed to deal with nonlinearity, which is defined as Substituting (22) and (23) into (24), the activation vector of the hidden layer is thus where S(z h ) means an element-wise operation for simplicity, i.e., S(z h )= S(z 1 ), ..., S(z N h ) T . In analogous to (21), another weight matrix V having a dimension of N o ×N h with entries {v ol } is introduced. Then, the input for the output layer is z o (t) = Vh(t) + b y , where b y is the vector of biases in the output layer, resulting in an output vector: Like other data-driven AI techniques, the operation of a RNN is categorized into two phases: training and predicting. The training of a neural network is typically based on a fast algorithm known as Back-Propagation (BP). Provided a training dataset, the network feeds forward input data and compares the resulting output y against the desired value y 0 . Measured by a cost function, e.g., C = y 0 −y 2 , prediction errors are propagated back through the network, causing iteratively updating of weights and biases until a certain convergence condition reaches. To provide an initial impression of this process, the BP algorithm in combination with gradient descent learning for a feed forward network is briefly depicted: Start from an initial network state where {W, V, b h , b y } are randomly set. 1) Input a training example (x, y 0 ). 2) Feed-Forward: For the hidden layer, its input z h and activation h can be computed using (22) as: where ∇ notates a vector whose entries are partial derivatives, namely ∇ y C = . In addition, S (z o ) stands for the derivative of the activation function with respect to its corresponding input z o . Given a sigmod function in BIB002 , for instance, we have ). 4) Back-Propagation: e y is propagated back to the hidden layer to derive the error vector there, i.e., 5) Gradient Descent: With the back-propagated errors, the weights and biases are able to be updated according to the following rules: where η stands for the learning rate. The weights and biases are iteratively updated until the cost function is below a predefined threshold or the number of epochs reaches its maximal value. Once the training process completed, the trained network can be used to process upcoming samples. The training of a RNN typically use a variant of the BP algorithm, known as back-propagation through time (BPTT). It requires to unfold a recurrent neural network in time steps to form a pseudo feed-forward network, where the BP algorithm is applicable. Upon this, other more advanced or efficient approaches such as real-time recurrent learning and extended Kalman filtering have been designed.