Orthogonal Stochastic Configuration Networks with Adaptive Construction Parameter for Data Analytics
As a randomized learner model, SCNs are remarkable that the random weights
and biases are assigned employing a supervisory mechanism to ensure universal
approximation and fast learning. However, the randomness makes SCNs more likely
to generate approximate linear correlative nodes that are redundant and low
quality, thereby resulting in non-compact network structure. In the light of a
fundamental principle in machine learning, that is, a model with fewer
parameters holds improved generalization. This paper proposes orthogonal SCN,
termed OSCN, to filtrate out the low-quality hidden nodes for network structure
reduction by incorporating Gram-Schmidt orthogonalization technology. The
universal approximation property of OSCN and an adaptive setting for the key
construction parameters have been presented in details. In addition, an
incremental updating scheme is developed to dynamically determine the output
weights, contributing to improved computational efficiency. Finally,
experimental results on two numerical examples and several real-world
regression and classification datasets substantiate the effectiveness and
feasibility of the proposed approach.