First of all, determining the weight between the input and the hidden layers in the KCNet can be seen as a special case of random weight generation in a NNRW, and the KCNet also uses a bounded nonlinear piecewise continuous activation function, which is the ReLU in our setting. Therefore, KCNet inherits the universal approximation property of NNRWs, which means that a single-hidden-layer network with randomly generated hidden nodes and with a broad type of activation functions can universally approximate any continuous target functions in any compact subset \(\mathcal {X} \subseteq \mathbb {R}^{d}\)  [1]}, [2]}, [3]}. So, theoretically, the more the number of hidden nodes in the KCNet (i.e., the wider the hidden layer), the better its performance (see the details of KCNet performance results in Section ).
