Pre-training.

Initialization is an essential step for both the optimization and for the training of the neural networks. Without proper initialization, the model collapses into selecting only one specific \(\beta \)  network while eliminating the contribution of others, as is observed in many other mixture network modeling. We have added pre-training to solve this problem and have compared it with regularization. We have shown that it is much more effective in terms of accuracy. Pre-training is an important stage in the \(\alpha \) -\(\beta \)  network training. The model could approach the base model of a single neural network classifier without proper pre-training because of the large gradients at the beginning of the training stage; large gradients cause the selector to saturate and select only one \(\beta \)  network. Therefore, the full capabilities of the \(\alpha \) -\(\beta \)  network can only be achieved using proper pre-training, which proves useful in finding subgroups of data as well as in yielding a better performance. We used the idea presented in [1]} to cluster the activity data. In detail, a base network was trained with sensor readings as input and activities as output.
Next, the CNN segment of the network was used to embed the sensor readings into the features which have proven to be descriptive of the input data [2]}, [3]}. Subsequently, we used K-means to cluster the input into a fixed number of clusters. Finally, we trained our \(\alpha \)  network to learn the mapping between sensor readings of activity to clusters.
<FIGURE>