Context-Awareness: Mixture of Experts Model
In many applications of machine learning, heterogeneous data can be divided into smaller homogeneous groups that can be modeled more accurately. Context-awareness, as an example, plays an important role in improving the performance of activity recognition systems [1]}, [2]}. In Mixture of Experts (MoE) models [3]} such group-level clustering and modeling comes as a single training step, rather than splitting data a prior then building models. Elements are clustered based upon their relationship and the next level modeling accuracy. MoEs have successfully served different applications from classification and regression tasks [4]}[5]} to phenotyping in medical datasets [6]}.
In [3]}[4]}[9]}, authors provided formulations of probabilities of observed given different experts.
The MoE structure consists of two components: A gate and several experts. The gate is often modeled by Gaussian Mixture Models (GMM) [10]}[11]} and neural networks [12]}, while expert modeling is more dependant on the application including SVM [12]} [14]}. This work used neural networks for both.
