In general, model selection and design is based on the tradeoff between prediction performance and model complexity [1]}, [2]}. On one hand, making predictions with high accuracy is the essential goal of learning a model [3]}. A model is expected to be able to capture the underlying patterns hidden in the training data and achieve predictions of accuracy as high as possible. In order to represent a large amount of knowledge and obtain high accuracy, a model with a high expressive capacity, a large degree of freedom and a large training set is required [4]}. To this extent, a model with more parameters and higher complexity is favored.
On the other hand, an overly complex model may be difficult to train and may incur unnecessary resource consumption, such as storage, computation and time cost [1]}. Unnecessary resource consumption should be avoided particularly in practical large scale applications [6]}. To this extent, a simpler model with comparable accuracy is preferred than a more complicated one.
