Active learning and data manipulation techniques for generating training examples in meta learning
Algorithm selection is an important task in different domains of knowledge. Meta learning treats this task by adopting a supervised learning strategy. Training examples in meta learning (called meta examples) are generated from experiments performed with a pool of candidate algorithms in a number of problems, usually collected from data repositories or synthetically generated. A meta learner is then applied to acquire knowledge relating features of the problems and the best algorithms in terms of performance. In this paper, we address an important aspect in meta learning which is to produce a significant number of relevant meta examples. Generating a high quality set of meta examples can be difficult due to the low availability of real datasets in some domains and the high computational cost of labelling the meta examples. In the current work, we focus on the generation of meta examples for meta learning by combining: (1) a promising approach to generate new datasets (called datasetoids) by manipulating existing ones; and (2) active learning methods to select the most relevant datasets previously generated. The datasetoids approach is adopted to augment the number of useful problem instances for meta example construction. However not all generated problems are equally relevant. Active meta learning then arises to select only the most informative instances to be labelled. Experiments were performed in different scenarios, algorithms for meta learning and strategies to select datasets. Our experiments revealed that it is possible to reduce the computational cost of generating meta examples, while maintaining a good meta learning performance.
Neurocomputing