One of the current challenges in machine learning is how to deal with data coming at increasing rates in data streams. New predictive learning strategies are needed to cope with the high throughput data and concept drift. One of the data stream mining tasks where new learning strategies are needed is multi-target regression, due to its applicability in a high number of real world problems. While reliable and effective learning strategies have been proposed for batch multi-target regression, few have been proposed for multi-target online learning in data streams. Besides, the existing solutions do not consider the occurrence of inter-target correlations when making predictions. In this work, we propose an online learning strategy for multi-target regression in data streams. The proposed strategy extends a single target decision tree online learning algorithm to allow multi-target online regression learning in data streams. For such, the proposed strategy, called Stacked Single-target Hoeffding Tree (SST-HT), uses the inter-target dependencies as an additional information source to enhance predictive accuracy. Throughout an extensive experimental setup, we evaluate our proposal against state-of-the-art decision tree-based algorithms for online multi-target regression. According to the experimental results, SST-HT presents superior predictive accuracy, with a small increase in the processing time and memory requirements.
From @math it is possible to obtain a prediction @math for a instance @math . When constructing @math one expects that the obtained values of @math are as close as possible to the real measured target values @math . Regarding algorithmic solutions, accordingly to @cite_5 MTR problems have been solved using two main strategies for the solutions: global methods and local methods. Global methods correspond to solutions that use a single model to predict all target variables at once. Those methods implicitly model the inter-target dependencies, and also offer more compact and less computationally costly solutions for MTR tasks, being more viable to online scenarios. The local methods, on the other hand, employ traditional STR solutions and often manipulate or modify the input space to insert the inter-target dependency information within the problem modelling. In this sense, local methods use multiple ST regressors to solve a MTR task, frequently more than one regressor for each target variable , which in turn make them more costly solutions than the global ones. The simplest local solution for MTR tasks, as previously mentioned, consists in creating a ST regressor for each target and simply ignoring the underlying inter-target dependencies.
@cite_5
2182259890
Many real life prediction problems involve predicting a structured output. Multi-target regression is an instance of structured output prediction whose task is to predict for multiple target variables. Structured output algorithms are usually computationally and memory demanding, hence are not suited for dealing with massive amounts of data. Most of these algorithms can be categorized as local or global methods. Local methods produce individual models for each output component and combine them to produce the structured prediction. Global methods adapt traditional learning algorithms to predict the output structure as a whole. We propose the first rule-based algorithm for solving multi-target regression problems from data streams. The algorithm builds on the adaptive model rules framework. In contrast to the majority of the structured output predictors, this particular algorithm does not fall into the local and global categories. Instead, each rule specializes on related subsets of the output attributes. To evaluate the performance of the proposed algorithm, two other rule-based algorithms were developed, one using the local strategy and the other using the global strategy. These methods were compared considering their prediction error, memory usage, computational time, and model complexity. Experimental results on synthetic and real data show that the local-strategy algorithm usually obtains the lowest error. However, the proposed and the global-strategy algorithms use much less memory and run significantly much faster at the cost of a slightly increase in the error, which make them very attractive when computation resources are an important factor. Also, the models produced by the latter approaches are much easier to understand since considerably less rules are produced.