One of the current challenges in machine learning is how to deal with data coming at increasing rates in data streams. New predictive learning strategies are needed to cope with the high throughput data and concept drift. One of the data stream mining tasks where new learning strategies are needed is multi-target regression, due to its applicability in a high number of real world problems. While reliable and effective learning strategies have been proposed for batch multi-target regression, few have been proposed for multi-target online learning in data streams. Besides, the existing solutions do not consider the occurrence of inter-target correlations when making predictions. In this work, we propose an online learning strategy for multi-target regression in data streams. The proposed strategy extends a single target decision tree online learning algorithm to allow multi-target online regression learning in data streams. For such, the proposed strategy, called Stacked Single-target Hoeffding Tree (SST-HT), uses the inter-target dependencies as an additional information source to enhance predictive accuracy. Throughout an extensive experimental setup, we evaluate our proposal against state-of-the-art decision tree-based algorithms for online multi-target regression. According to the experimental results, SST-HT presents superior predictive accuracy, with a small increase in the processing time and memory requirements.
Following the trend of using tree-based methods for streaming scenarios, proposed an extension for the FIMT-MT algorithm called iSOUP-Tree (incremental Structured Output Prediction Tree). Their proposal builds upon the research of @cite_8 by enabling the support to categorical features and employing an adaptive prediction model at the leaves. Instead of only using perceptrons at the leaves, iSOUP-Tree also maintains a mean predictor for each target and selects the best current model by monitoring a faded error metric for each of them. The authors adapted the iSOUP-Tree algorithm for multiple settings, including ensembles (Bagging and Random Forest) and Option Trees , and employed all the variations of the MTR method to tackle multi-label classification tasks .
@cite_8
2491694318
We study resource-limited online learning, motivated by the problem of conditional-branch outcome prediction in computer architecture. In particular, we consider (parallel) time and space-efficient ensemble learners for online settings, empirically demonstrating benefits similar to those shown previously for offline ensembles. Our learning algorithms are inspired by the previously published “boosting by filtering” framework as well as the offline Arc-x4 boosting-style algorithm. We train ensembles of online decision trees using a novel variant of the ID4 online decision-tree algorithm as the base learner, and show empirical results for both boosting and bagging-style online ensemble methods. Our results evaluate these methods on both our branch prediction domain and online variants of three familiar machine-learning benchmarks. Our data justifies three key claims. First, we show empirically that our extensions to ID4 significantly improve performance for single trees and additionally are critical to achieving performance gains in tree ensembles. Second, our results indicate significant improvements in predictive accuracy with ensemble size for the boosting-style algorithm. The bagging algorithms we tried showed poor performance relative to the boosting-style algorithm (but still improve upon individual base learners). Third, we show that ensembles of small trees are often able to outperform large single trees with the same number of nodes (and similarly outperform smaller ensembles of larger trees that use the same total number of nodes). This makes online boosting particularly useful in domains such as branch prediction with tight space restrictions (i.e., the available real-estate on a microprocessor chip).