Machine Learning for Survival Analysis: A Survey <s> Ensemble Learning. Ensemble learning methods <s> Predicted survival probability functions of censored event free survival are improved by bagging survival trees. We suggest a new method to aggregate survival trees in order to obtain better predictions for breast cancer and lymphoma patients. A set of survival trees based on B bootstrap samples is computed. We define the aggregated Kaplan-Meier curve of a new observation by the Kaplan-Meier curve of all observations identified by the B leaves containing the new observation. The integrated Brier score is used for the evaluation of predictive models. We analyse data of a large trial on node positive breast cancer patients conducted by the German Breast Cancer Study Group and a smaller 'pilot' study on diffuse large B-cell lymphoma, where prognostic factors are derived from microarray expression values. In addition, simulation experiments underline the predictive power of our proposal. <s> BIB001 </s> Machine Learning for Survival Analysis: A Survey <s> Ensemble Learning. Ensemble learning methods <s> Random forests are a combination of tree predictors such that each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest. The generalization error for forests converges a.s. to a limit as the number of trees in the forest becomes large. The generalization error of a forest of tree classifiers depends on the strength of the individual trees in the forest and the correlation between them. Using a random selection of features to split each node yields error rates that compare favorably to Adaboost (Y. Freund & R. Schapire, Machine Learning: Proceedings of the Thirteenth International conference, aaa, 148–156), but are more robust with respect to noise. Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the splitting. Internal estimates are also used to measure variable importance. These ideas are also applicable to regression. <s> BIB002 </s> Machine Learning for Survival Analysis: A Survey <s> Ensemble Learning. Ensemble learning methods <s> We introduce random survival forests, a random forests method for the analysis of right-censored survival data. New survival splitting rules for growing survival trees are introduced, as is a new missing data algorithm for imputing missing data. A conservation-of-events principle for survival forests is introduced and used to define ensemble mortality, a simple interpretable measure of mortality that can be used as a predicted outcome. Several illustrative examples are given, including a case study of the prognostic implications of body mass for individuals with coronary artery disease. Computations for all examples were implemented using the freely available R-software package, randomSurvivalForest. <s> BIB003 </s> Machine Learning for Survival Analysis: A Survey <s> Ensemble Learning. Ensemble learning methods <s> We present a statistical perspective on boosting. Special emphasis is given to estimating potentially complex parametric or nonparametric models, including generalized linear and additive models as well as regression models for survival analysis. Concepts of degrees of freedom and corresponding Akaike or Bayesian information criteria, particularly useful for regularization and variable selection in high-dimensional covariate spaces, are discussed as well. The practical aspects of boosting procedures for fitting statistical models are illustrated by means of the dedicated open-source software package mboost. This package implements functions which can be used for model fitting, prediction and variable selection. It is flexible, allowing for the implementation of new boosting algorithms optimizing user-specified loss functions. <s> BIB004 </s> Machine Learning for Survival Analysis: A Survey <s> Ensemble Learning. Ensemble learning methods <s> Minimal depth is a dimensionless order statistic that measures the predictiveness of a variable in a survival tree. It can be used to select variables in high-dimensional problems using Random Survival Forests (RSF), a new extension of Breiman's Random Forests (RF) to survival settings. We review this methodology and demonstrate its use in high-dimensional survival problems using a public domain R-language package randomSurvivalForest . We discuss effective ways to regularize forests and discuss how to properly tune the RF parameters ‘nodesize’ and ‘mtry’. We also introduce new graphical ways of using minimal depth for exploring variable relationships. © 2011 Wiley Periodicals, Inc. Statistical Analysis and Data Mining 4: 115–132 2011 © 2011 Wiley Periodicals, Inc. <s> BIB005
Bagging Survival Trees: Bagging is one of the oldest and most commonly used ensemble methods, typically reducing the variance of the base models that are used. In bagging survival trees, the aggregated survival function can be calculated by averaging the predictions made by a single survival tree instead of taking a majority vote BIB001 ). There are three main steps in this method: (i) draw B booststrap samples from the given data; (ii) for each bootstrap sample, build a survival tree and ensure that for all the terminal nodes, the number of events is greater than or equal to the threshold d; and (iii) by averaging the leaf nodes' predictions, calculate the bootstrap aggregated survival function. For each leaf node, the survival function is estimated using the KM estimator, and all the individuals within the same node are assumed to have the same survival function. Random Survival Forests: The random forest is an ensemble method specifically developed to make predictions using tree structured models BIB002 . It is based on a framework similar to bagging, with the main difference between them being that at a certain node, rather than using all the attributes, the random forest uses only a random subset of the residual attributes to select the attributes based on the splitting criterion. This randomization has been shown to reduce the correlation among the trees and thus improve the prediction performance. The Random Survival Forest (RSF) BIB003 ) extended Breiman's random forest method by using a forest of survival trees for prediction. Here there are four main steps: (i) draw B bootstrap samples randomly from the given dataset. This is also referred to as Out-Of-Bag (OOB) data because around 37% of the data is excluded in each sample. (ii) For each sample, build a survival tree by randomly selecting features and then split the node using the candidate features to maximize the survival difference between the child nodes. (iii) Build the full-size tree based on the constraint that the terminal node must be greater than or equal to a specific unique death. And (iv) using the non-parametric NA estimator, calculate the ensemble Cumulative Hazard Function (CHF) of the OOB data by taking the average of the CHF of each tree. BIB005 also provides an effective way to apply RSF for high-dimensional survival analysis problems by regularizing forests. Boosting: The boosting algorithm is one of the most widely used ensemble methods and is designed to combine base learners into a weighted sum that represents the final output of a strong learner. This algorithm iteratively fits a set of appropriately defined residuals based on the gradient descent algorithm (Hothorn et al. 2006; BIB004 . Hothorn et al. (2006) extended the gradient boosting algorithm to minimize the weighted risk functionβŨ ,X = β is a vector of parameters; and h(·|β U ,X ) is the prediction made by regressing U using a base learner. The steps taken to optimize this problem are as follows: (i) initializeŨ i = y i (i = 1, . . . , N ) , m = 0 andf 0 (·|βŨ ,X ); fix the number of iterations M (M > 1); (ii) fit h(·|β U ,X ) after updating the residualsŨ i (i = 1, . . . , N ); (iii) iteratively updatef m+1 (·) =f m (·) + vh(·|β U ,X ), where 0 < v ≤ 1 represents the step size; and (iv) repeat the procedures in steps (ii) and (iii) until m = M.