Machine Learning for Survival Analysis: A Survey <s> Regularized Cox models <s> In a Cox regression model, instability of the estimated regression coefficients can be reduced by maximizing a penalized partial log-likelihood, where a penalty function of the regression coefficients is substracted from the partial log-likelihood. In this paper, we choose the optimal weight of the penalty function by maximizing the predictive value of the model, as measured by the crossvalidated partial log-likelihood. Our methods are illustrated by a study of ovarian cancer survival and by a study of centre effects in kidney graft survival. <s> BIB001 </s> Machine Learning for Survival Analysis: A Survey <s> Regularized Cox models <s> In multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory, if not incorrect, if the prediction vectors are not orthogonal. Proposed is an estimation procedure based on adding small positive quantities to the diagonal of X′X. Introduced is the ridge trace, a method for showing in two dimensions the effects of nonorthogonality. It is then shown how to augment X′X to obtain biased estimates with smaller mean square error. <s> BIB002 </s> Machine Learning for Survival Analysis: A Survey <s> Regularized Cox models <s> Summary. The lasso penalizes a least squares regression by the sum of the absolute values (L1-norm) of the coefficients. The form of this penalty encourages sparse solutions (with many coefficients equal to 0). We propose the ‘fused lasso’, a generalization that is designed for problems with features that can be ordered in some meaningful way. The fused lasso penalizes the L1-norm of both the coefficients and their successive differences. Thus it encourages sparsity of the coefficients and also sparsity of their differences—i.e. local constancy of the coefficient profile. The fused lasso is especially useful when the number of features p is much greater than N, the sample size. The technique is also extended to the ‘hinge’ loss function that underlies the support vector classifier.We illustrate the methods on examples from protein mass spectroscopy and gene expression data. <s> BIB003 </s> Machine Learning for Survival Analysis: A Survey <s> Regularized Cox models <s> We missed an important reference in Section 3.4. In page 309 we stated that ‘. . . which is based on the recently proposed algorithm LARS of Efron et al. (2004). They proved that, starting from zero, the lasso solution paths grow piecewise linearly in a predictable way. They proposed a new algorithm called LARS to solve the entire lasso solution path efficiently by using the same order of computations as a single OLS fit. . . .’ The following sentence should have been included. The piecewise linearity of the lasso solution path was first proved by Osborne et al. (2000), who also described an efficient algorithm for calculating the complete lasso solution path. Reference Osborne, M. R., Presnell, B. and Turlach, B. A. (2000) A new approach to variable selection in least squares problems. IMA J. Numer. Anal., 20, 389–403. <s> BIB004 </s> Machine Learning for Survival Analysis: A Survey <s> Regularized Cox models <s> SUMMARY We consider the problem of estimating sparse graphs by a lasso penalty applied to the inverse covariance matrix. Using a coordinate descent procedure for the lasso, we develop a simple algorithm—the graphical lasso—that is remarkably fast: It solves a 1000-node problem (∼500 000 parameters) in at most a minute and is 30–4000 times faster than competing methods. It also provides a conceptual link between the exact problem and the approximation suggested by Meinshausen and B¨ uhlmann (2006). We illustrate the method on some cell-signaling data from proteomics. <s> BIB005 </s> Machine Learning for Survival Analysis: A Survey <s> Regularized Cox models <s> We introduce a pathwise algorithm for the Cox proportional hazards model, regularized by convex combinations of l1 and l2 penalties (elastic net). Our algorithm fits via cyclical coordinate descent, and employs warm starts to find a solution along a regularization path. We demonstrate the efficacy of our algorithm on real and simulated data sets, and find considerable speedup between our algorithm and competing methods. <s> BIB006 </s> Machine Learning for Survival Analysis: A Survey <s> Regularized Cox models <s> Survival Regression models play a vital role in analyzing time-to-event data in many practical applications ranging from engineering to economics to healthcare. These models are ideal for prediction in complex data problems where the response is a time-to-event variable. An event is defined as the occurrence of a specific event of interest such as a chronic health condition. Cox regression is one of the most popular survival regression model used in such applications. However, these models have the tendency to over fit the data which is not desirable for healthcare applications because it limits their generalization to other hospital scenarios. In this paper, we address these challenges for the cox regression model. We combine two unique correlation based regularizers with cox regression to handle correlated and grouped features which are commonly seen in many practical problems. The proposed optimization problems are solved efficiently using cyclic coordinate descent and Alternate Direction Method of Multipliers algorithms. We conduct experimental analysis on the performance of these algorithms over several synthetic datasets and electronic health records (EHR) data about heart failure diagnosed patients from a hospital. We demonstrate through our experiments that these regularizers effectively enhance the ability of cox regression to handle correlated features. In addition, we extensively compare our results with other regularized linear and logistic regression algorithms. We validate the goodness of the features selected by these regularized cox regression models using the biomedical literature and different feature selection algorithms. <s> BIB007
Regularizers There are also several useful extensions of the Lasso-Cox method. The adaptive Lasso-Cox (Zhang and Lu 2007) is based on a penalized partial likelihood with adaptively weighted 1 penalties λ P j=1 τ j |β j | on the regression coefficients, with small weights τ j assigned for large coefficients and large weights for small coefficients. In the fused Lasso-Cox BIB003 , the coefficients and their successive differences are penalized using the 1 -norm, while in the graphical Lasso-Cox BIB005 , the sparse graphs are estimated using the coordinate descent method by applying an 1 -penalty to the inverse covariance matrix. These extensions then solve survival problems in a similar way to the regular Lasso-Cox model, differing only in incorporating these different 1 penalties. Ridge-Cox: The Ridge regression was originally proposed by BIB002 and was successfully used in the context of Cox regression by Verweij et al. BIB001 . It incorporates an 2 -norm regularizer to select correlated features and shrink their values toward each other. The feature-based regularized Cox method (FEAR-Cox) (Vinzamuri and Reddy 2013) uses a feature-based non-negative valued regularizer R(β ) = |β | T M |β | for the modified least squares formulation of the Cox regression; the cyclic coordinate descent method is used to solve this optimization problem, where M ∈ R P ×P (P is the number of features) is a positive semi-definite matrix. Ridge-Cox is a special case of FEAR-Cox when M is the identity matrix. EN-Cox: Elastic Net (EN), which combines the 1 and squared 2 penalties, has the potential to perform feature selection and deal with the correlation between the features simultaneously BIB004 . The EN-Cox method was proposed by Simon et al. BIB006 , using the EN penalty term shown in Table 3 with 0 ≤ α ≤ 1 and introducing the log-partial likelihood function in Equation (14). Unlike Lasso-Cox, EN-Cox can select more than N features if N ≤ P. The Kernel Elastic Net (KEN) algorithm BIB007 , which as the name suggests uses the concept of kernels, compensates for the drawbacks of the EN-Cox, which is only partially effective at dealing with the correlated features in survival data. KEN-Cox it builds a kernel similarity matrix for the feature space in order to incorporate pairwise feature similarity into the Cox model. The regularizer used in KEN-Cox is defined as . . , P ) as its entries. Note that the equation for the KEN-Cox method includes both smooth and non-smooth 1 terms.