Kernel learning by unconstrained optimization
We study the problem of learning a kernel matrix from an apriori kernel and training data. An unconstrained convex optimization formulation is proposed, with an arbitrary convex smooth loss function on kernel entries and a LogDet divergence for regularization. Since the number of variables is of order O(n), standard Newton and quasi-Newton methods are too time-consuming. An operator form Hessian is used to develop an O(n) trust-region inexact Newton method, where the Newton direction is computed using several conjugate gradient steps on the Hessian operator equation. On the uspst dataset, our algorithm can handle 2 million optimization variables within one hour. Experiments are shown for both linear (Mahalanobis) metric learning and for kernel learning. The convergence rate, speed and performance of several loss functions and algorithms are discussed.
This paper presents a new robust method to estimate the parameters of ARMA models. This method makes use of the autocorrelations estimates based on the ratio of medians together with a robust filter cleaner able to reject a large fraction of outliers, and a Gaussian maximum likelihood estimation which handles missing values. The main advantages of the procedure are its easiness, robustness and fast execution. Its effectiveness is demonstrated on an example of the forecasting of the French daily electricity consumptions.