In semidefinite programming (SDP), a number of pre-processing techniques have been developed including chordal-completion procedures, which reduce the dimension of individual constraints by exploiting sparsity therein, and facial reduction, which reduces the dimension of the problem by removing redundant rows and columns. This paper suggest that these work in a complementary manner and that facial reduction should be used after chordal-completion procedures. In computational experiments on SDP instances from the SDPLib, a benchmark, and structured instances from polynomial and binary quadratic optimisation, we show that such two-step pre-processing with a standard interior-point method outperforms the interior point method, with or without the traditional pre-processing.
There is also a long history of work on facial reduction @cite_35 @cite_26 @cite_22 @cite_54 @cite_11 @cite_46 @cite_52 , including some of the most elegant papers in optimisation @cite_35 . The conic expansion method @cite_40 is in some sense the dual of facial reduction. Facial reduction has been used to pre-process @cite_53 @cite_27 degenerate semidefinite programs. Its applications to machine learning have been limited @cite_29 @cite_57 @cite_14 @cite_1 so far. For a quick overview of facial reduction and its relationship to degeneracy, we refer to @cite_15 .
@cite_35 @cite_26 @cite_14 @cite_22 @cite_15 @cite_54 @cite_53 @cite_29 @cite_52 @cite_1 @cite_57 @cite_40 @cite_27 @cite_46 @cite_11
2615873886 2794029565 2950973664 2097728881
Recent breakthroughs have been made in the use of semi-definite programming and its application to real polynomial solving. For example, the real radical of a zero dimensional ideal, can be determined by such approaches. Some progress has been made on the determination of the real radical in positive dimension by Ma, Wang and Zhi[5, 4]. Such work involves the determination of maximal rank semidefinite matrices. Existing methods are computationally expensive and have poorer accuracy on larger examples. In previous work we showed that regularity in the form of the Slater constraint qualification (strict feasibility) fails for the moment matrix in the SDP feasibility problem[6]. We used facial reduction to obtain a smaller regularized problem for which strict feasibility holds. However we did not have a theoretical guarantee that our methods, based on facial reduction and Douglas-Rachford iteration ensured the satisfaction of the maximum rank condition. Our work is motivated by the problems above. We discuss how to compute the moment matrix and its kernel using facial reduction techniques where the maximum rank property can be guaranteed by solving the dual problem. The facial reduction algorithms on the primal form is presented. We give examples that exhibit for the first time additional facial reductions beyond the first which are effective in practice with much better accuracy than SeDuMi(CVX). Conic optimization, or cone programming, is a subfield of convex optimization that includes linear, second-order cone, and semidefinite programming as special cases. While conic optimization problems arise in a diverse set of fields (including machine learning, robotics, and finance), efficiently solving them remains an active area of research. Developing methods that detect and exploit useful structure-such as symmetry, sparsity, or degeneracy-is one research topic. Such methods include facial and symmetry reduction, which have been successful in several applications, often reducing solve time by orders of magnitude. Nevertheless, theoretical and practical barriers preclude their general purpose use: to our knowledge, no solver uses facial or symmetry reduction as an automatic preprocessing step. This thesis addresses some of these barriers in three parts: the first develops more practical facial reduction techniques, the second proposes a more powerful and computationally efficient generalization of symmetry reduction (which we call Jordan reduction), and the third specializes techniques to convex relaxations of polynomial optimization problems. Throughout, we place emphasis on semidefinite programs and, more generally, optimization problems over symmetric cones. We also present computational results. Modeling the long-term facial aging process is extremely challenging due to the presence of large and non-linear variations during the face development stages. In order to efficiently address the problem, this work first decomposes the aging process into multiple short-term stages. Then, a novel generative probabilistic model, named Temporal Non-Volume Preserving (TNVP) transformation, is presented to model the facial aging process at each stage. Unlike Generative Adversarial Networks (GANs), which requires an empirical balance threshold, and Restricted Boltzmann Machines (RBM), an intractable model, our proposed TNVP approach guarantees a tractable density function, exact inference and evaluation for embedding the feature transformations between faces in consecutive stages. Our model shows its advantages not only in capturing the non-linear age related variance in each stage but also producing a smooth synthesis in age progression across faces. Our approach can model any face in the wild provided with only four basic landmark points. Moreover, the structure can be transformed into a deep convolutional network while keeping the advantages of probabilistic models with tractable log-likelihood density estimation. Our method is evaluated in both terms of synthesizing age-progressed faces and cross-age face verification and consistently shows the state-of-the-art results in various face aging databases, i.e. FG-NET, MORPH, AginG Faces in the Wild (AGFW), and Cross-Age Celebrity Dataset (CACD). A large-scale face verification on Megaface challenge 1 is also performed to further show the advantages of our proposed approach. We address the pose mismatch problem which can occur in face verification systems that have only a single (frontal) face image available for training. In the framework of a Bayesian classifier based on mixtures of gaussians, the problem is tackled through extending each frontal face model with artificially synthesized models for non-frontal views. The synthesis methods are based on several implementations of maximum likelihood linear regression (MLLR), as well as standard multi-variate linear regression (LinReg). All synthesis techniques rely on prior information and learn how face models for the frontal view are related to face models for non-frontal views. The synthesis and extension approach is evaluated by applying it to two face verification systems: a holistic system (based on PCA-derived features) and a local feature system (based on DCT-derived features). Experiments on the FERET database suggest that for the holistic system, the LinReg-based technique is more suited than the MLLR-based techniques; for the local feature system, the results show that synthesis via a new MLLR implementation obtains better performance than synthesis based on traditional MLLR. The results further suggest that extending frontal models considerably reduces errors. It is also shown that the local feature system is less affected by view changes than the holistic system; this can be attributed to the parts based representation of the face, and, due to the classifier based on mixtures of gaussians, the lack of constraints on spatial relations between the face parts, allowing for deformations and movements of face areas.