Generalized singular value decomposition

In linear algebra, the generalized singular value decomposition (GSVD) is the name of two different techniques based on the singular value decomposition. The two versions differ because one version decomposes two (or more) matrices (much like higher order PCA) and the other version uses a set of constraints imposed on the left and right singular vectors.
Higher order version.
The generalized singular value decomposition (GSVD) is a matrix decomposition on a pair of matrices which generalizes the singular value decomposition. It was introduced by Van Loan in 1976 and later developed by Paige and Saunders, which is the version described here. In contrast to the SVD, the GSVD decomposes simultaneously a pair of matrices with the same number of columns. The SVD and the GSVD, as well as some other possible generalizations of the SVD, are extensively used in the study of the conditioning and regularization of linear systems with respect to quadratic semi-norms. In the following, let formula_1, or formula_2. 
Definition.
The generalized singular value decomposition of matrices formula_3 and formula_4 isformula_5where
We denote formula_27, formula_28, formula_29, and formula_30. While formula_31 is diagonal, formula_32 is not always diagonal, because of the leading rectangular zero matrix; instead formula_33 is "bottom-right-diagonal". 
Variations.
There are many variations of the GSVD. These variations are related to the fact that it is always possible to multiply formula_34 from the left by formula_35 where formula_36 is an arbitrary unitary matrix. We denote 
Here are some variations of the GSVD: 
Generalized singular values.
A "generalized singular value" of formula_46 and formula_47 is a pair formula_48 such that 
formula_49We have 
By these properties we can show that the generalized singular values are exactly the pairs formula_52. We haveformula_53Therefore
This expression is zero exactly when formula_55 and formula_56 for some formula_57.
In, the generalized singular values are claimed to be those which solve formula_58. However, this claim only holds when formula_59, since otherwise the determinant is zero for every pair formula_48; this can be seen by substituting formula_61 above.
Generalized inverse.
Define formula_62 for any invertible matrix formula_36 , formula_64 for any zero matrix formula_65, and formula_66 for any block-diagonal matrix. Then defineformula_67It can be shown that formula_68 as defined here is a generalized inverse of formula_69; in particular a formula_70-inverse of formula_69. Since it does not in general satisfy formula_72, this is not the Moore–Penrose inverse; otherwise we could derive formula_73 for any choice of matrices, which only holds for certain class of matrices. 
Suppose formula_74, where formula_75 and formula_76. This generalized inverse has the following properties:
Quotient SVD.
A "generalized singular ratio" of formula_46 and formula_47 is formula_87. By the above properties, formula_88. Note that formula_81 is diagonal, and that, ignoring the leading zeros, contains the singular ratios in decreasing order. If formula_47 is invertible, then formula_91 has no leading zeros, and the generalized singular ratios are the singular values, and formula_92 and formula_93 are the matrices of singular vectors, of the matrix formula_94. In fact, computing the SVD of formula_95 is one of the motivations for the GSVD, as "forming formula_96 and finding its SVD can lead to unnecessary and large numerical errors when formula_97 is ill-conditioned for solution of equations". Hence the sometimes used name "quotient SVD", although this is not the only reason for using GSVD. If formula_47 is not invertible, then formula_99is still the SVD of formula_100 if we relax the requirement of having the singular values in decreasing order. Alternatively, a decreasing order SVD can be found by moving the leading zeros to the back: formula_101, where formula_102 and formula_103 are appropriate permutation matrices. Since rank equals the number of non-zero singular values, formula_104.
Construction.
Let 
Thenformula_124We also haveformula_125Thereforeformula_126Since formula_127 has orthonormal columns, formula_128. Thereforeformula_129We also have for each formula_130 such that formula_131 thatformula_132Therefore formula_133, andformula_134
Weighted version.
The weighted version of the generalized singular value decomposition (GSVD) is a constrained matrix decomposition with constraints imposed on the left and right singular vectors of the singular value decomposition. This form of the "GSVD" is an extension of the "SVD" as such. Given the "SVD" of an "m×n" real or complex matrix "M"
where
Where "I" is the identity matrix and where formula_137 and formula_138 are orthonormal given their constraints (formula_139 and formula_140). Additionally, formula_139 and formula_140 are positive definite matrices (often diagonal matrices of weights). This form of the "GSVD" is the core of certain techniques, such as generalized principal component analysis and Correspondence analysis.
The weighted form of the "GSVD" is called as such because, with the correct selection of weights, it "generalizes" many techniques (such as multidimensional scaling and linear discriminant analysis)
Applications.
The GSVD, formulated as a comparative spectral decomposition, has been successfully applied to signal processing and data science, e.g., in genomic signal processing.
These applications inspired several additional comparative spectral decompositions, i.e., the higher-order GSVD (HO GSVD) and the tensor GSVD. 