Angles between flats

The concept of angles between lines in the plane and between pairs of two lines, two planes or a line and a plane in space can be generalized to arbitrary dimension. This generalization was first discussed by Jordan. For any pair of flats in a Euclidean space of arbitrary dimension one can define a set of mutual angles which are invariant under isometric transformation of the Euclidean space. If the flats do not intersect, their shortest distance is one more invariant. These angles are called canonical or principal. The concept of angles can be generalized to pairs of flats in a finite-dimensional inner product space over the complex numbers.
Jordan's definition.
Let formula_1 and formula_2 be flats of dimensions formula_3 and formula_4 in the formula_5-dimensional Euclidean space formula_6. By definition, a translation of formula_1 or formula_8 does not alter their mutual angles. If formula_1 and formula_2 do not intersect, they will do so upon any translation of formula_2 which maps some point in formula_2 to some point in formula_1. It can therefore be assumed without loss of generality that formula_1 and formula_2 intersect.
Jordan shows that Cartesian coordinates formula_16 formula_17 formula_18 formula_19 formula_20 formula_21 in formula_6 can then be defined such that formula_1 and formula_2 are described, respectively, by the sets of equations
and
with formula_31. Jordan calls these coordinates canonical. By definition, the angles formula_32 are the angles between formula_1 and formula_2.
The non-negative integers formula_35 are constrained by
For these equations to determine the five non-negative integers completely, besides the dimensions formula_39 and formula_40 and the number formula_41 of angles formula_32, the non-negative integer formula_43 must be given. This is the number of coordinates formula_44, whose corresponding axes are those lying entirely within both formula_1 and formula_2. The integer formula_43 is thus the dimension of formula_48. The set of angles formula_32 may be supplemented with formula_43 angles formula_51 to indicate that formula_48 has that dimension.
Jordan's proof applies essentially unaltered when formula_6 is replaced with the formula_5-dimensional inner product space formula_55 over the complex numbers. (For angles between subspaces, the generalization to formula_55 is discussed by Galántai and Hegedũs in terms of the below variational characterization.)
Angles between subspaces.
Now let formula_1 and formula_2 be subspaces of the formula_5-dimensional inner product space over the real or complex numbers. Geometrically, formula_1 and formula_2 are flats, so Jordan's definition of mutual angles applies. When for any canonical coordinate formula_62 the symbol formula_63 denotes the unit vector of the formula_62 axis, the vectors formula_65 formula_66 formula_67 form an orthonormal basis for formula_1 and the vectors formula_65 formula_70 formula_71 form an orthonormal basis for formula_2, where
Being related to canonical coordinates, these basic vectors may be called canonical.
When formula_74 denote the canonical basic vectors for formula_1 and formula_76 the canonical basic vectors for formula_2 then the inner product formula_78 vanishes for any pair of formula_79 and formula_80 except the following ones.
With the above ordering of the basic vectors, the matrix of the inner products formula_78 is thus diagonal. In other words, if formula_83 and formula_84 are arbitrary orthonormal bases in formula_1 and formula_2 then the real, orthogonal or unitary transformations from the basis formula_87 to the basis formula_88 and from the basis formula_89 to the basis formula_90 realize a singular value decomposition of the matrix of inner products formula_91. The diagonal matrix elements formula_92 are the singular values of the latter matrix. By the uniqueness of the singular value decomposition, the vectors formula_93 are then unique up to a real, orthogonal or unitary transformation among them, and the vectors formula_94 and formula_95 (and hence formula_96) are unique up to equal real, orthogonal or unitary transformations applied simultaneously to the sets of the vectors formula_94 associated with a common value of formula_32 and to the corresponding sets of vectors formula_95 (and hence to the corresponding sets of formula_96).
A singular value formula_101 can be interpreted as formula_102 corresponding to the angles formula_51 introduced above and associated with formula_48 and a singular value formula_51 can be interpreted as formula_106 corresponding to right angles between the orthogonal spaces formula_107 and formula_108, where superscript formula_109 denotes the orthogonal complement.
Variational characterization.
The variational characterization of singular values and vectors implies as a special case a variational characterization of the angles between subspaces and their associated canonical vectors. This characterization includes the angles formula_51 and formula_111 introduced above and orders the angles by increasing value. It can be given the form of the below alternative definition. In this context, it is customary to talk of principal angles and vectors.
Definition.
Let formula_112 be an inner product space. Given two subspaces formula_113 with formula_114, there exists then a sequence of formula_3 angles formula_116 called the principal angles, the first one defined as
where formula_118 is the inner product and formula_119 the induced norm. The vectors formula_120 and formula_121 are the corresponding "principal vectors."
The other principal angles and vectors are then defined recursively via
This means that the principal angles formula_123 form a set of minimized angles between the two subspaces, and the principal vectors in each subspace are orthogonal to each other.
Examples.
Geometric example.
Geometrically, subspaces are flats (points, lines, planes etc.) that include the origin, thus any two subspaces intersect at least in the origin. Two two-dimensional subspaces formula_124 and formula_125 generate a set of two angles. In a three-dimensional Euclidean space, the subspaces formula_124 and formula_125 are either identical, or their intersection forms a line. In the former case, both formula_128. In the latter case, only formula_129, where vectors formula_120 and formula_121 are on the line of the intersection formula_132 and have the same direction. The angle formula_133 will be the angle between the subspaces formula_124 and formula_125 in the orthogonal complement to formula_132. Imagining the angle between two planes in 3D, one intuitively thinks of the largest angle, formula_133.
Algebraic example.
In 4-dimensional real coordinate space R4, let the two-dimensional subspace formula_124 be 
spanned by formula_139 and formula_140, and let the two-dimensional subspace formula_125 be 
spanned by formula_142 and formula_143 with some real formula_144 and formula_145 such that formula_146. Then formula_120 and formula_121 are, in fact, the pair of principal vectors corresponding to the angle formula_149 with formula_150, and formula_151 and formula_152 are the principal vectors corresponding to the angle formula_153 with formula_154
To construct a pair of subspaces with any given set of formula_3 angles formula_156 in a formula_157 (or larger) dimensional Euclidean space, take a subspace formula_124 with an orthonormal basis formula_159 and complete it to an orthonormal basis formula_160 of the Euclidean space, where formula_161. Then, an orthonormal basis of the other subspace formula_125 is, e.g.,
Extensions.
The notion of the angles and some of the variational properties can be naturally extended to arbitrary inner products and subspaces with infinite dimensions.
Computation.
Historically, the principal angles and vectors first appear in the context of canonical correlation and were originally computed using SVD of corresponding covariance matrices. However, as first noticed in, the canonical correlation is related to the cosine of the principal angles, which is ill-conditioned for small angles, leading to very inaccurate computation of highly correlated principal vectors in finite precision computer arithmetic. The sine-based algorithm fixes this issue, but creates a new problem of very inaccurate computation of highly uncorrelated principal vectors, since the sine function is ill-conditioned for angles close to /2. To produce accurate principal vectors in computer arithmetic for the full range of the principal angles, the combined technique first compute all principal angles and vectors using the classical cosine-based approach, and then recomputes the principal angles smaller than /4 and the corresponding principal vectors using the sine-based approach. The combined technique is implemented in open-source libraries Octave and SciPy and contributed and 
 to MATLAB.