Q:

Norm with symmetric positive definite matrix

If B is $n \times n$ real symmetric positive definite matrix, then $(x,y) = x^T By$ defines an inner product on $R^n$. How do you prove that $\|x\|=(x^T B x)^{1/2}$ is a norm on $R^n$?

A:

Any real inner product $(\cdot, \cdot)$ on any real vector space $V$, of finite dimension or not, defines a norm via $\Vert x \Vert^2 = (x, x)$ or $\Vert x \Vert = (x, x)^{1/2}$; the defining properties of the norm follow directly from the inner product axioms, which as I recall may be stated as:
1.)  $(x_1 + x_2, y_1) = (x_1, y_1) + (x_2, y_1)$,
$(x_1, y_1 + y_2) = (x_1, y_1) + (x_1, y_2), \;\; \forall x_1, x_2, y_1, y_2 \in V$;
2.) $(x, y) = (y, x), \;\; \forall x, y \in V$;
3.) $(ax, y) = (x, ay) = a(x, y), \; \; \forall x, y \in V, a \in \Bbb R$;
4.) $(x, x) \ge 0, \;\; \forall x \in V; \;\; (x, x) = 0 \Leftrightarrow x = 0$.
A norm $\Vert \cdot \Vert$ must obey:
A.) $\Vert x \Vert \ge 0, \;\; \forall x \in V; \; \; \Vert x \Vert = 0 \Leftrightarrow x = 0$;
B.) $\Vert ax \Vert = \vert a \vert \Vert x \Vert, \;\; \forall x \in V, a \in \Bbb R$;
C.) $\Vert x + y \Vert \le \Vert x \Vert + \Vert y \Vert, \;\; \forall x, y \in V$.
It is a straightforward matter to see that (A - C) follow from (1 - 4), provided we take $\Vert x \Vert = (x, x)^{1/2}$; indeed, we have
$\Vert x \Vert = 0 \Leftrightarrow (x, x) = \Vert x \Vert^2 =  0 \Leftrightarrow x = 0, \tag{1}$
from (4); here as elsewhere $(x, x) \ge 0$ is implicitly invoked to assure us that $\Vert x \Vert$ is well defined via $\Vert x \Vert^2 = (x, x) \ge 0$.  This establishes that $\Vert x \Vert$ satisfies (A).  As for (B), we have
$\Vert ax \Vert^2 = (ax, ax) = a^2(x, x) = \vert a \vert^2 \Vert x \Vert^2 \tag{2}$
by (3); thus
$\Vert ax \Vert = \vert a \vert \Vert x \Vert, \tag{3}$
providing verification.  As for (C),the so-called triangle inequality, it is the most complicated property of $\Vert \cdot \Vert$ to derive from (1 - 4); doing so is facilitated by making use of the Cauchy-Schwarz inequality,
$\vert (x, y) \vert \le \Vert x \Vert \Vert y \Vert, \; \; \forall x, y \in V, \tag{4}$
which itself follows from (1 - 4), though proving it here would take us too far afield.  A complete demonstration may be found in this widipedia entry.  Having Cauchy-Schwarz at hand, we see that
$\Vert x + y \Vert^2 = (x + y, x + y) = (x, x + y) + (y, x + y)$
$= (x, x) + (x, y) + (y, x) + (y, y) = \Vert x \Vert^2 + (x, y) + (y, x) + \Vert y \Vert^2 \tag{5}$
by repeated application of (1).  Furthermore,
$(x, y) \le \vert (x, y) \vert; \; \; (y, x) \le \vert (y, x) \vert, \tag{6}$
so that
$(x, y) + (y, x) \le \vert (x, y) \vert\ + \vert (y, x) \vert, \tag{7}$
whence (5) becomes
$\Vert x + y \Vert^2 = (x + y, x + y) \le \Vert x \Vert^2 + \vert(x, y) \vert + \vert (y, x) \vert + \Vert y \Vert^2; \tag{8}$
now applying Cauchy-Schwarz,
$\Vert x + y \Vert^2 \le \Vert x \Vert^2 + \Vert x \Vert \Vert y \Vert + \Vert y \Vert \Vert x \Vert + \Vert y \Vert^2$
$= \Vert x \Vert^2 + 2\Vert x \Vert \Vert y \Vert + \Vert y \Vert^2 = (\Vert x \Vert + \Vert y \Vert)^2; \tag{9}$
(9) immediately yields
$\Vert x + y \Vert \le \Vert x \Vert + \Vert y \Vert, \tag{10}$
establishing the triangle inequality for $\Vert x \Vert = (x, x)^{1/2}$.  Combining these results, we see that $\Vert x \Vert = (x, x)^{1/2}$ is in fact a norm on $V$ for any inner product $(\cdot, \cdot)$.
When $\dim V < \infty$, it is evident the map $V \times V \to \Bbb R$ taking $x, y \in V$ to $x^Ty$ is itself an inner poduct on $V$; (1 - 4) are easily verified in this case.  We are given, and it is also easy to verify from (1 - 4), that $(x, y) = x^TBy$ is an inner product on $V$ for positive definite symmetric $B$.  For example, for (2) we have
$(x, y) = x^TBy = (x^TBy)^T = y^TB^Tx = y^TBx = (y, x), \tag{11}$
in which the equality $x^TBy = (x^TB^Ty)^T$ holds since both sides are scalars, and $y^TB^Tx = y^TBx$ holds by virtue of the symmetry of $B$.  Likewise (4) follows immediately from the positive definiteness of $B$; (1) and (3) are even easier to derive.  These remarks extend to the case of $\dim V$ infinite, though it is customary to write $\langle x, y \rangle$ instead of  $x^Ty$ under such circumstances, since the notion "transpose of a matrix" is intrinsically linked to the expression of vectors and operators in some basis. But either way one views such matters, being that $(x, y) = x^TBy = \langle x, By \rangle$ is a bona fide inner product on $V$, it follows from what has been done above that $\Vert x \Vert = (x, x)^{1/2} = (x^TBx)^{1/2} = \langle x, By \rangle$ is a norm on $V$.  QED.
Notes:  In fact, when $\dim V < \infty$ there exists a positive definite symmetric matrix $C$ such that $C^2 = B$; this is seen by noting that $B$ symmetric implies it may be diagonalized by some orthogonal matrix $O$:  we have $O^TBO = \Lambda$ and thus $B = O \Lambda O^T$, with $\Lambda$ diagonal postive definite.  Then there also exists a unique positive definite diagonal matrix $\Lambda^{1/2}$ with $(\Lambda^{1/2})^2 = \Lambda$.  Setting $C = O \Lambda^{1/2} O^T$, we have $C^2 = O \Lambda^{1/2} O^T O \Lambda^{1/2} O^T = O \Lambda O^T = O \Lambda O^T = B$; also $C^T = (O \Lambda O^T)^T = O \Lambda O^T = C$.  Thus with $\Vert x \Vert^2 = x^TBx$ we have $\Vert x \Vert^2 = x^TC^2 x = x^T C^TCx = (Cx)^T(Cx)$, and in fact $(x, y) = x^TC^2y = x^T C^TC y = (Cx)^T(Cy)$.  This shows that the inner product $x^TBy$ and hence the norm $x^TBx$ may be viewed as  the ordinary inner product $x^Ty$ or norm $x^Tx$ transformed under the action of the linear map $C:V \to V$.   Expanding $x$ in an orthonormal eigenbasis $e_i$ of $B$ and/or $C$ (which exists since $B$ is symmetric) we may write $x = \sum x_i e_i$ whence $Cx = \sum x_i \lambda_i^{1/2} e_i$ and $x^TBy = (Cx)^T(Cy) = \sum \lambda_i^2 x_i y_i$; the effect of the transformation $C$ on $x^Ty$ is thus to re-scale each basis element $e_i$ by a factor $\lambda_i^{1/2}$.  Such $C$ with $C^2 = B$, $C^T = C$ likely also exist in the case $\dim V$ infinite, though something like the holomorphic functional calculus may be required to prove it.
Hope this helps.  Cheers,
and as ever,
Fiat Lux!!!

