Q:

Physical meaning of norm of a matrix

I know norm of a vector is a length of a vector from origin. So what is the motivation behind defining the norm of the matrix? What is the physical meaning of norm of a matrix?
Any help is appreciated.
Thanks.

A:

There are actually multiple ways to assign a norm to a matrix, in fact there are multiple ways to give a norm to a vector. With vectors in $\mathbb{R}^n$ the choice that is most "geometrically appealing" is the Euclidean one
$$\|(x_1,...,x_n)\| = \sqrt{x_1^2 + ... + x_n^2}$$
However, there are others, I'd advise looking up $\ell^p$-norms. 
One "obvious" choice for a matrix norm is simply to do a Euclidean norm by summing the squares of the entries and square-rooting that. But the algebra of the situation actually suggests something a little more interesting. That is, let $A$ be a matrix and $x$ a vector of appropriate dimensions, and let $\|x\|$ denote the Euclidean vector norm. Then we give the matrix the "operator norm"
$$\|A\| = \max\limits_{x \in \mathbb{R}^n}\frac{\|Ax\|}{\|x\|}$$
Which represents the max that the matrix $A$ stretches the vector $x$ in some sense. We choose the max so that the norm is positive definite. If $A$ sends any non-zero vector to a nonzero vector (that is, $A$ is nonzero) then $\|A\| > 0$.

A:

Take a real matrix $A \in \mathbb{R}^{n \times m}$ for example. Now let this matrix represent an input-output relation like $y = Au$. The $p$-norm of $A$ is $\sup\limits_{||u||_p \neq 0}\frac{||y||_p}{||u||_p}$ where $||v||_p$ denotes the $p$-norm of vector $v$. In other words, it is a measure of amplification of the input.

