A question regarding a step in power method justification (Writing a vector in terms of the eigenvectors of a matrix) Let $A$ be a $t \times t$ matrix. Can we present any $t \times 1$ vector, as a linear combination of eigenvectors of $A$?
I think this should not be the case unless all eigenvectors of $A$ happened to be linearly independent. (right?) 
But it seems to be used in the proof of power method below. 
Edit:
That is how the argument works: Suppose $x_0$ is an initial vector. Write it in terms of eigenvectors of $A$, i.e. $x_0 = \sum_i c_i v_i$ where $v_i$ is the $i^{th}$ eigenvector and $\lambda_i$ the corresponding eigenvalue. Let $x(n) = A^n x_0$.  We want to find $\lim_{n \to \infty} x(n).$ Based on the first assumption we have $x(n) = A^n \sum_i  c_iv_i= \sum_i c_i \lambda_i ^n v_i= \lambda_1^n \sum_i (\lambda_i/\lambda_1)^n v_i$  where $(\lambda_1, v_1)$ is the leading eigenpair.  Since $\lambda_i/\lambda_1 <1, \forall i\neq1$, all terms in the sum decay exponentially as $n$ becomes large and hence in the limit we get $x(n) = c_1 \lambda_1^n v_1$. In other words, $x= \lim_{n \to \infty} x(n)$ is simply proportional to the leading eigenvector of the matrix.  Equivalently,  $Ax = \lambda_1 x$.

As @oldrinb already said, eigenvectors corresponding to distinct eigenvalues are always linearly independent.
Next, a matrix has a basis of eigenvectors if and only if it's diagonalisable, which is not always the case, consider $$\begin{pmatrix}1&1\\0&1\end{pmatrix}.$$
This matrix has only one eigenvector (up to a constant factor) $\begin{pmatrix}1\\0\end{pmatrix}$.
Another issue arises when you want to have diagonalisability over $\Bbb R$, not $\Bbb C$. For example, a matrix $$\begin{pmatrix}0&1\\-1&0\end{pmatrix}$$ is diagonalisable over $\Bbb C$ and not diagonalisabe over $\Bbb R$.
So finally, any vector is a linear combination of eigenvectos $\iff$ there is a basis of eigenvectors $\iff$ the matrix is diagonalisable.
Your illustration with power method is not really relevant. It makes many assumptions on the matrix (in most cases we ask it to have real eigenvalues and the eigenvalue of the biggest absolute value to be of multiplicity one).
