How to differentiate product of vectors (that gives scalar) by vector? I'm trying to understand derivation of the least squares method in matrices terms:
$$S(\beta) = y^Ty - 2 \beta X^Ty + \beta ^ T X^TX \beta$$
Where $\beta$ is $m \times 1$ vertical vector, $X$ is $n \times m$ matrix and $y$ is $n \times 1$ vector.
The question is: why $$\frac{d(2\beta X^Ty)}{d \beta} = 2X^Ty$$
I tried to derive it directly via definition of derivative:
 $$\frac{d(2\beta X^Ty)}{d \beta} = \lim_{\Delta \beta \to 0} \frac{2\Delta\beta X^T y}{\Delta \beta} = \lim_{\Delta \beta \to 0}   2\Delta\beta X^T y \cdot \Delta \beta^{-1}$$
May be the last equality must be as in the next line, but anyway I don't understand why $$2\Delta\beta \Delta \beta^{-1} X^T y  $$And, what is $\Delta \beta^{-1}$? Vectors don't have the inverse form.
The same questions I have to this quasion:
$$(\beta ^ T X^TX \beta)' =2 X^T X \beta$$

What is meant by the vector derivative $\frac{dF}{d\beta}$ is the vector with components $\frac{dF}{d\beta_k}$. Then 
$$\frac{d}{d\beta_k}2\beta^T X^T y=\frac{d}{d\beta_k}\sum_{i,j}2\beta_i X_{ji} y_j=\sum_{i,j}2\delta_{ik} X_{ji} y_j=\sum_{j}2 X_{jk} y_j=(2X^T y)_k,$$ so indeed $\frac{d}{d\beta}(2\beta^T X^T y)=2 X^T y$ as desired.
