How to differentiate product of vectors (that gives scalar) by vector? I'm trying to understand derivation of the least squares method in matrices terms:
$$S(\beta) = y^Ty - 2 \beta X^Ty + \beta ^ T X^TX \beta$$
Where $\beta$ is $m \times 1$ vertical vector, $X$ is $n \times m$ matrix and $y$ is $n \times 1$ vector.
The question is: why $$\frac{d(2\beta X^Ty)}{d \beta} = 2X^Ty$$
I tried to derive it directly via definition of derivative:
 $$\frac{d(2\beta X^Ty)}{d \beta} = \lim_{\Delta \beta \to 0} \frac{2\Delta\beta X^T y}{\Delta \beta} = \lim_{\Delta \beta \to 0}   2\Delta\beta X^T y \cdot \Delta \beta^{-1}$$
May be the last equality must be as in the next line, but anyway I don't understand why $$2\Delta\beta \Delta \beta^{-1} X^T y  $$And, what is $\Delta \beta^{-1}$? Vectors don't have the inverse form.
The same questions I have to this quasion:
$$(\beta ^ T X^TX \beta)' =2 X^T X \beta$$

Let scalar field $f : \mathbb{R}^n \to \mathbb{R}$ be defined by
$$f (\mathrm x) = \mathrm a^T \mathrm x = a_1 x_1 + a_2 x_2 + \cdots + a_n x_n$$
Taking the $n$ partial derivatives,
$$\partial_1 f (\mathrm x) = a_1 \qquad \qquad \partial_2 f (\mathrm x) = a_2 \qquad \dots \qquad\partial_n f (\mathrm x) = a_n, \qquad$$
Hence, the gradient of $f$ is
$$\nabla f (\mathrm x) = (a_1, a_2, \dots, a_n) = \mathrm a$$
