Partial Derivatives involving Vectors and Matrices Let $Y$ be a $(N \times 1)$ vector, $X$ be a $N \times M$ matrix and $\lambda$ be a $M \times 1$ vector. 
I am wondering how I can evaluate the following partial derivative.
\begin{align}
\frac{\partial (Y-X\lambda)^T (Y-X\lambda)}{\partial \lambda_j}
\end{align}
where $j = 1 \ldots P$
I run into such problems fairly often and would very much appreciate it if anyone could post a guide on differentiate expressions involving vectors and matrices (and in particular transposes / products of matrices).

See the entry on Matrix Calculus in Wikipedia, or search for "matrix calculus" on the internet. In your particular case,
$$
\frac{\partial (Y-X\lambda)^T (Y-X\lambda)}{\partial\lambda^T}=-2(Y-X\lambda)^T X
$$
and hence the partial derivative w.r.t. $\lambda_j$ is the $j$-th entry of the row vector $-2(Y-X\lambda)^T X$.
