If $y = W^T x$, what is $\frac{\partial y}{\partial W}$? I would like to derive the derivative of a vector by matrix, i.e. $y = W^Tx$, where $W$ is a matrix, $x,y$ are vectors. What is $\frac{\partial y}{\partial W} = \frac{\partial W^T x}{\partial W}$?

Follow-up:
Define another function $z = a^T y = a^T W^Tx$, so that $z$ is a scalar. We know that $\frac{\partial z}{\partial W}$ is a matrix with the same of $W$. At the same time, by chain rule
\begin{equation}
\frac{\partial z}{\partial W} = \frac{\partial z}{\partial y} \cdot \frac{\partial y}{\partial W} 
\end{equation}
where $\frac{\partial z}{\partial y}$ is a $1\times N$ vector ($N$ is the dimension of $y$). So it seems the dimensions of matrices in left and right hand side don't match. Any explanations?

Let $f : \mathbb R^{m \times n} \to \mathbb R^n$ be defined by
$$f (\mathrm X) := \mathrm X^T \mathrm a$$
The $i$-th entry of $f$ is the scalar
$$f_i (\mathrm X) = \mathrm e_i^T \mathrm X^T \mathrm a = \mathrm a^T \mathrm X \mathrm e_i = \mbox{tr} (\mathrm e_i \mathrm a^T \mathrm X) = \mbox{tr} ((\mathrm a \mathrm e_i^T)^T \mathrm X) = \langle \mathrm a \mathrm e_i^T, \mathrm X \rangle$$
Hence, the derivative of $f_i$ with respect to $\mathrm X$ is the $m \times n$ matrix
$$\partial_{\mathrm X} f_i (\mathrm X) = \mathrm a \mathrm e_i^T$$
