Matrix Derivative Product Rule $\frac{d}{dx} \langle x,y\rangle Ax $ Let $A\in\mathbb{R^{n\times n}}$,$x,y\in\mathbb{R^{n}}$. What is the derivative of the following expression?
$$
\frac{d}{dx} \langle x,y\rangle Ax 
$$
What I tried:
Applying the product rule doesn't work because the first term (see next equation) is inconsistent dimensionally, or I am applying it in the wrong way. Let $f(x) = \langle x,y \rangle, g(x) = Ax$, then
$$ \frac{d}{dx} f(x)g(x) = \frac{df(x)}{dx} g(x) +  f(x)\frac{dg(x)}{dx} = xAx + \langle x,y \rangle A$$.
Here the first term in the last equation doesn't make sense dimensionally, any help or suggestion would be appreciated.

Avoiding coordinates one may alternatively compute $d_pf(v)$, where $f(x)=\langle x,y\rangle Ax$.  For that purpose take a differential path $c$, defined on an interval containing $0$, satisfying $c(0)=p$ and $\dot c(0)=v$, where $\dot c(t)$ is a shorthand for $\frac{d}{dt}c(t)$.  Now compute
$$\begin{align}
d_pf(v)&=\frac{d}{dt}\big|_{t=0}f(c(t))\\
&=\frac{d}{dt}\big|_{t=0}\bigl(\langle c(t),y)\rangle Ac(t)\bigr)\\
&=\bigl(\langle \dot c(t),y\rangle A\cdot c(t)+\langle c(t),y\rangle A\cdot\dot c(t)\bigr)\big|_{t=0}\\
&=\langle v,y\rangle Ap+\langle p,y\rangle Av.
\end{align}$$
Edit: If you insist to write the differential without argument (but should one?), proceed as follows:
$$\begin{align}
d_pf(v)&=\langle v,y\rangle Ap+\langle p,y\rangle Av\\
&=A\bigl(\langle v,y\rangle p+\langle p,y\rangle v  \bigr).
\end{align}$$
Observe that
$$\langle v,y\rangle p=
p\langle y,v\rangle=py^Tv,
$$
hence
$$\begin{align}
d_pf(v)&=A\bigl(py^Tv+\langle p,y\rangle v  \bigr)\\
&=A\bigl(py^T+\langle p,y\rangle I\bigr)v
\end{align}
$$
and finally
$$d_pf=A\bigl(py^T+\langle p,y\rangle I \bigr)$$
But, to be honest, I like the expression $d_pf(v)$ more for symmetry reasons.  And it works for any inner product.
