derivative of quadratic function without transposes I'm trying to solve an equation of the following form:
$$ \frac{\partial}{\partial X} A'XA'X $$
where $X$ and $A$ are both equal-length column vectors (and so that $A'XA'A$ is scalar). From looking online I've seen derivatives of forms like $X'X$, but I'm unsure how it would work here when neither $X$ is transposed. My guess was
$$ 2A'AX $$
just to ensure that the derivative is also scalar, but I don't have confidence in that.

Two items; given a square symmetric matrix $B$ and a column vector $X$ that fits, the gradient of $X'BX,$ written as a column, is indeed $2BX.$
Next, the transpose of a one by one matrix is always itself; what else could it be? So $$  A'X = (A'X)' = X'A'' = X' A.  $$ As a result, your expression is
$$ A'X A' X = (A'X) A' X = X' A A' X = X' (A A') X.   $$
Taking symmetric square $B = A A'$ we get $X' B X.$
Your final expression needs to be $$  2 A A' X.  $$ Yours makes no sense; there is no such thing as $AX.$ If you meant $2(A'A)X$ it is still wrong.
I suggest you write this out in full detail for  dimension 1 and dimension 2. For dimension 2, give $A$ constant entries $a,b$ and give $X$ entries $x,y.$ As a column vector, what is the gradient of $(ax+by)^2 = a^2 x^2 + 2 a b x y + b^2 y^2?$
Using dot product, the gradient, as a row, is 
$$ 2 (A \cdot X) A',  $$
as a column
$$ 2 (A \cdot X) A.  $$
