gradient descent with respect to a matrix I am trying to solve a maximize a scalar function f(X), where X is a matrix. I want to solve this using gradient descent, I have taken the derivative of f(X) w.r.t X. This seems a naive question, but does anyone know what to do next? Thanks so much!
Have you tried the naive answer that you can find in the wikipedia? http://en.wikipedia.org/wiki/Gradient_descent
Yes, I read that. I didn't want to mislead other people into the wrong solution. The thing is I'm not sure how to use it correctly, suppose X is an m by n matrix, the derivative of f(X) would be an n by m matrix. So should I take the transpose of the derivative first before using gradient methods? If so, why?