What is Conjugate Gradient Descent of Neural Network? How is it different from Gradient Descent technique? I came across a resource but was unable to understand the difference between the two methods. It has mentioned in the procedure that  the next search direction is determined so that it is conjugate to previous search directions. What does this sentence mean? And what is line search mentioned in the web page?Can anyone please explain it with the help of a diagram?
What is conjugate gradient descent?
neural network;gradient descent;supervised learning
What does this sentence mean? It means that the next vector should be perpendicular to all the previous ones with respect to a matrix. It's like how the natural basis vectors are perpendicular to each other, with the added twist of a matrix:$\mathrm {x^T A y} = 0$ instead of $\mathrm{x^T y} = 0$And what is line search mentioned in the webpage?Line search is an optimization method that involves guessing how far along a given direction (i.e., along a line) one should move to best reach the local minimum.