Numerical approximation of derivaties I am taking a course on deep learning and I a have a good understanding of High School math.
In the course they gave a formula saying that the derivative of a function $f(x)$
$f'(x) = \frac\lim{h\to0}\frac {f(x+h)-f(x)}{h} $ (formula 1) 
Which is the first principle of derivatives
Then they gave another formula saying that in numerical approximation the first principle don't work well, The formula was
$f'(x)=\frac\lim{h\to0} \frac{f(x+h)-f(x-h)}{2h}$ (formula 2)
They also said that the order of error in the first method is of $h$ and for the second is $h^2$
Which is much better assuming h is less than one.
I don't understand how this works because take $f(x)=x^3$ the triangle considered when we take $(x,f(x))$ and $(x+h,f(x+h))$ will be smaller than the triangle considered when taking  $(x-h,f(x-h))$ and $(x+h,f(x+h))$ so shouldn't formula 1 give a better approximation of slope than the second. Since bigger triangles tend to make more errors?

In the first case, the error term (using Taylor) is of the form 
${ 1 \over 2} h f''(\xi)$ whereas for the second it is
${1\over 4} h (f''(\xi_+)- f''(\xi_-))$ and if $f$ is smooth then 
$|f''(\xi_+)- f''(\xi_-)| \le L|\xi_+- \xi_-| \le L |h|$, which is where the $h^2$ comes from.
Roughly speaking the even terms tend to cancel.
