Example of forward difference approaching derivative only up to $O(h)$ According to the article on Wikipedia about finite differences, the forward and backward difference by $h$ of a function $f(x)$ divided by $h$ approach the derivative to order $O(h)$, i.e.
$$\frac {\Delta_h f(x)}{h} -f'(x)= \frac{f(x+h)-f(x)}{h} -f'(x) \in O(h)$$
$$\frac {\nabla_h f(x)}{h} -f'(x)= \frac{f(x)-f(x-h)}{h} -f'(x) \in O(h)$$
However, from Taylor's theorem, I'd expect this to be $O(h^2)$, which following the article, is only the case for the central difference.
$$\frac {\delta_h f(x)}{h} -f'(x)= \frac{f(x+h/2)-f(x-h/2)}{h} -f'(x) \in O(h^2)$$
Can anybody explain why? Or give an example of when the forward (/backward) difference only approach to order $O(h)$?

Taylor's theorem gives
$$
 f(x+h) = f(x) + h f'(x) + O(h^2)
$$
(for $h \to 0$) and therefore "only"
$$
\frac{f(x+h)-f(x)}{h} -f'(x) = O(h)
$$
A simple example is $f(x) = x^2$, where
$$
\frac{f(x+h)-f(x)}{h} -f'(x) = \frac{(x+h)^2-x^2}{h} - 2x = h
$$

For central differences, the function
$g(h) = f(x+h) - f(x-h)$ has the property that  not only
$g(0) = 0$ but also $g''(0) = 0$, therefore
$$
 f(x+ \frac h2) - f(x - \frac h2) = g(\frac h2) = \frac h2 g'(0) + O(h^3)
 = h f'(x) + O(h^3) \, .
$$
