Why is the error of $O(h^2)$ when using Taylor expansion and centered approximation for the first derivative We know that the approximation of the first derivative by centered approximation is given by
$$ f'(x) = 
\frac{f(t+h) - f(x-h)}{2h} + O(h^2)$$
The quality of the above approximation is determined by the error. For smaller values of $h&gt;0$ the error should be as small as possible. To determine this error we use the Taylor expansion

\begin{align*}
y(t+h) &amp;= y(t) + y'(t)h + \frac{y''(t)}{2}h^2+\frac{y'''(t)}{3!}h^3+ O(h^4)\\
y(t-h) &amp;= y(t) - y'(t)h + \frac{y''(t)}{2}h^2-\frac{y'''(t)}{3!}h^3+ O(h^4)\\
y(t+h) - y(t-h) &amp;= 2y'(t)h + 2\frac{y'''(t)}{3!}h^3 + O(h^4)\\
\frac{y(t+h)-y(t-h)}{2h}-y'(t) &amp;= \frac{y'''(t)}{3!}h^2 + O(h^3) = O(h^2)
\end{align*}

an thus we conclude
$$
y'(t) = \frac{y(t+h)-y(t-h)}{2h}+O(h^2)
$$

Here is my question

We could have arbitrarily decided to take the Taylor expansion with the polynomial of fourth and fifth order namely writing the first line as follows: 
$$
y(t+h) = y(t) + y'(t)h + \frac{y''(t)}{2}h^2+\frac{y'''(t)}{3!}h^3+\frac{y''''(t)}{4!}h^4+\frac{y'''''(t)}{5!}h^5+ O(h^6)
$$
Thus the last line would have been: 
$$
\frac{y(t+h)-y(t-h)}{2h}-y'(t) = \frac{y'''(t)}{3!}h^2 + \frac{y'''''(t)}{5!}h^4 + O(h^5) = O(h^4)
$$
and conclude that the error is of order $= O(h^4)$

Why is this not true?
The correct last line is

$$
\frac{y(t+h)-y(t-h)}{2h}-y'(t) = \frac{y'''(t)}{3!}h^2 + \frac{y'''''(t)}{5!}h^4 + o(h^4) = \Theta(h^2)
$$ (unless $y'''(t)=0$).
Because for smaller value of $h$ which is less than $1$, $h^2$ is greater than $h^5$. Hence, if the error of an algorithm is $O(h^2)$ is in $O(h^5)$ for small values of $h$ and not vice versa.