Second derivative "formula derivation" I've been trying to understand how the second order derivative "formula" works:
$$\lim_{h\to0} \frac{f(x+h) - 2f(x) + f(x-h)}{h^2}$$
So, the rate of change of the rate of change for an arbitrary continuous function. It basically feels right, since it samples "the after $x+h$ and the before $x-h$" and the $h^2$ is there (due to the expected /h/h -> /h*h), but I'm having trouble finding the equation on my own.
It's is basically a derivative of a derivative, right? Newtonian notation declares as $f''$ and Leibniz's as $\frac{\partial^2{y}}{\partial{x}^2}$ which dissolves into:
$$(f')'$$ and
$$\frac{\partial{}}{\partial{x}}\frac{\partial{f}}{\partial{x}}$$
So, first derivation shows the rate of change of a function's value relative to input. The second derivative shows the rate of change of the actual rate of change, suggesting information relating to how frequenly it changes.
The original one is rather straightforward:
$$\frac{\Delta y}{\Delta x} = \lim_{h\to0} \frac{f(x+h) - f(x)}{x + h - x} = \lim_{h\to0} \frac{f(x+h) - f(x)}{h}$$
And can easily be shown that $f'(x) = nx^{n-1} + \dots$ is correct for the more forthcoming of polynomial functions. So, my logic suggests that to get the derivative of a derivative, one only needs to send the derivative function as input to finding the new derivative. I'll drop the $\lim_{h\to0}$ for simplicity:
$$f'(x) = \frac{f(x+h) - f(x)}{h}$$
So, the derivative of the derivative should be:
$$f''(x) = \lim_{h\to0} \frac{f'(x+h) - f'(x)}{h}$$
$$f''(x) = \lim_{h\to0} \frac{  \frac{ f(x+2h) - f(x+h)}{h} - \frac{ f(x+h) - f(x)}{h}  }{h}$$
$$f''(x) = \lim_{h\to0} \frac{  \frac{ f(x+2h) - f(x+h) - f(x+h) + f(x)}{h}  }{h}$$
$$f''(x) = \lim_{h\to0} \frac{  f(x+2h) - f(x+h) - f(x+h) + f(x)  }{h^2}$$
$$f''(x) = \lim_{h\to0} \frac{  f(x+2h) - 2f(x+h) + f(x)  }{h^2}$$
What am I doing wrong? Perhaps it is the mess of it all, but I just can't see it. Please help.

Using the Taylor series expansions of $f(x+h)$ and $f(x-h)$,

$$ f(x+h) = f(x) + f'(x)h+f''(x)\frac{h^2}{2} + f'''(x)\frac{h^3}{3!}+\cdots $$
$$ f(x-h) = f(x) - f'(x)h+f''(x)\frac{h^2}{2} - f'''(x)\frac{h^3}{3!}+\cdots  $$

Adding the above equations gives

$$   \frac{f(x+h) - 2f(x) + f(x-h)}{h^2} =  f''(x) + 2\frac{f''''(x)}{4!}h^2+\cdots $$

taking the limit of the above equation as $h$ goes to zero gives the desired result

$$ \Rightarrow f''(x) = \lim_{h\to0} \frac{f(x+h) - 2f(x) + f(x-h)}{h^2} \,.$$

