Analysis Question from Berkeley Problems in Mathematics I'm wondering if the following is correct. The original question(1.1.21, Fa96) asks to prove that
\begin{align*}
f''(x) = \lim_{h\rightarrow0}\dfrac{f(x+h) - 2 f(x) +f(x-h) }{h^2}
\end{align*}

I thought it was straightforward and proposed the following solution:

\begin{align}
f'(x) &amp;= \lim_{h\rightarrow0}\dfrac{f(x+h) - f(x)}{h} 
\\
f'(x-h)&amp;= \lim_{h\rightarrow0}\dfrac{f(x) - f(x-h)}{h}\\
f''(x) &amp;=\lim_{h\rightarrow0} \dfrac{f'(x) - f'(x-h)}{h} \\
\end{align}

Substitute $f'(x)$ and $f'(x-h)$ and we'll end up with our desired identity. This solution is different from the one proposed in the book so I want to make sure it is correct.
Strictly speaking no your solution is not correct.

We have a function
$$
f'(x) = \lim_{h \to 0} \frac{f(x+h)-f(x)}{h}
$$
but note if we try to evaluate $f'(x-h)$ then the $h$ inputted and the $h$ in the limit are two different variables (one is being taken to $0$ while the other is being taken to be constant). So really what we have is
$$
f'(x-h') = \lim_{h \to 0} \frac{f(x-h'+h) - f(x-h')}{h}
$$
and now looking at $f''$ we get
$$
f''(x) = \lim_{h' \to 0} \frac{f'(x) - f(x-h')}{h'} = \lim_{h' \to 0} \frac{\lim_{h \to 0} \frac{f(x+h)-f(x)}{h} - \lim_{h \to 0} \frac{f(x-h'+h) - f(x-h')}{h}}{h'}
$$
And now assuming sufficient differentiability of $f$ we get
$$
f''(x) = \lim_{h' \to 0} \lim_{h \to 0} \frac{f(x+h)-f(x)-f(x-h'+h)+f(x-h')}{h h'}
$$
but this requires justification in that you can interchange these limits (or take them both to $0$ at the same rate).
Hint: Another approach is to use Taylor series as

$$ f(x+h) \approx f(x)+f'(x)h+\frac{f''(x)}{2!}h^2 $$
$$ f(x-h) \approx f(x)-f'(x)h+\frac{f''(x)}{2!}h^2. $$

See a related problem.