Is the symmetric definition of the derivative equivalent? Is the symmetric definition of the derivative (below) equivalent to the usual one?
\begin{equation}
\lim_{h\to0}\frac{f(x+h)-f(x-h)}{2h}
\end{equation}
I've seen it used before in my computational physics class. I assumed it was equivalent but it seems like it wouldn't matter if there were a hole at $x=h$ in the symmetric derivative, whereas with the usual one it wouldn't be defined. Which is kinda interesting...
If they're not equivalent - is there a good reason as to why we should use the common one? Or is the symmetric one actually more useful in some sense because it "doesn't care" about holes?

As I noted in a comment to the other answer, Milly's computation is incorrect.  I am posting this answer to rectify the situation.  The symmetric derivative is defined to be
\begin{align}
\lim_{h\to 0} \frac{f(x+h)-f(x-h)}{2h}.
\end{align}
If $f$ happens to be differentiable, then the symmetric derivative reduces to the usual derivative:
\begin{align}
\lim_{h\to 0} \frac{f(x+h)-f(x-h)}{2h}
&= \lim_{h\to 0} \frac{f(x+h)-f(x)+f(x)-f(x-h)}{2h} && (\text{add zero}) \\
&= \frac{1}{2} \left( \lim_{h\to 0} \frac{f(x+h)-f(x)}{h} + \lim_{h\to 0} \frac{f(x) - f(x-h)}{h}\right) \\
&= \frac{1}{2} \left( \lim_{h\to 0} \frac{f(x+h)-f(x)}{h} - \lim_{-h\to 0} \frac{f(x+h) - f(x)}{-h}\right) && (\ast)\\
&= \frac{1}{2} \left( f'(x) + f'(x) \right) && (\text{since $f'(x)$ exists}) \\
&= f'(x).
\end{align}
I'm begin a little trixy at $(\ast)$.  Notice that we can think about the difference quotient as being the slope of a secant line through the points $(x,f(x))$ and $(x+h,f(x+h))$.  This slope is given by
\begin{equation*}
\frac{f(x+h)-f(x)}{(x+h)-x}.
\end{equation*}
Multiplying through by $-1$, this becomes
\begin{equation*}
-\frac{f(x+h)-f(x)}{(x+h)-x}
= \frac{f(x+h)-f(x)}{x-(x+h)}
= \frac{f(x+h)-f(x)}{-h}
\end{equation*}
Taking $h$ to zero (which is the same as taking $-h$ to zero) on the left gives $-f'(x)$, justifying $(\ast)$.  Again, this proves the key statement:

Proposition: If $f$ is differentiable at $x$ (in the usual sense), then $f$ is symmetric differentiable at $f$, and the symmetric derivative agrees with the usual derivative.

The converse does not hold.  The usual example is the absolute value function which is not differentiable at zero, but which is symmetric differentiable at zero (with derivative zero):
\begin{equation*}
\lim_{h\to 0} \frac{|0+h|-|0-h|}{2h}
= \lim_{h\to 0} \frac{|h|-|h|}{2h}
= \lim_{h\to 0} \frac{0}{2h}
= 0.
\end{equation*}
In particular, this demonstrates that the symmetric derivative is a generalization of the usual derivative.  It cannot be equivalent, because it can meaningfully define the derivative of a larger class of functions.
As to the why? of the symmetric derivative (and why we don't use it instead of the usual derivative), I think that adequate answers can be found attached to this question and in comments elsewhere on MSE.
