Why does this approximation of square roots using derivatives work this way? I came up with this way to estimate square roots by hand, but part of it doesn't seem to make sense.
Consider how $f(n) = \sqrt{n^2+\varepsilon} \approx n$ when $\varepsilon$ is small. Therefore, using the tangent line with slope $f'(n) = \frac{n}{\sqrt{n^2+\varepsilon}}$ to approximate $f$ gives
$f(n) \approx n +\varepsilon\cdot\frac{n}{\sqrt{n^2+\varepsilon}}$
However, if the original approximation is substituted in the denominator, this gives
$f(n) \approx n + \frac{\varepsilon n}{n} = n + \varepsilon$
Which obviously makes no sense. However, if the chain rule is skipped while taking the derivative, the approximation becomes
$f(n) \approx n + \frac{\varepsilon}{2n}$
Which is a good approximation. For example, when $\varepsilon = 1, \sqrt{n^2+1} \approx n+\frac{1}{2n},$ and when $\varepsilon = n, \sqrt{n^2+n} \approx n + \frac{1}{2}$
Why does this work only when the derivative is done incorrectly? I feel like I am missing something obvious, but I can't see why this works the way it does.

$\newcommand{\eps}{\varepsilon}$In order to make a first-order approximation, you must fix three pieces of data: A differentiable function $f$, a point $a$ of the domain where the approximation is taken, and a point $x$ at which $f(x)$ is approximated. In this situation,
$$
f(x) \approx f(a) + f'(a) (x - a),\quad\text{or}\quad
f(a + \eps) \approx f(a) + \eps \cdot f'(a).
\tag{1}
$$
In your set-up, you write "$f(n) = \sqrt{n^{2} + \eps}$". This is already unusual, in that $\eps$ is built into the definition of $f$.
Next, by writing "$f'(n) = n/\sqrt{n^{2} + \eps}$", you're taking $a = n$. Normally, first-order approximation is done in a way that $f'(a)$ is easy to evaluate exactly; here, $f'(a)$ involves a denominator to be approximated. That said, logically there's no problem.
In the third line,
$$
f(n) \approx n + \eps \cdot \frac{n}{\sqrt{n^{2} + \eps}},
\tag{2}
$$
each summand is problematic:


*

*First, the left-hand side is $f(a)$, not $f(x)$.

*Second, the "constant term" $n$ is not $f(a)$, i.e., $f(n) = \sqrt{n^{2} + \eps} \neq n$. Line (2) is not of the form (1).

*Third, "$\eps$" plays two distinct roles: A number "baked into" the definition of $f$, and a small number representing $x - a$ in (1).
Here's what I suspect you were aiming for: define $g(x) = \sqrt{x^{2}}$ and $h(x) = \sqrt{x}$. You want the linear approximation of $g$ at $a = n$, and the approximation of $h$ at $a = n^{2}$:
$$
g(n + \eps) \approx n + \eps \cdot \frac{n}{\sqrt{n^{2}}};\qquad
h(n^{2} + \eps) \approx n + \eps \cdot \frac{1}{2\sqrt{n^{2}}}.
$$
The first is exact (so long as $n$ and $n + \eps$ are positive) since $g(x) = x$ for $x > 0$. The second is the "chain rule skipped" approximation you were expecting.
