At a minimum or a maximum why does the first approximation make no difference with small variations? In an ordinary function like the temperature—one of the properties of the minimum is that if we go away from the minimum in the first order, the deviation of the function from its minimum value is only second order.
At any place else on the curve, if we move a small distance the value of the function changes also in the first order. But at a minimum, a tiny motion away makes, in the first approximation, no difference.
Can anyone explain geometrically , algebraically or otherwise why this is true?
For the background of where this comes from in case of any confusion the history can be easily found in the first short paragraph below figure 19-7   provided by this link http://www.feynmanlectures.caltech.edu/II_19.html#Ch19-SUM
( Not sure how to cut paste pictures yet, but figure 19-8 is one click away. ) The other caveat is that this question may be more appropriate for a physicist. 

We say that a function $f: \mathbb{R} \rightarrowtail \mathbb{R}$ is differentiable at a point $a \in \text{Int}(\text{dom}(f))$ if there exists $A \in \mathbb{R}$ and $r: \text{dom}(f) \to \mathbb{R}$ function with $\lim\limits_{x \to a} \frac{r(x)}{x-a}$ so that
$$f(x)=f(a)+A(x-a)+r(x)$$
And of course $A=f'(a)$.
So if we have that $f'(a)=0$, then
$$f(x)=f(a)+r(x)$$
So there is no first order change in $f$.  
An example: $f(x):=x^2$ and $a:=0$. Then we have that
$$f(x)=f(0)+f'(0)(x-0)+r(x)$$
Substituting back everything:
$$x^2=0+0*x+r(x)$$
$$r(x)=x^2$$
So as you can see, the change around $a=0$ is second order. On the other hand, if we pick a different $a$, for example $a:=1$, we get that
$$f(x)=f(1)+f'(1)(x-1)+r(x)$$
$$f(x)=1+2(x-1)+r(x)$$
So we have a first order term as well. And the higher order term will be:
$$r(x)=x^2-1-2(x-1)$$
$$r(x)=x^2-2x+1$$
$$r(x)=(x-1)^2$$
I used here that I already know $f'(a)$, but you can calculate it this way:
$$f(x)-f(a)=A(x-a)+r(x)$$
$$\frac{f(x)-f(a)}{x-a}=A+\frac{r(x)}{x-a}$$
And now you can let $x \to a$, use the properties of $r$, and you will get that $A$ is just $f'(a)$.
