Can you always decrease the variance by removing outliers? Consider a finite set of real numbers. Let its variance be $V$. Let the highest number be $h$ and the lowest number be $l$, and let's assume $l < h$.
Let  $x$ be an arbitrary number with $l < x < h$.
Now create a new set by removing one element equal to $h$ and replacing it with $x$. Call the variance of this new set $V_h$.
Also create a new set by removing one element equal to $l$ and replacing it with $x$. Call the variance of this new set $V_l$.
Is it true (and is there a proof!) of the statement: "either $V_l < V$ or $V_h < V$"?
To show that both are not always true, consider the set ${0,0,0,1}$ and replace one of the $0$'s with $0.9$. The variance goes up.

I think I may have found the proof.
Number the points $p_1,p_2,...,p_N$ from lowest to highest.
Replace one of the extreme points, $p_E$ where $E \in \{1,N\}$, by some $x$ with $p_1<x<p_N$. Call the new set ${q_1,...,q_N}$.
Now
$$
Var(q) = \frac{1}{N} \sum (q_i - \bar{q})^2 
\le \frac{1}{N} \sum (q_i - \bar{p})^2  
$$
since the variance is minimized around the mean.
So, we want to prove that 
$$
\sum (q_i - \bar{p})^2 < Var(p)
$$
i.e. that
$$
\sum (q_i - \bar{p})^2 < \sum (p_i - \bar{p})^2.
$$
This reduces to
$$
(x-\bar{p})^2 < (p_E - \bar{p})^2
$$
since the sets only differ in one element.
Now choose $E$ so that $x-\bar{p}$ has the same sign as $p_E - \bar{p}$. As $p_E$ was an extreme point, we know that the above inequality holds.
