Prove any function for $L^\infty$ norm can be approximated Prove that any function, continuous on an interval of $\mathbb R$, can be approximated by polynomials, arbitrarily close for the $L^{\infty}$ norm (this is the Bernstein-Weierstrass theorem). Let $f$ be a continuous function on $[0,1]$. The $n$-th Bernstein polynomial is:
\begin{align}
\displaystyle B_n(x)=\sum_{k=0}^{n}\binom{n}{k}x^k(1-x)^{n-k}f\left(\frac{k}{n}\right)
\end{align}

Here we start off with a very simple LOTUS (Law of the unconcious statistician) problem. So first we start off with:
$$\displaystyle S_n(x)=\frac{B^{(n,x)}}{n}$$
Here $\displaystyle B^{(n,x)}$ is a binomial random variable with parameters $n$ and $x$. We must prove that $\displaystyle B_n(x)=\mathbb E(f(S_n(x)))$, or:
$$\displaystyle \sum_{k=0}^{n}\binom{n}{k}(1-x)^{n-k}f\left(\frac{k}{n}\right)=\mathbb E\left(f\left(S_n(x)\right)\right) $$
By LOTUS we have:
$$\mathbb E(f\left( S_n\right))=\sum_{k=0}^{n}\mathbb P(S_n)f(k)$$
Then noting that the PMF of a Bin is:
$$\mathbb P\left(S_n=k\right)=\binom{n}{k}x^k(1-x)^{n-k}$$
We get: 
$$\sum_{k=0}^{n}\binom{n}{k}x^k(1-x)^{n-k}f\left(\frac{k}{n}\right)=\mathbb E\left(f\left(S_n\right)\right)$$
And with this we can prove that: $\displaystyle ||B_n-f||_{L_{\infty}([0,1])}\rightarrow 0$ as $n\rightarrow \infty$. $f$ is real, defined and continuous on $[0,1]$. Thus as n goes to infinity:
$$\displaystyle \left|\left|\sum_{k=0}^{n}\binom{n}{k}(1-x)^{n-k}f\left(\frac{k}{n}\right)-f(x)\right|\right|_{L_{\infty}^{([0,1])}}\rightarrow 0$$
So let us begin. Consider the random variable $\displaystyle f\left(\frac{S_n}{n}\right)$ from the previous problem. The expected value of this polynomial is a Bernstein polynomial, or:
$$\displaystyle \sum_{k=0}^{n}\binom{n}{k}p^k(1-p)^{n-k} f\left(\frac{k}{n}\right)$$
Now let us consider continuity and the law of large numbers. By continuity we can fix an $\epsilon > 0$, and say $\exists \alpha > 0$ s.t $0\leq x,y\leq 1$ and $|f(x)-f(y)|<\epsilon$. Thus now by the law of large numbers we can say, $\displaystyle \exists n_0\in \mathbb Z$, independent of the parameter $p$, s.t.
$$\displaystyle \mathbb P_n\left(\left|\frac{S_n}{n}-p\right|>\alpha\right)< \epsilon\hspace{5mm}\forall n\geq n_0$$
Thusly:
$$\displaystyle \left|\mathbb E_n\left[f\left(\frac{S_n}{n}\right)\right]-f(p)\right|=\left|\sum_{k=0}^{n}\left(f\left(\frac{k}{n}\right)-f(p)\right)\mathbb P_n(S_n=k)\right|$$
Working out the absolute values:
$$\displaystyle \sum_{\left|\frac{k}{n}-p\right|\leq \alpha}\left|f\left(\frac{k}{n}\right)-f(p)\right|\mathbb P_n(S_n=k)+\sum_{\left|\frac{k}{n}-p\right|> \alpha}\left|f\left(\frac{k}{n}\right)+f(p)\right|\mathbb P_n(S_n=k)$$
Then using the Law of Large numbers to make the subsititution:
$$\displaystyle \sum_{\left|\frac{k}{n}-p\right|\leq \alpha}\epsilon \mathbb P_n(S_n=k)+\sum_{\left|\frac{k}{n}-p\right|> \alpha}2 \sup_{0\leq x \leq 1}|f(x)|\mathbb P_{n}(S_n=k)$$
Simplifying:
$$\displaystyle = \epsilon \mathbb P_n\left(\left|\frac{S_n}{n}-p\right|\leq \alpha\right)+2\sup_{0\leq x\leq 1}|f(x)|\mathbb P_n\left(\left|\frac{S_n}{n}-p\right|>\alpha\right)$$
Therefore in conclusion we get for every $n\geq n_0$,
$$\displaystyle \left|\mathbb E_n\left[f\left(\frac{S_n}{n}\right)\right]-f(p)\right|\leq\epsilon+2\epsilon \sup_{0\leq x\leq 1}|f(x)|$$
Which clearly shows that we can be make $\displaystyle \left|\mathbb E_n\left[f\left(\frac{S_n}{n}\right)\right]-f(p)\right|$ arbitrarily small with respect to $p$.
