Is the function $F(x)=\int_{0}^{x} f(t) dt$ differentiable at 0? Let 
$$ 
f(x)=\left\{\begin{matrix}
\cos\frac{1}{x} &amp; x\neq0\\ 
0, &amp; x=0
\end{matrix}\right.$$

I think the answer is not differentiable at 0 because of the chain rule, but I don't know how to prove it.
It is indeed differentiable with derivative zero: Of course, $\int_0^h \cos(1/x)\;\mathrm dx\to 0$, and it happens to go very quickly. This can be seen by a change of variables as follows.
$$
\int_0^h \cos(1/x)\;\mathrm dx=\int_{1/h}^\infty \frac{\cos(x)}{x^2}\mathrm dx
$$
which is easier to analyze. Namely, through an integration by parts 
$$
\int_{1/h}^\infty \frac{\cos(x)}{x^2}\mathrm dx=
-h^2\sin(1/h)+\int_{1/h}^\infty \frac{2}{x^3}\sin(x)\;\mathrm dx
$$
so, 
$$
\left|\int_{1/h}^\infty \frac{\cos(x)}{x^2}\mathrm dx\right|=
\left|-h^2\sin(1/h)+\int_{1/h}^\infty \frac{2}{x^3}\sin(x)\;\mathrm dx \right|\\
\leq 
h^2|\sin(1/h)|+\int_{1/h}^\infty \frac{2|\sin(x)|}{x^3}\;\mathrm dx
$$
by the triangle inequality. Evaluating the integral and using the obvious bound on $\sin$ we have
$$
\left|\int_{1/h}^\infty \frac{\cos(x)}{x^2}\mathrm dx\right|\leq
2h^2
$$
Using the above in the difference quotient:
$$
\left|\frac{1}{h}\int_0^h \cos(1/x)\;\mathrm dx\right|=
\left|\frac{1}{h}\int_{1/h}^\infty \frac{\cos(x)}{x^2}\mathrm dx\right|\leq 
2|h|\to 0
$$

Remark: If you keep playing the integration by parts game, you'll see it actually vanishes to the order of $O(h^n)$ for any $n\in \mathbb{N}$, i.e. faster than any polynomial.
If the function is not continuous at a point, the function at that point is not differentiable, too. 

To check the continuity of the function at $x=0$, we must compute it's L.H.L and R.H.L

$$L.H.L=\lim_{x\to0^-}cos(\frac{1}{x})$$
(Something oscillating between $[-1,1]$) 

$$R.H.L=\lim_{x\to0^+}cos(\frac{1}{x})$$
(Something oscillating between $[-1,1]$) 

Now, neither L.H.L nor R.H.L is defined. So, we can conclude that limit at $x=0$ doesn't exist and$f(x)$ is not continuous at $x=0$. Hence, $f(x)$ is not differentiable at $x=0$ .