Prove that $\sum_{n\leq x}d^2(n)=O(x\log^3 x)$ 
Prove that $$\sum_{n\leq x}d^2(n)=O(x\log^3 x),\tag1$$
  where $d(n)$ is the divisor function:
  $d(n)=\sigma_0(n)=\sum_{a\mid n}1.$

I can prove that $$\sum_{n\leq x}d(n)=x\log x+(2\gamma -1)x+O(\sqrt x),\tag2$$ but how to prove $(1)$?

Here is a different argument. Note that $d(ab)\leq d(a)d(b)$ for all integers $a$ and $b$. We thus have
$$\sum_{n\leq x} d(n)^2 = \sum_{ab\leq x} d(ab) \leq \sum_{ab\leq x} d(a)d(b) = \sum_{a\leq x} d(a) \sum_{b\leq x/a} d(b) \leq 2\sum_{a\leq x} d(a) {x\over a} \log x,$$
where in the final inequality I used your bound on $\sum_{n\leq x} d(n)$. Now note
$$\sum_{a\leq x} {d(a)\over a} = \sum_{cd\leq x} {1\over cd} \leq \sum_{c\leq x}\sum_{d\leq x} {1\over cd} \leq (2\log x)^2.$$
