Is there a non-abelian group $G$ such that $\operatorname{Aut}(G)$ is abelian? Is there a non-abelian group $G$ such that $\operatorname{Aut}(G)$ is abelian?  I have not been able to find an example.

A non-abelian group can have an abelian group of isomorphisms (automorphisms).
An affirmative answer to this question was given by G. A. Miller  in 1913 . 
Title of the paper is "A non abelian group whose group of isomorphism is abelian"
He constructed a non-abelian group $G$ of order $64$ such that $Aut(G)$ is abelian and
has order $128$. 
Added In this paper Author has defined  $p + 1$ non-isomorphic groups of order $p^8$ (for each $p$) whose automorphism groups are elementary abelian of order $p^{16}$.
This link may also help you.
