Difference between Statistics and Machine Learning? What would I learn as a Ph.D Student in Statistics that I wouldn't learn and vise-versa in Machine Learning Ph.D program?
Machine learning is a quickly developing branch of technology, based on some basic statistics but also including elements of computer science, high performance computing, etc. 

Unfortunately, ML/AI is currently a buzzword and many departments want to have a course on it assuming it will attract students and bring money. This means that when it comes to taught courses, there is no consensus yet what should definitely be included and how the course can be structured. There are examples of ML courses drawing significantly from the classical foundations of Statistics, courses more focused on use of software, more vocal courses with significant discussion of ethics / legal issues around artificial intelligence. Similarly, a PhD project in ML can be either highly theoretical (e.g. proving a convergence or stability of training), or highly practical (e.g. building 100s of NNs until one of them does better than others for some reason), or whatever else. 

Statistics is a more established science, which has many applications. Depending on the area of applications, there may be some variations (e.g. Medical Stats may be slightly different from Statistical Physics), but the core methods are still the same, largely based on probability theory, calculus and linear algebra. I would not expect a PhD thesis in Stats to avoid maths completely  (something which I can not say about ML).