Want to do mathematical physics - if first choice dept not a good fit, can I switch? I am writing on behalf of someone else.  But the question will be easier to pose if I put myself in his shoes and speak in the first person.

I will soon graduate with a Bachelor's in economics and mathematics in a Third World country.  I have, however, also studied physics. I actually have more physics than mathematics courses on my transcript.  I want to study for a PhD in mathematical physics in the U.S. I would need support (e.g. TAship).

Where I live, subject GREs are only offered once a year, and one is not permitted to take two different ones on the same day.  It is too far and expensive for me to travel to another country to take a second subject GRE.

Suppose, for the purpose of this question:


I decide to take the Physics GRE, and apply to physics departments
I get admission to a physics department, and a TAship
I take a combination of both math and physics courses and do well
After several semesters I realize I'd rather that my home department were math, instead of physics



  How hard would it be to apply to switch over to a math department (either in the same university, or a different one), assuming I had taken sufficient math courses, and done very well in them?


Note: The similar question is somewhat helpful, but I would like to see a specific focus on the math/physics interface; also, please consider the specific option of pausing at completion of a masters, with a switch of department at that point.
It seems like "you" should do a lot more research into existing graduate programs, since you haven't mentioned maybe the most obvious thing you should do: apply to an applied math Ph.D. program.  You should look carefully at what different programs expect, since "applied math" is a slippery concept; however, as such, applied math programs don't have an uniform an expectation of student background as pure math or physics programs.  Interdisciplinary programs are hot at the moment, but unlike a usual math or physics degree, what's on offer will vary dramatically from university to university.  In my new hometown of Waterloo alone, there are (I think usually funded) 


Masters in Applied Math (https://uwaterloo.ca/applied-mathematics/graduate-students/applying), 
Perimeter Scholars International (https://www.perimeterinstitute.ca/training/about-psi), and 
a special stream in the Institute for Quantum Computing (https://uwaterloo.ca/institute-for-quantum-computing/programs/graduate-studies).


That's a bit out of the norm, but there are many other programs out there.