Can a person of average intelligence get a PhD in physics or math if he or she works hard enough? Or is a PhD a ceiling that they'll never be able to reach, no matter how hard they worked?  

By average intelligence I mean a person who has an IQ of 100.
I recently reread Malcolm Gladwell's book Outliers, which has a discussion of this very point, i.e., IQ versus academic success.  The gist of it is that, like it or not, the IQ test robustly measures something so that a score below X will, with high probability, prevent academic success at level Y.  E.g. a child with an IQ below 50 will have trouble taking classes with other students.  I just checked the passage, and he claims, quoting from Arthur Jensen's book Bias in Mental Testing, that with high probability someone with an IQ of below 115 cannot graduate from a university with grades that will get them into graduate school. 

One key phrase in the above is with high probability.  Thus no one is claiming that such success is impossible, but it becomes significantly more unlikely.  Another important point -- in fact, the whole point of this part of the book -- is that while having an IQ above the threshold seems to be important, the importance quickly disappears past the threshold.  In particular, though we may disagree whether the IQ threshold for getting a PhD in mathematics is 100 or 115 or whatever, beyond a certain level there is no significant correlation between IQ and academic success.  You might expect someone with an IQ of 175 to have a huge mathematical advantage over someone with an IQ of 125.  But this really seems not to be the case.

Let me say finally that this answer should not in any practical sense be discouraging to anyone.  If you or anyone else is wondering whether you have the intellectual capacity to get a PhD in mathematics -- you don't find out the answer by taking an IQ test, you find out the answer by studying mathematics and doing the best you can.  Someone who has a measured IQ of, say, 95 but gets a degree from a reputable university with excellent grades in challenging courses and competitive test scores: of course such a student should seriously consider getting a PhD in mathematics if they are interested, and should not be dissuaded at all by their "low IQ".