Agreed. In fact I don't think there will really be that much impact at all (other than making the tax code a little bit more fair).  First, there's a big difference between a $500k HOME and a $500k MORTGAGE. Most buying an expensive home do so from trading up through their lives and carrying equity from a previous home. That usually means a <$500k mortgage even on a $600k or $700k home. 

Second, you still can deduction the first $500k's worth of interest, meaning a $600k mortgage doesn't pay a ton more. Only those with really high mortgages are gonna pay a lot more, and people in that income bracket can usually afford it.

As you say, a little impact at the margins but that's a pretty small % of homebuyers.