It's a complicated network of factors, but zoning regulations are a big part of this, most particularly in urban areas. Matt Yglesias has written a lot about this issue. The interesting this is that it is an issue that bisects the usual left-right divides.