Should recommenders be from the same discipline as the program one's applying to? If I want to go to graduate school to study physics and am a math major, how bad is it to have recommendation letters from math professors, mostly? The problem is that I just have four physics courses and after being picky about professors, I might not end up with all physics professors. Further, math professors know me better.

So, will it hurt if my recommenders are more mathematicians than physicists?
To point out one pitfall in kmdouglass's answer, there is one situation in which getting letters from out-of-discipline people can affect an applicant's chances: if the letter writers do not directly support the candidate's application to the specific discipline. In other words, if you're a mathematician applying to physics programs, your math professors should be explaining why you will be a great physics graduate student, not a great math graduate student. (Or why you will be a great graduate student in any department.) "Dissonance" between the letters of recommendation and where you're applying could make some reviewers question if you're seriously interested in applying to physics departments, which could hurt your chances (albeit perhaps only slightly).