What should a student do, when a professor delays submitting his letter of recommendation? I'm applying to Stanford, which requires 2 letters of recommendation from teachers, due in 2 days.

Both teachers have known about this since mid-September, but one of them hasn't submitted his letter.  I've asked him, and he tells me that he has one prepared, but he hasn't submitted it yet.

I can't add a third letter that would qualify because of the way that the common application works.

Should I wait for the deadline to pass, hoping that he turns his letter in at the last minute?

Or should I ask someone else to write a letter in his place?

I'm mainly considering the first option, because I have no reason not to trust him, but I'm not sure what will happen if he doesn't pull through.

Turns out that they do accept letters of recommendation after the deadline, so long as the application is in on time.
Politely and very gently remind the professor that the letter of recommendation is due. That's really all you can do. If your professors says they will submit it on time, try to believe them.

As you have found out after posting the question, to the extent that it doesn't hold up other parts of their process, people on admissions committees understand that your flakey letter writers don't mean that you are flakey and will usually do what they can to accept or consider late letters. Often, departments will remind students or letter writers that their letters are missing, at, or even after, a deadline.