Is it okay for a professor to leave the classroom only 5 min past the class start if nobody has shown up? Last week, because of a subway delay due to bad weather I arrived 15 minutes late to class to find out my professor had left 10 minutes before because nobody else had arrived.

Was it okay for him to do that? Would it be rude to complain about this matter?

We are just 4 students between mid 20's to mid 30's, this is a grad school elective course with a single session of 3 hours every week, starting at 17:00 ending at 20:00.

I'm also a former TA and currently a lecturer of an undergrad Physics 101 course, so I'm aware of how it feels when even half of the class is missing.

Update version 2 Today I was the only one to attend (unsure about the reasons my classmates had, the weather was OK this time). And talked to the professor about the issue. He said this situation wasn't his problem. About the weather/commute issue he said this: "Of course I know that happens, I'm much older than you". Then decided there wasn't enough people, handed me the class notes saying we will skip theory again because "everyone is irresponsible" (exact words again) and left. I'm dumbfounded a second time.

Final update I decided to let it be for now. I'm not going to pick a fight alone like a crazy person (my classmates don't see a problem, neither with the professor's behaviour, nor theirs). If the situation takes a turn to the worse, I'll just withdraw for the course and hope someone else teaches it next year. Thanks for everyone's interest :) never expected so many comments!
I think this is less about regulations and more about respect and conditioning the students. Also, don't feel offended, you are not the (primary) reason for what happened.
Legitimate Reasons for Leaving
Respect
The fact that 0% of the students arrived shows the professor an

immense lack of interest in his teaching subject (&quot;I disdain the subject that you spend years researching and thus I disdain you.&quot;)
OR complete disregard of his didactic skills (&quot;You are so bad at teaching, that I don't think I can gain anything from your lecture.&quot;)
OR disregard of his personal status (&quot;You are such an unimportant person and your status is so much below me, that you will have to adjust your life to mine and wait for my arrival.&quot;)

Even if the class only consists of 5 students, he might extrapolate that everyone has one or more of these convictions. If he waits for students to arrive, he accepts his lower status and/or personal offense.
Operant Conditioning
If the professor waits for students to arrive, they will be conditioned that coming late  has no negative consequences. Canceling the class will incur a penalty to students and might force them to take arrival times more serious in order to gain the advantage of having the course material explained to them.
Recommendation
Communicate (in person or via e-mail) to the professor

that you are very sorry for your absence
state your reasons (should be severe enough)
and make sure to allude to one more of the three points mentioned in the respect section.

If he feels that you have legitimate reasons for being late and that your absence was not a sign of disrespect, I am sure that he will be more inclined to wait longer next time you are late or might even offer you to come by his office to &quot;pick him up&quot; for the lecture.

OP's Update
(This should probably be a new question, since the update significantly changes the circumstances.)
I am very sorry that the professor refused to give a lecture for you alone.
It is not your fault that you were the only present and thus it is imho unfair that you have to bear the consequences.
Updated Recommendation
I would approach him again

start with how unsatisfactory the situation for him and you is

again alluding to the respect section and


then ask him: Why do I have to bear the consequences for the action of other students? hoping that he is rational enough to realize his mistake. Basically you should try to make him understand that he does the same to you that other students are doing to him.

Maybe he was just disappointed about the low presence again and is an overly emotional person. Unfortunately that is not something you will be able to change.
If this does not resolve the situation I would ask around the department for existing regulations regarding this.