What determines how many master/PhD students a given professor can take (STEM)? It never really crossed my mind if there was an upper limit as to the number of students a given professor can take on. This is because some professor have dozens of students, while other have only a handful. I know a professor who has 2 PhD students and that's it. This seems to be a department dependent variable.

However, a rumor got out that one of the professor in my department (not my own) was attempting to take on a student but since the student did not have external scholarship, therefore the professor was not able to take him on. 

It is unusual because students do not need external funding to get into the graduate program at my school. I hypothesized that the professor's funding cap was all used up by his current students, therefore he could not take on anymore students without additional funding. However, I am not sure how this funding cap mechanism work.

Is there an upper limit to the number of students a professor can take on? If so, what determines it? An insider's perspective on this would be greatly appreciated. 

(My question is mainly with respect to engineering and STEM major within North American context)
At American universities, each department is allowed to accept a certain number of PhD students each year.  This number is determined annually by a dean or provost's office, primarily based on funding and student retention.  It could also be limited by the supervisory capacity of the faculty, but funding is usually more important.  Each department has its own procedure for matching students with supervisors.  Often the procedure is for students to go and ask faculty to supervise them.  In principal, there is nothing stopping a single faculty member from taking all the graduate students.  This probably happens from time to time in small departments.  

In the case of the rumour you mention, we can guess that this professor does not have access to additional TA/RA funding at this time, and is currently only taking students with external funding.  Typically TA funding is linked to the department, while RA funding is linked to a particular faculty member's grant.