What happens to surplus referee reports? As far as I understand, it is commonplace for editors to invite M reviewers when N &lt; M would suffice. After N reports arrive, they take a decision and inform the authors. What happens to the N+1st, N+2nd, ... referee reports that arrive after the decision has been taken?

As a referee, I have only been told “thanks, we won’t need your report anymore” a very small number of times, and as an author I’ve never been told “look, there’s one more report for you”. So I wonder what happens in practice.

EDIT: I have in mind journals, not conferences. A conference is more driven by deadlines, probably.
From my experience, things happen differently. Comments do highlight it correctly.

Initially, N reviews are solicited by the editor. If they disagree (or, I have to assume, some reviewers miss their deadlines), more reviews are solicited. I've been reviewer #5 few times.

Typical values for N are: 


2, with third as a tie-break – a typical grant application review scheme;
3, with 4 or even 5 as a tie-break - seen more often with journal papers.