How many referees is too many for a journal or conference paper submission? I recently reviewed a paper for a Math/Comp-Sci conference. The author's response came back with responses to 12 referees. 

Am I wrong in thinking it absurd that there are this many referees on a paper?  

For one, I, like many researchers, expect a paper to have about 2-3 referees, so make a rule of refereeing about this many papers for every paper I publish.  I will cannot referee 12 papers for ever paper I publish.   What does this do to the referee pool? 

Secondly, having so many referees seems to make the decision of any single referee as insignificant. If I spend 2 days on a paper, as is not uncommon in math, I would like to think my opinion counts for more than one vote in 12. 

Finally, responding to 12 referees is quite a burden for the author;  especially if referees make suggestions about style or presentation, which could be contradictory.     

I am not just ranting. Is there another side to this that I am missing?
I fully agree that 12 referees is too many. 2 to max 4 referees is fine. More referees will indeed leave the authors with conflicting comments, and does not contribute to the quality of the review.