Is there hard evidence that the grant peer review system performs significantly better than random? Funding agencies in academia use peer review to assess the quality of submitted grant proposals and to decide which proposals should be funded. I have some doubts that this system actually works based on the following findings:


A recent study in PNAS "found no agreement among reviewers in
evaluating the same application".
We have by ourselves performed boot strapping on reviews provided
for the same proposal submitted at multiple calls in multiple years
(yes, the guidelines allowed this) and obtained on average almost exactly the overall average acceptance rate of the calls.


My question is now in the opposite direction ... Given that the grant review and panel system is quite costly: is there hard evidence that the grant peer review system provides results significantly better than random?

E.g. Do studies exist where e.g. 50 proposals were funded based on the standard review process and 50 random proposals were funded at the same call and then 5 or 7 years later the number of high profile papers, citations or similar was counted? Or do studies exist were re-evaluations show the same outcome as the original evaluations of proposals?
No such study exists.

You have to realize that the current model of funding research through grants is rather recent. Fifty years ago, research was funded with recurring credits: a lab/researcher got a fixed amount of money each year to do research as they saw fit. Only applied researchers got industrial grants to develop precise new applications, but this wasn't the core of their funding.

The current system arose due to the combination of two factors:


The growing hegemony of the neo-liberal ideology in the 80s-90s and the commodification of society as a whole. According to this ideology, research as a whole is supposedly more efficient (for an ill-defined notion of "efficient") if researchers compete with one another on the global market of research funding. They will "naturally" tend to research what works in order to keep their funding.
Politicians' will to govern research. Research, and especially fundamental research, is a notoriously difficult thing to govern: you never know where an idea might lead, or what new ideas will come up along the way. You don't know what will work and what won't. You don't know what will be applicable and what will remain a nice theoretical idea. Otherwise it wouldn't be research.
But rulers want their word in this: they want to tell their voters that their policies improved lives in a very direct way as a result of such and such decision they took. This is why words like "governance" and "steering" are becoming more and more prevalent in modern academia: administrators are not in the business of just helping researchers carry out their research, they are in the business of ensuring that researchers are researching what they are supposed to. And how do you force someone to do what you want, short of physical violence? Through economical violence.


Of course, this yields absurd situations, such as the fabled year when the French National Research Agency spent more on administration than on research. And such thinking paves the way for short-term research with zero long term goals, and especially not goals reachable in more than five years (the usual length of "long-term" grants). But these decisions are not driven by rational thought; they are driven by ideology.