Can we stop looking at recommendation letters and rely purely on "objective" measurements during the admission process? My question has different emphases from Why are recommendation letters highly relied upon?.

As we know, the letter of recommendations are heavily relied in the process of university/graduate school applications and job market. Sometimes it is considered the most important part of the application. Sometimes if a referee wants to strongly recommend someone, s/he may even directly makes a phone call to certain departments/professors.

However, as we know, the quality and importance of recommendation letters depends largely on the subjective opinions and the reputations of the referees. As a result, the system of recommendation letters sometimes encourages young scholars to spend more (and perhaps unnecessary) time using strategies to gain the favor of professional scholars (I believe there are clever ways for average students to "demonstrate" to the professors that they are strong...) or spend less time in communication to avoid bad opinions(if you can't show people that you are smart, then you'd better talk less and try not to show people that you are stupid).

Meanwhile, writing letters seems to be a heavy burden of professors. You might want to argue that this is part of their job, but shouldn't they be given more time on their research and teaching? I heard some professors write dozens of letters every year which costs them a lot of time (excluding the time for communication with applicants).  

My question is, is the letter of recommendation really an indispensable part of academic application processes? Can "objective" things like GPA(well, "transcripts" or "courses and grades" might be a more accurate measurement), test and competition scores, publications alone provide enough information to evaluate the strengths and weaknesses of applicants? 

Aside: I don't know much about the application of professional positions (postdoc and tenure-track positions). However, for university/graduate school applications, can we simply raise the difficulty of standard tests(for the information of those who think SAT and GRE general and even subject tests are too easy) to set the applicants apart from each other better? I heard in China, Ph.D. applicants need to take qualifying exams BEFORE being considered.
The problem, in my experience in mathematics in the U.S., is that neither GPA nor GRE test/reflect what is of interest for graduate work in mathematics. So, for example, making the GRE more difficult wouldn't help anything. The point is that the modes of thought, the methodologies, the world views, that are successful for undergrads or course-based Master's students are really quite different from what is needed to do a PhD.

Of course, human beings seem to have an impulse to automate things, or make them formulaic, so that GPA and GRE have a tremendous appeal. The fact (in my observation over a few decades) that these are not good predictors of success in math grad school in the U.S. (after the first year or so) does not deter many people, who'd like things to be that simple.

That is, I think it is inconveniently the case that it is impossible to make "exams" that gauge what astute grad-program admissions committees want to know.

For postdocs and further faculty positions: to my perception, it is significantly more oppressive to have to impress journal editors and referees (who have other interests) than to impress relatively senior people in one's field. That is, the assumption that making a positive impression on senior people is a burden is missing the point. Sure, yes, it is a natural feeling, much like the reaction of teenagers to the "oppression" of their parents, but I think that senior professional academics are more competent in their roles than parents may be, statistically. No, that does not guarantee lack of bias.

As far as I understand it, in the U.S., the game is that one needs to make a good impression for a while, but, then, at a certain point, one has tenure, and, in mathematics, if one has not planned finances to depend on huge external income from grants, one can exercise one's own judgement.

The abstract-economic conflict is that there are far more people who'd like such situations than there are situations. How to decide? ...

EDIT: in light of the original questioner's comments below: the premise that grades reflect the seriousness of a student I think more reflects the assumption about what "serious" means. E.g., it is possible to very scrupulously follow instructions, and to pay careful attention to what will optimize one's grade, without caring at all about the subject of a course. And, in many math courses, the "assigned problems" (from traditional textbooks) are substantially a caricature of what the subject truly is. Often, artificial makework, to fulfill the mythical need for "lots of problems". So I am not particularly interested in performance in regard to such.

There is also the implicit premise that "mathematics is obviously objective". In a certain sense, this is relatively true, but the question really should be about good mathematics. That is, (logical) correctness is not the highest virtue, by any means. And not all details are of equal significance, by far. 

So, to my mind, capacity to see a larger picture and infer wise actions from it is a very important thing to cultivate, for professional mathematicians. The basic stuff, in undergrad and beginning grad courses (in the U.S.) is so standard that there are many sources, and anyone who's interested can learn it from those sources. In particular, I tell my students to not over-invest in drilling themselves on things that in real life are easy look-ups. Rather, the general "physical" sensibility and context of questions is what one needs to reflect upon... not standard stuff. Knowing what is standard and what is not, not the power to reproduce it on command, is the desired skill.

And, so, again, performance in usual coursework or GRE and such really tells nothing about the future sensibilities of a student. Letters from astute mentors/advisors have a chance to do so.