The NFL Draft, the Weight of an Ox, and the Art of the Rank List
By Benjamin Doolittle, MD
| February 27th, 2019 |
The ERAS applications are scored. Applicants and program directors alike have hung their interview suits back in the closet. It is time for the single most important moment of a residency program — the rank list meeting. Of all days, this day is the most important, even more so than Match. A program's culture and quality is greatly influenced by who shows up in June; it is the rank meeting that loads the dice.
How well do we craft our rank list? How well do we weigh in the balance of a transcript versus a personal impression? How much do letters of recommendation count? What if an applicant had an off day? It is a humbling process. There are so many who are much more accomplished than I was at their stage. Who am I to judge them?
How Do Program Directors Create a Rank Order List?
I asked several colleagues about what they do for their rank meeting. Each response was different. One program director sits with her administrative coordinator, tabulates the interviewees' scores, and loads the list in the computer. Just the two of them. Another colleague invites current residents to evaluate each applicant's file and weigh in on the rank list. One colleague holds the rank meeting over lunch. One program's rank meeting lasts 5 hours — for four spots.
It's a pretty broad range. What is the right way to do this? No one really knows, but I think the deliberative exercise and participation from the group makes sense.
The Draft and the Match
I have found nothing in the medical literature about what a rank meeting should look like. There are few parallels in other industries. The closest perhaps is professional sports. The Draft — baseball, football, basketball — looks a lot like the Match.
On the surface, these systems seem impossible to compare, like apples to oranges — medical student as professional athlete? — but the similarities are uncanny. PDs and coaches alike seek top candidates who have devoted their life's work to excellence. We both lean on statistics — yards rushed, USMLE Step I scores. We weigh the reputation of the institution. Did the athlete play in Division I, Big 10? Did the applicant graduate from a top-10 medical school? The Draft and the Match are binding agreements. Only in the rarest of circumstances have athletes or medical students rejected their positions.
Before the Rank List
As angst-inducing as the whole process can be, the Match is actually a humane innovation that makes the whole process more efficient than the practice was earlier in the 20th century. The Match was established in 1952 in response to several problems. Prior to the Match, hospitals would offer positions directly to medical students. They, in turn, would be pressured to accept or reject the offer. Keen to fill their slots, hospitals would try to offer their positions earlier than their competitors, with some offers coming as early as the beginning of third year. This became an unending cycle of anxiety by hospitals and applicants alike. Attempts were made to streamline the process. In the 1940's, a uniform date was established when hospitals made offers to applicants. Initially, students were given 10 days to respond. This was unacceptable to hospitals. The time until a response was then shortened to 12 hours. Even this was unacceptable to hospitals. Follow-up phone calls and telegrams would pressure applicants to decide. Nothing worked. With all stakeholders fed up, in 1950, the Association of American Medical Colleges (AAMC) voted for a trial run of a shared algorithm, a centralized match, in which all residencies could participate. We have tweaked the algorithm over the years, but the system has stuck because it works for both applicants and programs alike.
As a Program Director, What Works About Rank List Meetings for Me?
I think the deliberation of the group makes sense. I am intrigued by James Surowiecki's book, The Wisdom of Crowds, where he points out that spectators at a county fair can accurately guess the weight of an ox when their predictions are averaged together. Surowiecki proposes that for a group to be wise it must have four qualities. First, members should have a diversity of opinion. Second, their opinions are not determined by those around them. Third, members are able to draw from their own experience and local insights. Fourth, there is an open mechanism where private opinion can be morphed to collective decision.
I am not sure if a group of core faculty, within our shared institution, is completely wise. But I know my colleagues together are wiser than I am alone. I do not trust myself — with my unconscious biases and personality quirks — to do a fair-minded job.
Is it enough? Each year, we all find out — this year on March 15th.
Benjamin R. Doolittle, MD, is an expert in burnout and wellness in residents and physicians. He is an associate professor and program director of internal medicine and pediatrics at the Yale University School of Medicine and the medical director of the Yale Medicine-Pediatrics Practice.
Daniel F Dedrick March 1, 2019 at 10:54 am - Reply >
While I am now retired and do not know if my successor continues the approach I used for nearly three decades, my system (forgive me) MATCHED the author's suggestions. Our panel included resident representatives, a full spectrum of sub specialty faculty, and a mix of genders, races, and sexual orientations. Everyone voted with an equal voice, although quite rarely my chair would, without explanation, exercise a veto. All interviewers participated. Note that my former program was one of the largest in our specialty.
Just for my own interest, I always compared my own personal ranking for comparison to the official list generated by our group. My track record was good, but I was intrigued that nearly all differences between my own rankings and our group filing was due to concerns raised by resident interviewers. Maybe there is a lesson for us?