R.P.I. May Belong Outside the N.C.A.A. Tournament Bubble The N.C.A.A., tired of hearing criticism from data geeks and television commentators about how it selects and seeds the 68-team men's basketball tournament, had a brilliant insight a couple of years ago. If these pundits were so smart, why not invite them into the process and see if they could do any better themselves? The N.C.A.A. has run a mock selection committee in February in Indianapolis, inviting about two dozen journalists from around the country to replicate the process as closely as possible. I was invited to participate this year, along with a motley crew of others, including Greg Anthony of Turner Sports and John Gasaway, a droll analyst who writes data-driven articles for ESPN and Basketball Prospectus. We huddled in a small conference room at the Conrad Hotel and actually selected, seeded and bracketed the field. It certainly wasn't easy - and not just because I've spent most of the past few months thinking about whether Rick Santorum can get back into the Republican race rather than whether Northwestern can sneak into the N.C.A.A. field. Once you get past the first 15 or 20 at-large teams, the bubble is exceptionally large, and pretty much every squad has a case to be made. The bubble is also quite forgiving, especially with the recent expansion in the field. California, for instance, may well make it into the tournament even if it loses the Pac-12 tournament despite having zero wins against teams ranked in the top 50. We spent most of our time debating these marginal cases. Teams as far down the food chain as Tennessee had their passionate defenders, even though they were only two games over .500 at the time. But there was a fundamental tension in the process: were we supposed to be picking the best teams or the most deserving ones? One example of the distinction is Xavier: did it deserve some consideration because a number of its players were suspended after a midseason brawl with Cincinnati? Xavier was probably more talented than its record at the time and might beat another potential 12th seed head-to-head. And yet, it was hard to call Xavier deserving since its problems were of the players" making. When I posed this question to David Worlock, the associate director for the men's tournament, his answer was unambiguous: we were supposed to be picking the best teams. The committee members spoke frequently of the "shirts and skins test": who would beat whom if they actually played a game? Under this paradigm, one particular factor tended to dominate in our discussion of the teams: quality wins. Which teams had showed they could win against tough opponents, especially away from home? The focus on quality wins, it turns out, contradicts another criterion that the selection committee weighs heavily: the Ratings Percentage Index, or R.P.I. The system, developed in 1981 in the era of the DOS prompt and the Commodore 64, tends to give computer rankings and other objective attempts at analysis a bad name. Over the long run, R.P.I. has predicted the outcome of N.C.A.A. games more poorly than almost any other system. And it shows some especially implausible results this season. Southern Mississippi, for instance, was somehow ranked ahead of Missouri, even though it has endured seven losses to Missouri's four (some of them against middling teams like Houston, Texas-El Paso, Alabama-Birmingham and Denver). The committee's use of R.P.I. is not quite as obsessive as you might think: more advanced systems like those developed by Ken Pomeroy and Jeff Sagarin were just a mouse click away, they told us - and it was perfectly well within the rules to look at them. The discussion of each team, moreover, was exceptionally thorough. It was clear from the officials we met that the committee has plenty of basketball knowledge and cares passionately about getting things right. But R.P.I."s fingerprints were all over the process. When a computer monitor displayed the teams that we were considering for the bubble, the R.P.I. ranking was listed suggestively alongside them. The color-coded "nitty gritty" worksheets that the committee has developed, and which often frame the discussion about the bubble teams, use the R.P.I. rankings to sort out the good wins and the bad losses. The irony is that almost any other computer ranking system would do a better job of what the N.C.A.A. is trying to accomplish. These systems tend to produce more intuitive results, and to reward teams more for quality wins, especially away from home. I developed a quick statistical procedure to identify teams that have a conspicuously poor R.P.I. ranking relative to the number of quality wins they have secured. Five teams stood out by this test. Missouri was the most obvious case: despite having gone an exceptional 9-3 against the R.P.I. top 50, the system ranks the Tigers 16th. Kansas, Notre Dame, Kansas State and Cincinnati also had a seemingly low R.P.I. ranking. The problem, in turns out, is not that computer rankings in general are incapable of perceiving their talent: it's that R.P.I. is the wrong system. Missouri, 16th in the R.P.I. rankings, was instead rated anywhere from the fifth to ninth by the more reliable systems. Cincinnati rated as high as 20th in one system - much better than its R.P.I. rank of 58. And the other systems see Kansas as a No. 1 seed, as most analysts do, while the R.P.I. rankings do not. In the end, the committee will probably get most of its decisions right. In fact, my analysis has showed that a team's seed line does have some power to predict the outcome of tournament games even once its power ratings are accounted for. That suggests that the committee's sausage-making process pays some dividends, and it has some good insight about who would win the shirts-and-skins test. But to a large extent, the N.C.A.A. process is a case of two wrongs making a right, counterbalancing R.P.I. flaws by making a series of mental adjustments to it. Using a more reliable system as the point of departure would give the committee a better way through the unenviable task of sorting through the bubble.