of eight (12 percent) got a C grade:
Providing consumers with easy-to-read, easy-to-access information on their choices is valuable, both in restaurant window signs and through the health department’s website, which allows users to search for specific restaurants and access information about their inspections.
As it currently stands, though, the grading system is too coarse, because it masks wide variations in the quality of restaurants receiving the same grade, making it hard to say quite what it means to be an A-rated restaurant in New York City. Closer inspection of the underlying data reveals a suspicious distribution of restaurants near the cut-off point between an A and a B.
A restaurant receiving any score from 0 to 13 points gets an A, but the difference from one end of that range to the other is substantial. A zero score means that inspectors found no violations at all, while 13 points means they found a host of concerns. A (hypothetical) restaurant where hot food items were not being held above the required temperature of 140 degrees, toxic chemicals were improperly labeled or stored in a way that contamination of the food might occur, and the restroom had no toilet paper or trash can would get 12 points, according to the inspection system outlined in the Blue Book — and qualify for an A.
The graph below shows the distribution of A and B-rated restaurants inspected between July 28, 2010, when the grading began, and Jan. 3, 2011, when we compiled the data for this post. The horizontal axis tracks the number of violation points, and the vertical axis tracks the number of restaurants scoring in that category. The blue bars are A-rated restaurants, and the green bars are B-rated.
During this period, only 381 restaurants scored a perfect zero, while 1,012 scored 12 points and 721 scored 13 — just barely squeaking in to the A category. The data suggests that when you visit an A-rated restaurant, the odds are that it barely made the grade.
A more pressing statistical problem arises from the sharp break in the distribution that occurs around the grade cut-off point. Keep in mind that the 13-point threshold is somewhat arbitrary; there is likely to be little difference between a restaurant with 13 points worth of violations and one with 14 points.
Take a look, though, at the number of restaurants whose scores barely qualified for an A, and those whose scores just missed. Four times as many restaurants scored 11, 12 or 13 than scored 14, 15, or 16.
There are at least two possible explanations for this unusual distribution. One has to do with a provision of the system that allows non-“A” restaurants to request a re-inspection in hopes of earning a better score, and to hang a “Grade Pending” sign in their windows in the meanwhile. The spike in restaurants with 11, 12, or 13 points could be the result of restaurants with initial scores in the B range cleaning up their acts just enough to qualify for an A.
An alternative explanation could be related to the inspectors themselves. Knowing that restaurants that get B grades are likely to appeal them, inspectors may be more likely to rate a restaurants on the cusp with an A-range score; after all, the difference between 13 points and 14 is marginal, but the difference between an A and a B is meaningful. Given the subjective nature of the inspection process and the discretion that inspectors have to assign scores, the data suggest that inspectors may be disproportionately likely to assign restaurants a just-made-it A score than a just-missed B.
Given the current distribution of scores, what would happen if the cut-off for an A grade were to be raised slightly, to 12 or perhaps 11 or 10 points? Substantively, a difference of one or two points in a restaurant’s violation score is probably very small, but the shift in the number of A ratings would be quite significant.
Lowering the threshold for an A grade by one point would cut the proportion of restaurants earning an A by six percentage points, from 56 percent to 50. If it were lowered by two points, the percentage of A-rated rest