 leave them in the bottom 10 percent or 20 percent over a one, two, or three-year period.
Education is not the only sector where policy by algorithm is currently in vogue.
The Obama administration in May announced a new plan to hold hospitals more accountable for outcomes involving Medicare patients. The formula to be applied in judging their efficiency would look not only at the cost of the services while the patient is hospitalized, but also for the cost of services performed by doctors and other health care providers in the 90 days after the patient leaves the hospital. Under the plan, a hospital that conducted, say, a hip replacement, would get a lower reimbursement rate if the patient later needed follow-up for an infection, even if the infection develops weeks after the original operation.
But the high promise of policy by algorithm mutates into cause for concern when data are thin, algorithms theory-bare and untested, and results tied to laws that enshrine automatic rewards and penalties. Current applications of value-added models for assessing teachers, for example, enshrine standardized tests in reading and math as the outcomes of import primarily because those are the indicators on hand. A signature element of many examples of contemporary policy by algorithm, moreover, is their relative indifference to the specific processes that link interventions to outcomes; there is much we do not know about how and how much individual teachers contribute to their students’ long-term development, but legislators convince themselves that ignorance does not matter as long as the algorithm spits out a standard that has a satisfying gleam of technological precision.
Google makes up for what it might lack in theory and process-knowledge by continually tweaking its formula. The company makes about 500 changes a year, partly in response to feedback from organizations complaining that they have been unjustly “demoted,” but largely out of a continued need to stay ahead of others who keep trying to game the system in ways that will benefit their company or clients. State laws are unlikely to be so responsive and agile.
Both data and algorithms should be an important part of the process of making and implementing education policy, but they need to be employed as inputs into reasoned judgments that take other important factors into account. The last thing we need are accountability policies that undermine education as a profession or erode the elements of community and teamwork that mark and make good schools. But when law and policy outrun knowledge, the results are likely to be unanticipated, paradoxical, and occasionally perverse.
The opinions expressed in Rick Hess Straight Up are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.