What's in the Algorithms That Pervade Our Lives?
Vital aspects of our lives, such as the schools we attend, the jobs we get, where we live, whether we qualify for loans and how much we'll pay for insurance increasingly rely on invisible and impenetrable
So do government policies, ranging from timing of trash collection to policing patterns to your fate should you run afoul of the law.
Don't think decisions that rely on abstract mathematical models are necessarily any wiser than decisions made by the flawed humans. It's not just that our prejudices are built into our models. It's also that models built with the purest of intentions have unintended consequences. In her richly informative and disturbing book
Weapons of Math Destruction
argues that the "dark side of big data" is increasing economic inequality and creating a "toxic cocktail for democracy."
She describes how in late 2015 the political data firm, Cambridge Analytica, paid British academics to gather Facebook profiles of U.S. voters with demographic detail and records of each voter's "likes." The researchers then developed a psychographic analysis of 40 million voters, sorting them into different personality types. She reports that groups working with the Ted Cruz presidential campaign used these algorithms to create TV ads targeted to different types of voters and placed with programming that each type would be most likely to watch. The social and political significance of this kind mathematical modeling is being revealed repeatedly in ongoing news stories.
Financial troubles raise can your auto insurance rates
On a more personal level, O'Neill explains that insurance companies use the vast amount of data available about us to make mathematical calculations of various risks. Fair enough. But O'Neill, who is also a skilled story-teller, explains how auto insurers creating their own rating systems decided to use money management-measured by credit scores-as a proxy for responsible driving. As a result, a New York driver whose credit rating slipped from excellent to good faced a $255 hike on auto insurance. In Florida, adults with clean driving records and poor credit scores paid an average of $1,552 more than the same drivers with excellent credit and a drunk driving conviction.
Why would auto insurers focus on credit scores? Automated systems can plow through them efficiently and at enormous scale. O'Neil says it's also profitable. If an insurer can get an extra $1,552 a year from a driver with a clean record, why not? The victims are likely to be poor, less educated, less likely to know they are paying more, and less equipped to challenge the charge if they do.
Wildly irregular work schedules that change with little notice in retail and food businesses are another innovation from big data, and they bring havoc to lives of low paid workers trying to juggle school, child care, transportation and other responsibilities. But businesses profit by running with minimum manpower using tools for sophisticated analysis of peak customer traffic and staffing needs.
Some algorithms for crime and justice have unintended results
Police departments nationwide use
hot spot crime predicting algorithms
algorithms based on geographic locations where crimes are most likely. Those models have their virtue, O'Neill writes. The problem is that if murder and arson are the focus, will be a lot of down time. So the focus is often broadened to less serious crimes, such as vagrancy, aggressive panhandling, minor drug transactions and public drinking, offenses that are endemic to poor neighborhoods and would probably go unreported without police presence. But these more common nuisance crimes result in more arrests, creating a "pernicious feedback loop." The arrests generate new data, which justifies more policing, and generates more arrests. Again, those impacted tend to be minorities. "In our largely segregated cities," O'Neill writes, "geography is a highly effective proxy for race."
A risk for recidivism assigned to 7,000 criminal defendants by a private company's algorithm was found to