A machinery for producing rationalisations

I thought this was extremely powerful by Virgina Eubanks in Automating Inequality. She explains on pg 121-122 how machinic learning systems can operate as a form of triage, sorting people in order to distribute scarce resources in a seemingly more rational fashion:

COunter INTELligence PROgram of the FBI), for example, focused on civil rights activists for both their race and their political activism. But wiretaps, photography, tailing, and other techniques of old surveillance were individualized and focused. The target had to be identified before the watcher could surveil. In contrast, in new data-based surveillance, the target often emerges from the data. The targeting comes after the data collection, not before. Massive amounts of information are collected on a wide variety of individuals and groups. Then, the data is mined, analyzed, and searched in order to identify possible targets for more thorough scrutiny. Sometimes this involves old-school, in-person watching and tracking. But increasingly, it only requires finer sifting of data that already exists. If the old surveillance was an eye in the sky, the new surveillance is a spider in a digital web, testing each connected strand for suspicious vibrations. Surveillance is not only a means of watching or tracking, it is also a mechanism for social sorting. Coordinated entry collects data tied to individual behavior, assesses vulnerability, and assigns different interventions based on that valuation. “Coordinated entry is triage,” said Molly Rysman, the Housing and Homeless deputy for LA’s Third District. “All of us have thought about it like a natural disaster. We have extraordinary need and can’t meet all of that need at once. So you’ve got to figure out: How do we get folks who are going to bleed to death access to a doctor, and folks who have the flu to wait? It’s unfortunate to have to do that, but it is the reality of what we’re stuck with.” In his prescient 1993 book, The Panoptic Sort, communication scholar Oscar Gandy of the University of Pennsylvania also suggests that automated sorting of digital personal information is a kind of triage. But he pushes further, pointing out that the term is derived from the French trier, which means to pick over, cull, or grade marketable produce. “Although some metaphors speak for themselves, let me be clear,” he writes. In digital triage, “individuals and groups of people are being sorted according to their presumed economic or political value. The poor, especially poor people of color, are increasingly being treated as broken material or damaged goods to be discarded.”

But as she goes on to write on pg 122, those systems support moral judgements which can operate as rationalisations for those we don’t help and actions we don’t take:

But if homelessness is a human tragedy created by policy decisions and professional middle-class apathy, coordinated entry allows us to distance ourselves from the human impacts of our choice to not act decisively. As a system of moral valuation, coordinated entry is a machine for producing rationalization, for helping us convince ourselves that only the most deserving people are getting help. Those judged “too risky” are coded for criminalization. Those who fall through the cracks face prisons, institutions, or death.