My notes on Mantello, P. (2016). The machine that ate bad people: The ontopolitics of the precrime assemblage. Big Data & Society. https://doi.org/10.1177/2053951716682538

Since 9/11 the politics of prediction and risk have created an alliance between security agencies, technology firms and other commercial actors which seeks to create a precrime assemblage: the first generation sought to identify threats through data mining (“search habits, financial transactions, credit card purchases, travel history, and email communications”) but the next generation are “becoming intelligent assemblages capable of integrating data from a multitude of nodes in order to foresee and preempt harmful futures” (pg 2). These advances are being facilitated through cloud computing, machine learning and limitless storage.

The beta versions of these assemblages are being tested in real world situations, rendering it urgent for us to understand their implications. The first is what it means for criminal justice as a whole when the focus is on the anticipation of crime rather than dealing with its occurrence after the fact. The second is the expansion of surveillance into everyday life driven by the public-private alliances which are driving the agenda. The scope of surveillance is increasing but so too is to civic participation in it, driven by gamified mechanisms which “encourages citizens to do the securitization footwork of the state by offering them the opportunity to participate in do-it-yourself, reward-centered, pro-active, networked and, at times, and gamified versions of automated governance” (pg 2).

Peter Mantello argues that the allure of technological innovation is legitimating these developments, promising greater impartiality and efficiency, while the reality of their operation is extending juridicial reach in order to identify non immediate threats to the established order. The pre-crime assemblage will function “to preserve the domains of its masters, who will control immense existential and predictive data that will allow them to shape public perceptions, mold social behavior, and quell possible opposition, thereby ensuring the exception incontrovertible and infinite life” (pg 2).

He uses Massumi’s conception of ontopower to theorise this process, “a mode of power driven by an operative logic of preemption is spreading throughout the various structures, systems, and processes of modern life” (pg 3). Pre-emption itself is long standing but the preoccupation with speculative feelings of non imminent threats was, he argues, born out of the reaction to 9/11. If I understand correctly, the point is that risks are increasingly pre-empted rather than managed, with risk management becoming an anticipatory lens through actors and organisations proactively prepare for imagined futures.

Exceptionalism becomes legitimate under these circumstances, as anticipated threats are used to justify actions which would have otherwise been regarded as illegitimate. A mechanism like the “public safety orders” enacted by the New South Wale police expand the principle of anti-terror policing to civic law enforcement: “they shift the balance further away from the principles of due process where people are innocent until proven guilty and more toward a new era where crimes are committed before they happen, citizens are disappeared without recourse to defense, and where guilt and imprisonment are based on suspicion, rumor, association, or simply left to the intuitive ‘gut feeling’ of police officers” (pg 4). This goes hand-in-hand with an affirmation of the unpredictability of the future. Randomness and uncertainty mean that crimes cannot be avoided but this is why anticipatory work is seen as so important to minimise the threats on the horizon.

This anticipatory work tends to diffuse responsibility into an apparatus of knowledge production, identifying networks of connections or regional hot spots which become the locus of an intervention. A whole range of assets are deployed in the preparation of these interventions, as described on pg 5 in the case of Hitachi’s Public Safety Visualization Suite 4.5:

This includes mining data from an array of various nodes such as remote video systems (hotels/city streets/commercial and private properties/transporta- tion lines), gunshot sensors that alert CCTV cameras, vehicle license plate recognition systems, wireless com- munications, Twitter and other social media, mobile surveillance systems as well as useful data from smart parking meters, public transit systems, and online newspapers and weather forecasts.

Data visualisation plays a crucial role in this by “compressing vast amounts of invisible data into visible signifiers” (pg 5). However the uncertainty, ambiguity and construction which characterises the data itself is lost in the apparent self-evidence of the ensuing representations. The navigability, scalability, and tactility of the interface then mediates interaction with this experienced reality. The performative power falls away, as diverting police resources to ‘hotspots’ only to discover ‘more crime’ there (either comparable to what could be found elsewhere or encouraged by the aggravating factor of heavy handed police) comes to function as a legitimation of the apparatus itself. The approach also compounds existing inequalities through its reliance on historical apparatus about patterns of arrest in order to predict future offending.

What I found fascinating was the slippage in the software. An example on pg 6 concerns ‘at risk’ lists, intended to be the basis for social service interventions prior to any policing action, instead being used as target lists for people who were assumed to be likely offenders. This on the ground slippage highlights the importance of understanding the organisational context within which new tools are deployed, as a means to understand how their original intentions may mutate in the context of application.

The terrifying turn underway is from the deployment of past data to the harvesting of present data in real time. As Mantello puts it, this involves “the real-time extraction of personal data from an individual’s daily life—monitoring their patterns, routines, habits, emotional tendencies, preferences, idiosyncrasies, and geo- spatial coordinates” (pg 7). Enthusiasts claim that the broader the data that is harvested, the easier it will be to identify ‘criminal signatures’ at ever earlier points in time. This converges with what Zuboff has called surveillance capitalism in which behavioural data is leveraged to persuade rather than simply to predict. How might this modus operandi be enacted as part of the pre-crime assemblage? There is a truly dystopian horizon to such a project, described on pg 7:

Yet there is also the distinct dystopian possibility, in its never- ending ontopolitical pursuit to colonize and regulate all aspects of social life, that it may suppress dissent and discourage nonconformist thought or behavior. Already we are seeing such practices occur today with the increasing trends of self-censorship in social media due to fear of state surveillance and authoritarian reprisal

The gamified form this takes can be seen in Sesame Credit, produced in collaboration with Alibaba, as part of the early stages of China’s opt in social credit system, with rewards on offer for those who perform in ways that meet expectations. But as this becomes mandatory in 2020, we can expect this to go hand-in-hand with the proactive avoidance of people deemed to have poor social credit and potential sites where negative social credit behaviours may thrive. The author also considers the example of opt-in blackboxes in cars, where rewards on offer for those who agree to such monitoring but which eventually may be rolled out for everyone as part of a transformation of insurance. The City of Boston security app, Citizen Connect, offers ‘street cred’ recognition points for repeated contributions: “users who actively report on suspicious persons, ongoing crime, random acts of violence, or municipal infrastructure hazards get promoted to special ‘‘patrols’’ where they earn special badges of civic distinction” (pg 9).

I find it hard to read this excellent piece by Alfie Brown and not speculate about long term trends… how easy is it to imagine a world in a state of ecological collapse dominated by a few corporate city states fortified against the wastelands at their walls, as well as the millions of migrants fleeing climate catastrophe? He also makes the important point that coverage of these developments too easily frames this in contrast to the presumed democratic landscape ‘here’ and this misses the real significance of these possibilities.

Having long claimed to be apolitical, Jack Ma, the billionaire co-founder and executive chairman of the tech giant Alibaba, was recently revealed to be a member of the ruling Communist party of China (CCP). It’s another in a long list of links between corporate and state apparatus that stretch far beyond the borders of China. Nevertheless, a glimpse into the projects the company is working on in Cloud Town, considered in light of these revelations, should set the alarm bells ringing with fear of a dystopian future of state and corporate control.

Technologies in development at Cloud Town range from AI pedestrian crossing lights that use facial recognition to identify the age of a road-crosser and give them a longer green light if they are old/slow enough, to AI drone cars that can respond to passengers needs.

The greatest feature of the car, explained the proud representative, is that its media panel, linked to the user’s smartphone, reads patterns of movement, food choices and potentially even photos and comments, and then crosses this with millions of data sets to make predictions about what the user might like to eat and how they might like to travel there or have the food travel to them. In short, the new citizen outsources part of their decision-making processes, and maybe even part of their desire, to Alibaba. Our very impulses are mapped and planned in advance. The triangulation between data, predictive technology and desire could be the single most important relationship taking us into the dystopian smart city future.

In recent months, there has been increasing media coverage of the terrifying network of reeducation camps in which the Chinese government has interned hundreds of thousands of the Uighur people. This is only one part of a broader system of social control in which what Timothy Grose calls a ‘virtual custody’ has been constructed through the proliferation of “convenience police stations” at 200 metre intervals, a digital surveillance apparatus and state sanctioned home invasions in which “big brothers and big sisters” conducted 24m home visits, 33m interviews and 8m “ethnic unity” activities in less than two years. What I hadn’t realised was the role that China’s social credit system plays in this:

Yet the vast majority of detainees have not been convicted of any crime. Instead, the Communist party relies on an arbitrary social taxonomy – referred to officially as a “social credit system” – to identify targets. Metrics such as age, faith, religious practices, foreign contacts and experience abroad sort Muslims into three levels: “safe”, “normal” or “unsafe”. Those labelled “unsafe” face an imminent risk of detention.

https://www.theguardian.com/commentisfree/2018/nov/13/china-mass-incarceration-muslims-unchallenged-uighur

My understanding is that the social credit sanctions elsewhere in China have been predominately targeted at people in their capacity as consumers. This is not to minimise it because being locked out of credit and purchasing due to being designated ‘dishonest’ is an enormously significant penalty liable to impact upon every facet of life.

But are we seeing the next stage of this process in the oppression of the Uighurs? How will this trial of the social credit system be combined with other trials when the system is rolled out in full? Are we seeing a concrete techno-fascism being constructed before our very eyes? Not the diffuse fears and harms surrounding surveillance capitalism but a totalitarian system of datafication with reeducation camps at their core? While the potential role of private companies in the operation of the social credit system remains uncertain, firms have signed contracts for implementation with local governments. If the system operates effectively in China how long before these and other firms begin to offer related services to governments around the world?