In the Atlas of AI Kate Crawford Lewis Mumford’s concept of megamachine in order to make sense of contemporary AI-driven systems which incorporate the work of many thousands of actors. Mumford’s exemplar of this was the Manhattan Project in which 130,000 workers operated in secrecy to produce the first nuclear weapons which were used in Hiroshima and Nagasaki. Crawford plausibly suggests that we can understand our current global computational infrastructures as megamachines, writing on pg 48 on the distributed materiality upon which seemingly weightless processes depend for their operation:
Artificial intelligence is another kind of megamachine, a set of technological approaches that depend on industrial infrastructures, supply chains, and human labor that stretch around the globe but are kept opaque. We have seen how AI is much more than databases and algorithms, machine learning models and linear algebra. It is metamorphic: relying on manufacturing, transportation, and physical work; data centers and the undersea cables that trace lines between the continents; personal devices and their raw components; transmission signals passing through the air; datasets produced by scraping the internet; and continual computational cycles. These all come at a cost.
In the next section she talks about Amazon’s logistical operations which could be understood as another megamachine, drawing attention to how broken and battered human labour exists as a supplement to the Kiva robots which do the heavy lifting in the vast warehouses. From pg 54:
Robotics has become a key part of Amazon’s logistical armory, and while the machinery seems well tended, the corresponding human bodies seem like an afterthought. They are there to complete the specific, fiddly tasks that robots cannot: picking up and visually confirming all of the oddly shaped objects that people want delivered to their homes, from phone cases to dishwashing detergent, within the shortest amount of time. Humans are the necessary connective tissue to get ordered items into containers and trucks and delivered to consumers. But they aren’t the most valuable or trusted component of Amazon’s machine.
There is a continual process of machinic learning underway, in which customer feedback on things such as broken items is used to inform decisions about what boxes similar items will be packed in on future occasions. As she puts it, “Workers, however, are forced continually to adapt, which makes it harder to put their knowledge into action or habituate to the job” (pg 56). Is this a dynamic inherent in retaining human labour for completing systems which resist full automation? By its nature these tasks will not be fixed because if they were there would no longer be a need for a human being, with their stubbornly non-replicable capacity to make decisions in response to novel stimuli.
I’m not sure that boxing is the best example of this dynamic but it’s an important issue to raise nonetheless. If human labour is reducing to filling the gaps in AI megamachines, these gaps will by their nature will likely be prone to shifting in their nature as things change upstream in the system. Furthermore, these changes will be by their nature opaque and alienating, leaving the human worker responding to mechanic signals emanating from deep inside the computational megamachine, perhaps without the intervention of any human agent. The existing tendency to generate alienation inherent in division of labour will be radicalised as the parameters of that division are modulated in real time without human intervention. Much as the contribution of digital labour becomes opaque to those outside the machine, the machine itself becomes opaque to those operating within it.
With generative AI we’re likely to see this dynamic enter further into knowledge-work, beyond areas like call centres, taskwork and click farms where it has been common for some time. This recent Guardian essay by Laura Preston reflects on the experience of being an English Literature Graduate student filling in for an AI chatbot for real estate firms.
Brenda, the recruiter told me, was a sophisticated conversationalist, so fluent that most people who encountered her took her to be human. But like all conversational AIs, she had some shortcomings. She struggled with idioms and didn’t fare well with questions beyond the scope of real estate. To compensate for these flaws, the company was recruiting a team of employees they called the operators. The operators kept vigil over Brenda 24 hours a day. When Brenda went off-script, an operator took over and emulated Brenda’s voice. Ideally, the customer on the other end would not realise the conversation had changed hands, or that they had even been chatting with a bot in the first place. Because Brenda used machine learning to improve her responses, she would pick up on the operators’ language patterns and gradually adopt them as her own.
https://www.theguardian.com/technology/2022/dec/13/becoming-a-chatbot-my-life-as-a-real-estate-ais-human-backup
It’s particular sinister these staff are being asked to train Brenda through their responses, in the process ensuring their own redundancy. Her education was typical of staff who included “PhDs in performance studies and comparative literature, as well as a number of opera singers, another demographic evidently well suited for chatbot impersonation”. Their work through the Chatbot itself involved an automation of the work of estate agents who were no longer tasked with follow up on prospects over the phone or internet. This illustrates how automating the work of one group creates work for another, in spite of the prominence of the AI system in facilitating this transformation. The point of their job was to review the responses of Brenda to clients and change it if needed:
The word dog, for example, might compel Brenda to tag a message PET_POLICY, which would conjure some generic message about pet deposits from the property’s database. Once Brenda cued up her response, a three-minute timer appeared next to the message. When the three minutes elapsed, Brenda’s message was sent to the prospect. My job was to review the message and enter any changes before the timer ran down.
https://www.theguardian.com/technology/2022/dec/13/becoming-a-chatbot-my-life-as-a-real-estate-ais-human-backup
Before my first shift, I had imagined the operators were like ventriloquists. Brenda would carry on a conversation, and when she started to fail an operator would speak in her place. In reality, I rarely spoke for Brenda. Most of her missteps were errors of comprehension. She would seize on the wrong keyword and cue up a non-sequitur, or she would think she did not know how to answer when she actually had the right response on hand. In these situations, all I had to do was fiddle with the classifications – just a mouse click or two – and Brenda was moving along. In other instances, a prospect would pose a series of questions (What’s the rent? And utilities? When can I move in?) and Brenda would string together a composite response that collated so much information she sounded hostile. In these cases, I softened her aggressive recitation of facts with line breaks and merry affirmations. I wasn’t so much taking over for her as I was turning cranks behind the curtain, nudging her this way and that. Our messages were little collaborations. We were a two-headed creature, neither of us speaking on our own, but passing the words between us.
https://www.theguardian.com/technology/2022/dec/13/becoming-a-chatbot-my-life-as-a-real-estate-ais-human-backup
Occasionally the tag “HUMAN_FALLBACK” was used to mark interactions where Brenda needed to cede control entirely to a human operator. This often happened in response to difficult individual circumstances (“A substitute teacher told Brenda she couldn’t make the required income because if she did her disabled son would no longer qualify for his benefits”, “A man in his 70s told Brenda that his wife had died of a brain injury; after her medical bills bankrupted him, he had been evicted”) illustrating how human suffering can be a confounding variable in automated processes. What’s startling about the piece is how she describes the machinic qualities she took on herself in the process of doing the job, an experience not a million miles away from what being paid to run social media accounts did to my mind over time:
Eventually I reached a level of virtuosity where I could clear the inbox without much mental effort. The work no longer felt language-based. I was not reading messages one word after another, but perceiving each message as a unified cipher, as if the block of text were an image. My eyes would apprehend the web of critical words – pets, rent, utilities – and my hands would hit keys like notes in a musical passage. I stopped worrying about Brenda’s tone and began letting any message through as long as it was factually accurate. I realised that when Brenda sounded odd and graceless, people were less likely to get intimate, which meant less HUMAN_FALLBACK, which meant less effort for me. Months of impersonating Brenda had depleted my emotional resources. I no longer delighted in those rambling, uninhibited messages, full of voice and human tragedy. All I wanted was to glide through my shifts in a stupor. It occurred to me that I wasn’t really training Brenda to think like a human, Brenda was training me to think like a bot, and perhaps that had been the point all along.
https://www.theguardian.com/technology/2022/dec/13/becoming-a-chatbot-my-life-as-a-real-estate-ais-human-backup
If human labour will increasingly be reduced to filling in the gaps in computational megamachines, what will that mean for the humans who labour? This question leaves me wanting to return to the Economic and Philosophical Manuscripts of 1844, always my favourite bit of Marx (in fact if I’m honest the only part of his ouvre that has ever really moved me) and one which I suspect contains insights which will be deeply relevant to the horizon of radicalisation alienation from work opening up around us. Rob Horning points to this extract from Capital, originally cited by Sohn-Rethel:
Factory work … confiscates every atom of freedom, both in bodily and in intellectual activity. Even the lightening of the labour becomes an instrument of torture, since the machine does not free the worker from the work, but rather deprives the work itself of all content. Every kind of capitalist production, in so far as it is not only a labour process but also capital’s process of valorization, has this in common … Owing to its conversion into an automaton, the instrument of labour confronts the worker during the labour process in the shape of capital, dead labour, which dominates and soaks up living labour-power. The separation of the intellectual faculties of the production process from manual labour, and the transformation of those faculties into powers exercised by capital over labour, is, as we have already shown, finally completed by large-scale industry erected on the foundation of machinery.
As he writes, “it is easy to anticipate that “AI” will become an “instrument of torture” for workers, not only in threatening their livelihood but in “augmenting” their productivity by “confiscating every atom of freedom” in their activity and dictating the pace, decisions, and formulas to which they must adhere”. His point, drawing on Sohn-Rethel, is that “capital itself is always an “artificial intelligence” that emerges from the concentration of economic power”. It reminds me of Nick Land’s poeticisation of the obscenity of capital, described by Mark Fisher:
His idea was that capital was the most intense force ever to exist on earth—that the whole of terrestrial history had led to the emergence of this effectively planetary artificial intelligence system which therefore can be seen as retrospectively guiding all of history towards its own emergence—a bit like Skynet in the Terminator films.
Land’s work is this intense poeticisation of the power of capital. It’s interesting that that work came out in the Nineties at that moment of the high triumph of capital, after the collapse of the Soviet system at the end of the Eighties. Land’s work was really a play on—a development of—a kind of remix of earlier, ostensibly left-wing thought—particularly the work of Deleuze and Guattari and Lyotard—and they tried to imagine precisely a kind of postcapitalism that would try not to involve retreating from capitalist modernity but trying to go all the way through it.
In a similar register, Kate Crawford points to Marx’s concept of the worker being incorporated into the machine as an appendage, arguing that contemporary automation is a continuation of longstanding dynamcis of industrialisation, with the difference being that “employers now observe, assess, and modulate intimate parts of the work cycle and bodily data—down to the last micromovement—that were previously off-limits to them” (pg 58):
“In handicrafts and manufacture, the worker makes use of a tool; in the factory, the machine makes use of him. There the movements of the instrument of labor proceed from him, here it is the movements of the machine that he must follow. In manufacture the workers are parts of a living mechanism. In the factory we have a lifeless mechanism which is independent of the workers, who are incorporated into it as its living appendages.
This is why I think we should be careful with the idea that the ‘robots are coming’ for our jobs or the normatively ambiguous accelerationist responses likely to be provoked by our present conjuncture. As Aaron Benanav has argued there are methodological issues with how these claims are generated, embodying a formalist approach inadequate to understand complex ecologies of labour. But these responses are also doing political work in propping up what Nick Cave calls ‘algorithmic awe‘ that leaves us fixated on the technology.
Instead we need to be looking at the political economic context in which this technology is being diffused. It’s the end of cheap food, energy and capital when climate crisis precludes return to a stable equilibrium. If long term inflation is a continual feature of our economic environment as a consequence then it leaves us mired in the politics of distributing the costs; the high interest rates mandated by central bank orthodoxy depress capital investment while immiseration tends to weaken the power of labour. If I understand these macroeconomic trends correctly (and I’m out of my element here so please tell me if you think I don’t!) then the incentive will be towards cheaper automative fixes within existing structures, rather than the large scale investment required for wholesale replacement of human workforces with automated systems.
Mark Carrigan 