In recent weeks I’ve been preoccupied by the thought that while I’m confident AGI, particularly the idea that it will emerge from LLMs, is a bullshit notion I’m far from certain. I’ve wondered if this uncertainty is more widespread than it seems and in fact plays a role in the competitive dynamics driving investment in the area. If you think it’s unlikely but you recognise you might be wrong, withdrawal from competition carries the risk of your competitors realising the gains if AGI is possible after all. The probability might be small but the consequences are vast. Indeed if the wildest predictions were true then the consequences are effectively infinite, in the sense the entire economic model would be transformed. Who could justify withdrawing from the race to build the machine god under these circumstances?
It’s an unsettling thought because… AGI is bullshit. This piece by Arjun Ramani and Zhengdong Wang helps me flesh out my dismal in a more concrete way. The claims of economic impact are ultimately about productivity growth: a general purpose technology enabling us to do more with less across the full range of economic activities. In its wildest formulation productivity growth would be exponential as AGI drives the innovation process itself. Yet as they point out uneven productivity growth across sectors dampens the link between productivity growth and economic growth:
Such a world is hard to achieve. As the economist William Baumol first noted in the 1960s, productivity growth that is unbalanced may be constrained by the weakest sector. To illustrate this, consider a simple economy with two sectors, writing think-pieces and constructing buildings. Imagine that AI speeds up writing but not construction. Productivity increases and the economy grows. However, a think-piece is not a good substitute for a new building. So if the economy still demands what AI does not improve, like construction, those sectors become relatively more valuable and eat into the gains from writing. A 100x boost to writing speed may only lead to a 2x boost to the size of the economy.[2]
What Ethan Mollick called the ‘jagged frontier’ has important implications here. Not only are the affordances of LLMs widely variable across tasks, it can be difficult to identify this variability without significant time and research. This means that not only are variable productivity gains intrinsic to LLMs as currently constituted, realising those gains takes a lot of investment on the ground in a way that often might not pay off in practice. In this sense I think we can say that whatever contribution LLMs might make to productivity growth will, by its nature, be unbalanced.
Even those knowledge-manipulation tasks which might seem susceptible to LLM-driven innovation in reality have bottle necks which make the claim of generalised innovation untenable. They point out that continued limitations of robotics are a further bottleneck, given how much productivity relies on physical engagement with the world. This crucial class of bottlenecks are ones which cannot be solved by an increase in the capability of LLMs. As they put it: “AI must transform all essential economic sectors and steps of the innovation process, not just some of them” and that “Otherwise, the chance that we should view AI as similar to past inventions goes up” i.e. it should be considered a “normal technology”.
This is compounded by the spiralling technical constraints which AI labs are now encountering. High quality data is drying up. The returns on scaling are declining. The demand for compute is rising faster than technical innovation can lower the costs. The regulatory environment is liable to become significantly more challenging as the real world costs of rapid diffusion become politically salient. Furthermore when we consider the reality of rapid diffusion we find examples where LLM use might lower productivity. They cite this example from Callum Williams:
GPT-4 is a godsend for a NIMBY facing a planning application. In five minutes he can produce a well written 1,000-page objection. Someone then has to respond to it… lawyers will multiply. “In the 1970s you could do a multi-million-dollar deal on 15 pages because retyping was a pain in the ass,” says Preston Byrne of Brown Rudnick, a law firm. “AI will allow us to cover the 1,000 most likely edge cases in the first draft and then the parties will argue over it for weeks.”
I don’t think this means that we should dismiss the productivity gains entirely. Andrew McAfee cites the estimate that “close to 80% of the jobs in the U.S. economy could see at least 10% of their tasks done twice as quickly (with no loss in quality) via the use of generative AI”. Where I differ from most sociologists is that I do find this claim prima facie plausible, with the caveat that realising that potential would require organisations that aren’t tyrannising their staff through aggressive cost cutting and the ultimate goal of automation them as much as possible. It requires professional autonomy, abundance of time, permission to experiment etc. Exactly what is lacking in (most) organisations that are preoccupied by productivity growth. These are the kinds of sociological considerations which are strikingly absent from the booster discourse. I was struck for example how McAfee cites the rapid expansion of context windows as if this is a straight forward contributor to productivity:
A final way to see the rapid improvement in generative AI systems is to look at the size of their “context windows,” or how much information they can accept from users. If, for example, a user wanted generative AI to summarize or rewrite a report, the report would need to be included in the context window along with the request. In 2020 state-of-the-art systems had a context window that could accommodate about 7 and a half pages of text. By late 2023 the window was 40 times larger and was able to accept about 300 pages of text.
The capacity to throw in vast amounts of context just to see if it helps encourages wasteful and costly use of LLMs. It also makes it easier to produce the pages of deliberately or accidentally blocking material referred to by Williams above. There’s not a linear relationship between the capabilities of the systems and the contribution they can make to productivity, both in terms of realising that contribution and the unintended consequences of widespread use of more capable models. Particularly within organisations that have incentive structures which have evolved under conditions of informational scarcity in which the existence of outputs could be treated as a cypher for the work which went into them and the characteristics of the person who did this work. This is exactly what research which individualises occupational roles and unbundles these roles into tasks will always miss:
A 2023 study examined all the tasks done by workers throughout the American economy to see which of them could be done at least at least twice as rapidly with no loss in quality via the use of generative AI. The research concluded that for about 20% of all workers, half or more of their tasks fell into this category. When the threshold was reduced to 10% of tasks, 80% of workers qualified. For example, interpreters and translators, survey researchers, and public relations specialists all had at least two-thirds of their tasks eligible for significant productivity improvement via generative AI. At the other end of the spectrum, workers including short-order cooks, athletes, and oil and gas derrick operators had no tasks in this category.
The unintended consequences spread through the network structure of the organisation, as well as the internal relations between the tasks which are currently bundled together into an occupational role. The effects which emerge at the intersection between two when things start changing are always going to be incredibly complex, as anyone who has ever had an organisational restructuring imposed on their workplace can tell you. (See Chesterton’s fence and a piece I’ve lost about the master spreadsheet that secretly keeps the organisation running) There’s an ecological complexity to working together which is completely missed by these reductive and taxonomic modes of analysing the character of workplaces. However McAfee is certainly correct in my view about the significance of the diffusion mechanism:
For one thing, much of the required infrastructure is already in place. Once new generative AI systems are developed they can be deployed around the world as quickly as web pages and apps can. A large and growing number of powerful applications using this technology are immediately available at no cost to anyone with an Internet-connected device. Others are available by subscription. This wide availability applies to both end users of the technology and developers who want to build new tools with it. Furthermore, as generative AI continues its rapid improvement, many of these improvements will propagate globally as soon as they’re released.
But this is actually a problem for productivity because it means diffusion can be individualised and chaotic. Individuals take up the tools in privatised ways, shaped by a social penalty on admitting LLM use, liable to create all sorts of short and medium term problems for organisations. They’re then incentivised to buy enterprise solutions as a way of managing the chaos. Again the sociology of the diffusion looks very different to a narrow reading of the economics of diffusion. Likewise the idea that you ‘just talk’ to the LLM:
Another reason to expect this technology to spread quickly is that it is an easy one for people to start working with. They just talk to it. Most of generative AI’s users don’t have to master a new user interface or programming language; they instead use natural human language. It requires time and practice to become proficient at interacting with generative AI, but it doesn’t require many “computer skills.” The technology, in short, is immediately available to people and quickly useful to them
There’s a scholastic fallacy here in which it’s imagined that most people are in the position to do this, with the professional autonomy to define their own workflows and the cultural capital required to articulate them consistently. However this shouldn’t lead us to be reassured about the existence of at least a class of jobs with this autonomy because I fear these are exactly the people (*cough* academics *cough*) who are liable to self-discipline with LLMs in order to sustain productivity increases which permit deliberate understaffing to intensify as an occupational strategy. So not automating the workforce or even restructuring the bundle of tasks but simply expecting fewer staff to do more because they’re now operating at a higher level of productivity. In this cases there’s an organisational incentive towards strategic ignorance on the part of management about this.
Mark Carrigan 