This is a pre-print of a chapter by Milan Stürmer and Mark Carrigan, forthcoming in AI in 5 Minutes (Equinox Publishing). Please cite the published version once available.
Many educators are rightly critical of the idea that new technology will immediately “fix” education. This idea is often called “solutionism,” the view that complex social problems can be addressed by reaching for new technologies. Large language models (LLMs) like ChatGPT are a particular type of what Morgan Ames (2019) calls a “charismatic technology.” They are not just simple, tool-like objects but technologies whose charismatic promise overrides individual, social, institutional, or other kinds of agency. The implementation of LLMs will primarily be dependent not on its technical capabilities delivering empirical outcomes per se but rather on our devotion to its promises. This means that if we take the promise of the technological obsolescence of schools not just as the rantings of a bad-tempered teenager, we find a murky picture in which the educational implications of LLMs might complicate our understanding of what education is for.
Before we can explore whether access to LLMs means that schools no longer serve their existing purpose, we need to consider what purpose schools serve in the first place. The philosopher Gert Biesta (2024) argues that education has three fundamental tasks. Firstly, it provides the next generation with the understanding and skills that they need to act in the world, either as private citizens or in particular fields of activity (“qualification”). Secondly, it ensures they are inducted into the traditions and values that make up the culture in which schooling takes place (“socialization”). Thirdly, it supports them in developing the capacity to act in independent and meaningful ways, helping them to become individuals who are able and inclined to shape their own lives (“subjectification”). Once we recognize how distinctive these functions are, it becomes easier to get beyond the temptation to simply imagine that knowledge acquired in school could simply be acquired at home instead, in an automated form of homeschooling in which the promise of LLMs replaces the hopes invested in former media such as home video or online encyclopedias (Weller 2020). Could we imagine access to ChatGPT replacing each of these tasks? Let’s take them in order.
Qualification
“Qualification” is, from the perspective of learners, often the worst part of the school experience. It instrumentalizes education, reducing it to “preparing” learners for the practical challenges they will be expected to confront in their future lives. So while it is easy to imagine how LLMs could support the qualification function of schools in immediate and effective ways, doing so would intensify the bleakest and most life-denying features of compulsory schooling. If qualification is oriented toward preparing learners for entry into a working world that is the object of little, if any, optimism, then this will inevitably affect how the promises of LLMs are received by learners. In this setting, any opportunity to get “good enough” at these things without the hassle will be something that learners reach for enthusiastically. LLMs could be seen as a quicker and marginally less off-putting way to get through tasks that learners were not enthusiastic about in the first place.
The problem is that, as John Dewey pointed out long ago, the “ideal of using the present simply to get ready for the future contradicts itself. It omits, and even shuts out, the very conditions by which a person can be prepared for [their] future” (Dewey 1938, 49). The more we devise the obtaining of qualifications as frictionless, as a thing that school just provides and LLMs provide even better, the weaker the attachment to qualities like “ingenuity, curiosity, independence, perseverance, daring” (Carr 2014, 182) becomes—qualities that prepare us to extract the full meaning of each experience in the future. Being confronted with the promises of LLMs, we can see that we do not want schooling to be what Dewey described as “pouring knowledge into a mental and moral hole which awaits filling” (1916, 61), and once filled, that’s the end of that. As we get older, a lack of curiosity and hunger for novelty can leave us moving through life in increasingly closed and narrow ways. The legal philosopher Roberto Unger talks about this in terms of the “little deaths” we all experience: “A carapace of routine, of compromise, of silent surrenders, of half-term solutions, and of diminished consciousness begins to form” producing a “rigidified version of the self” in which we “die small deaths, many times over” (2014, 25). The layers of habit and expectation close us down and cut us off from the world. LLMs could precipitate this “death by installments” (Unger 2014, 25) by outsourcing thinking and suppressing the inclination toward discovery.
Socialization
Turning to socialization, the issue becomes more complicated as we must consider the technology itself because LLMs have to be “socialized” themselves. To understand this, we need to distinguish between pre- and posttraining. Pretraining is the initial phase where the model is exposed to vast amounts of text (for instance, entire libraries of books, Wikipedia) to learn the statistical regularities of language. In this stage, the model is not explicitly taught what is “right” or “wrong”; rather, it picks up patterns, syntax, and general knowledge in an open-ended, self-supervised fashion (e.g., predicting the next word). For example, if we prompted an LLM with “I can use a large language model to” then it would, with just the pretraining, simply be able to make a plausible inference about what word comes next in the sentence. This autocomplete can be a useful feature when writing text messages or emails, but it would be difficult to make grandiose claims about its capacity to revolutionize education.
This is where the posttraining comes in. Posttraining (sometimes known as fine-tuning or alignment) happens after the general language capabilities have been learned. In this phase, human feedback, curated datasets, or specific task objectives shape how the model should respond. This is when developers refine the model’s “character” and ethical guidelines, instruct it on which answers are acceptable, and tailor its style to desired norms and values. It’s the posttraining which encodes the capacity to distinguish between better and worse responses in the LLM, effectively giving it a functional equivalent to moral and practical judgment, even if the reality of this could not be more different from what is found in human beings. To the extent the LLMs have been deliberately socialized, they also acquire, we suggest, the capacity for deliberate socialization. Once they have the capacity to distinguish between better or worse responses, they have the capacity to present the user with value-laden responses to queries that require some kind of evaluation. In being normatively influenced, they gain the capacity to normatively influence.
The philosopher who led the posttraining team for Claude, Amanda Askell, explained in an interview how their approach was predicated on the model being placed in a position of talking to millions of people around the world, raising the question of the ethical and epistemic virtues you would like someone in that position to exhibit (Fridman 2024). In effect, they’ve tried to raise Claude as an affable cosmopolitan, able to talk across the wide variety of cultural contexts in which it would operate while avoiding passive reinforcement or needless confrontation with the diverging values encountered in the process. This is a very specific vision of what it means to be a good interlocutor. The model will articulate principles that it adheres to, reflecting a liberal common sense. In this sense, it embodies a certain view of being in the world that is strikingly similar to the ethos of UNESCO’s (2020) global competence framework.
If we start from the assumption that these are desirable values to instill in the next generation, this might seem unproblematic. But this new mechanism of socialization is emerging at precisely the moment in history when big tech companies and governments across the globe are openly turning against these values. Since LLMs must have value judgments encoded to be functional, they are inevitably entangled in political controversy over competing values. If they weren’t able to distinguish between better and worse responses, they would simply revert to being a turbo-charged version of your smartphone’s autosuggest function. Even if we try to define “useful” responses in the most minimal terms imaginable, you will still find yourself stuck in the terrain of values.
Obviously, contestation of the values encountered through school is not new, as the long political battles over national curricula attest to. However, it is nevertheless an entirely different matter to hand these decisions over to the few companies powerful enough to produce workable models. And even if we imagine the most idealistic model, transparently posttrained with democratic oversight and open to political contestation, it will, crucially, always lack mediation through an autonomous individual. Learners always encounter a difference between the convictions of their teachers and the values encoded in their curriculum. This not only functions as another layer of oversight in cultures that treasure individual judgment, but it is also, in itself, part of the socialization into the values of a culture that simultaneously transmits a disjunction between the normative system and the individuals living with it. This disjunction is key to a socialization that remains open to contestation and novelty. Archer (2012) argues that experiences of normative dissensus are crucial in enabling us to develop our own moral sense.
Subjectification
What about subjectification? This is the educational function for LLMs that seems likely to provoke the most controversy because the most immediate uses of LLMs seem to fall below the public radar, supporting users through the interpersonal dilemmas and private dramas, which are ubiquitous features of life. The emotional intelligence that Anthropic has worked hard to instill in the model Claude, even if we insist that it’s merely a performance of this, shows up for increasing numbers of users as a friendly and supportive presence continually available through their smartphone, tablet, or laptop. Obvious safeguarding issues notwithstanding, young people have always looked to the affordances of technology to make sense of themselves and their circumstances. It’s easy to imagine cases in which the continual presence of a genial interlocutor willing to talk at length about deeply personal issues could be beneficial to a young person questioning their identity. However, while this might be a “safe space” for identity exploration, it’s questionable whether it can support the development of genuine autonomy in the manner suggested by Biesta’s (2024) concept. It could certainly support them in the reflexive and relational engagement analyzed by Archer (2012) if they already have acquired the capacity to meaningfully articulate their identity struggles. But in itself, it’s unlikely to fulfill this role and could possibly distract from it through the opportunity costs involved in spending a great deal of time alone, talking to an LLM.
While the first wave of post-2022 LLM investment and development will most likely self-destruct due to the complete lack of economic viability or simply because it will have burnt up the planet’s remaining energy sources, the charismatic promise of autonomous conversational agents in education will certainly outlive the hype of the GPT-moment. While we should categorically deny the irrational drive to “technological perfection and human futility” (Mumford 1964, 266), we can use these discussions to reappraise and revalue the values of education. At the same time, we need to make sure that these arguments are not used to intellectually sanctify the ongoing attacks on education budgets everywhere in the name of a technological fix. Once adopted, these trends will be hard to reverse.
About the authors
Dr Milan Stürmer is a researcher at the Erasmus School of Philosophy in Rotterdam, where he coordinates the Erasmus Institute for Philosophy and Technology. Since 2023, he has been the vice president of the European Society for Process Thought.
Mark Carrigan is a digital sociologist working at the Institute for Education at the University of Manchester. Trained as a philosopher and sociologist, his research aims to bridge fundamental questions of social ontology with practical and policy interventions to support the effective use of emerging technologies within education.
