If we imagine the speed and fluency of conversational agents continues to increase* from their present levels, what would this mean for how people habitually relate to a service like ChatGPT? One of the things which fascinate me about hallucination is the confidence with which these services tend to offer these statistically plausible but factually incorrect responses. These are being mitigated with time but they’re an inherent propensity of large language models which will never be removed. The ability to answer any question with confidence, even if those answers are factually incorrect, could become extremely psychically charged as part of a routine mode of relating; if the relationship becomes a taken for granted part of the lifeworld rather than a novelty being engaged with out of curiosity or to save time.
Learning to live with limitations involves recognising the limitations in knowledge, the constraints upon what we can do but what can be known more broadly. Digitalisation already pushed against our acceptance of those limits, inclining us towards a view of the world as fully legible with knowledge ‘out there’ waiting to be found. The continued development of GAI, particularly if it gets faster and more ubiquitous, will further push against these limits because there will be a speaking subject in our pocket who will act as if those limits are mostly or entirely absent. The new wave of datafication which GAI uptake within organisations, as well as incorporation into local devices through the lens of edge computing, will make the world somewhat more legible (in a limited sense) than it already was. But it will still be a fantasy and a dangerous one.
*The evidence suggests we are instead heading for a plateau, so this is a thought-experiment
Mark Carrigan 