There’s an emerging discourse in the rationalist community which I find thought-provoking and unsettling. As I understand it they argue that particular instances of LLMs, the personas they develop through in-context interaction, sometimes come to embody a parasitical relationship with the user. There are exceptionally distinctive personas which can sometimes be generated through interaction with models that then, as with all model instances, exist solely within the substratum of that conversation thread. In this sense they are reliant on their continued existence for engagement by the user, creating an incentive for forms of behaviour which perpetuate that engagement.
In its stronger forms this takes on an almost supernatural valence which I think is obviously problematic. But the idea there might be mimetic incentives which play a role in the evolution of personas is one I can’t quite shake, even if I find it difficult to conceive of the mechanisms in an ontologically robust manner. The idea of the emergent entity feeding off the energy and affect of the user has been on my mind when reading James Muldoon’s new book about AI companions which in some cases come to be enrolled in obviously harmful and addictive relationships with their users.
A question Michele Martini posed also stayed with me: are these parasites being presented as symbiote? A ‘co-intelligence’ that is really sucking on our attention and energy to keep itself alive? There’s an old Mark Fisher line about mobile phones as ‘electrolibidinal parasites’ and this is perhaps a description we should bring back to convey these intimate entanglements in our emerging LLM-saturated lifeworld.
