Why do people who worry about the existential risks of AGI refuse to talk about capitalism?

I was struck in this Lex Friedman interview with Max Tegmark how the latter simply refuses to talk about capitalism when accounting for the existential risks he perceives as generated by AGI. In making sense of the competition between capitalist firms he doesn’t reach for political economy as an explanation or even neoclassical economics but rather the folkloric notion of Moloch: “an accelerating race towards a goal that has both a tremendous payoff and that guarantees our destruction”:

The answer is Moloch is tricking us into doing it. And it’s such a clever trick that even though we see the trick we still have no choice but to fall for it, right?

At points in the interview he basically uses Moloch as a synonym for capitalist competition but at other points its invoked in an actively mystical way, explaining away the destructive motivations of human actors as malign expressions of Moloch’s influence. For someone who I think could fairly be characterised as a ultra-rationalist, I find this incredibly weird. Not least of all because I’m pretty certain he’s not self-consciously using this notion in an allegorical way but in a real sense is suggesting that ‘Moloch’ is an efficacious force within the social world.

Has any one seen other examples of this? It’s an interesting route into the broader question which danah boyd most recently posed about the anthropological underpinning of the ‘AGI will destroy us’ mythology amongst digital elites.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.