I was struck in this Lex Friedman interview with Max Tegmark how the latter simply refuses to talk about capitalism when accounting for the existential risks he perceives as generated by AGI. In making sense of the competition between capitalist firms he doesn’t reach for political economy as an explanation or even neoclassical economics but rather the folkloric notion of Moloch: “an accelerating race towards a goal that has both a tremendous payoff and that guarantees our destruction”:
The answer is Moloch is tricking us into doing it. And it’s such a clever trick that even though we see the trick we still have no choice but to fall for it, right?
At points in the interview he basically uses Moloch as a synonym for capitalist competition but at other points its invoked in an actively mystical way, explaining away the destructive motivations of human actors as malign expressions of Moloch’s influence. For someone who I think could fairly be characterised as a ultra-rationalist, I find this incredibly weird. Not least of all because I’m pretty certain he’s not self-consciously using this notion in an allegorical way but in a real sense is suggesting that ‘Moloch’ is an efficacious force within the social world.
Has any one seen other examples of this? It’s an interesting route into the broader question which danah boyd most recently posed about the anthropological underpinning of the ‘AGI will destroy us’ mythology amongst digital elites.
Mark Carrigan 