I was struck when reading the new Kevin Roose book how each of the AI labs is expressly motivated by a belief the other labs can’t be trusted to develop AGI. As he reports on loc 167, this was Sam Altman’s motivation for OpenAI:
“Been thinking a lot about whether it’s possible to stop humanity from developing AI,” Altman wrote in a 2015 email to Musk about the project. “I think the answer is almost definitely not. If it’s going to happen anyway, it seems like it would be good for someone other than Google to do it first.”
Anthropic was in turn motivated by a belief that OpenAI could not in fact be trusted. The most accurate way to read OpenAI/Anthropic, at least in their origins, is as a schism over safety in the same organisation. Hence why a group of people who I think sincerely believe this technology could destroy the world are workign towards building it as rapidly as possible. Meanwhile this was Deep Mind’s motivation as well, though slightly less personally framed than with the others. From loc 394:
And in 2010, after Hassabis had finished his PhD, he and Legg decided to start a company and try to build AGI themselves. They reasoned that if AGI was possible, someone would eventually build it, and it would be good if the people who got there first were people who were interested in developing it safely.
