This is an excellent interview with Timnit Gebru about the current hype surrounding generative AI. She describes the rush towards ever increasing large language models as a corporate pissing contest driven by executives scared of being left behind:
And then the higher ups be like: Why are we not the biggest? Why don’t we have the biggest model? Why don’t we? I’m just like: Oh, my God, why? What’s the purpose of having the biggest? What kind of pissing contest is this?
https://techwontsave.us/episode/151_dont_fall_for_the_ai_hype_w_timnit_gebru
I found the conversation in the latter half of the podcast particularly helpful for thinking about how the current economic environment (high interest rates, declining consumer spending, tanking share prices) mean this will be seized upon as a technological innovation, after crypto and the metaverse failed to catch fire as the ‘next big thing’. This was expertly exploited by OpenAI who launched ChatGPT upon the world in a way primed to create a viral sensation. The impending release of GPT-4 will push this contest for bigness to the next level:
We’re possibly just weeks away from the debut of GPT-4. While OpenAI has been mum on the details of this much-anticipated release, it has been rumored that GPT-4 will contain 100 trillion parameters, which would make it the largest LLM in the world.
While it’s been fashionable to downplay the importance of big data in recent years, the “bigness” of the LLMs is the precise source of all the new capabilities and the excitement. In fact, researchers are eagerly awaiting what new capabilities they might squeeze out of LLMs as they crank the size even higher.
https://www.datanami.com/2023/02/03/like-chatgpt-you-havent-seen-anything-yet/
Mark Carrigan 