This was extremely interesting in the Qualcomm* CEO’s interview with the FT. In essence I think he’s saying that the consumer economics of GAI won’t work until it’s in mobile devices, but there will never be enough compute on a mobile device to match the performance of a cloud-based model, which means (a) the local LLM will be parasitic upon the cloud-based LLM in ever more clever ways (b) it will rely on contextual data to support the inferences upon which this relationship depends:
When you think about AI models, they’re coming to devices. A number of different models are being ported to Snapdragon [Qualcomm’s mobile chip platform]. I think [mobile] platforms are becoming more open, with more potential for innovation. We [think] a hybrid model is going to be important, especially if you look at the high cost and high energy consumption of the data centres with gen AI. You want to be able to make use of distributed computing. A lot of the models that you see today are running on the cloud. But if there’s an inference version of that model [installed] in the device, the device will give the cloud model a head start. When you make a query on a device, it is way more precise, because the device has real context: where is Tim? What was Tim doing before? So the result is that this hybrid model is cheaper. Because when you leverage the computation on the device, the query is more precise and it’s actually a more efficient way of doing computing [than running AI apps in the cloud].
https://www.ft.com/content/dbc0984b-4801-4aeb-bcab-480704c34161
This suggests to me an early form of the digital daemon is coming soon, led by a desire to jump start the decelerating upgrade cycle of smart phones. But that access to these superpowers is going to be extremely data-hungry, leading to another round of competitive datafication. It’s all so depressingly familiar yet weirdly novel at the same time.
*Does anyone else think this firm sounds like a qualitative methods in communications research conference?
Mark Carrigan 