20VC: AI Scaling Myths: More Compute is not the Answer | The Core Bottlenecks in AI Today: Data, Algorithms and Compute | The Future of Models: Open vs Closed, Small vs Large with Arvind Narayanan, Professor of Computer Science @ Princeton

430.171 - 451.845 Arvind Narayanan

So if we look at what's happened historically, the way in which compute has improved model performance is with companies building bigger models. In my view, at least the biggest thing that changed between GPT-3.5 and GPT-4 was the size of the model. And it was also trained with more data, presumably, although they haven't made the details of that public and more compute and so forth.