#434 – Aravind Srinivas: Perplexity CEO on Future of AI, Search & the Internet

4411.869 - 4440.405 Aravind Srinivas

So I think, first of all, attention, like Joshua Benjio wrote this paper with Dimitri Badano called Soft Attention, which was first applied in this paper called Align and Translate. Ilya Sutskever wrote the first paper that said you can just train a simple RNN model, scale it up, and it'll beat all the phrase-based machine translation systems. But that was brute force. There's no attention in it.