#434 – Aravind Srinivas: Perplexity CEO on Future of AI, Search & the Internet

4493.047 - 4520.089 Aravind Srinivas

You can back-propagate through every input token in parallel. So that way you can utilize the GPU computer a lot more efficiently because you're just doing matmos. And so they just said, throw away the RNN. And that was powerful. And so then Google Brain, like Vaswani et al, the transformer paper, identified that, okay, let's take the good elements of both. Let's take attention.