#434 – Aravind Srinivas: Perplexity CEO on Future of AI, Search & the Internet

8570.722 - 8593.079 Aravind Srinivas

So it's very important for you to track the tail latency, and we track it at every single component of our system, be it the search layer or the LLM layer. In the LLM, the most important thing is the throughput and the time to first token. We usually refer to it as TTFT, time to first token, and the throughput, which decides how fast you can stream things. Both are really important.