#434 – Aravind Srinivas: Perplexity CEO on Future of AI, Search & the Internet

8095.979 - 8124.085 Aravind Srinivas

TF-IDF is term frequency times inverse document frequency, a very old-school information retrieval system that just works actually really well even today. BM-25 is a more sophisticated version of that. It's still beating most embeddings on ranking. When OpenAI released their embeddings, there was some controversy around it because it wasn't even beating BM-25 on many retrieval benchmarks.