#446 – Ed Barnhart: Maya, Aztec, Inca, and Lost Civilizations of South America

2571.465 - 2596.622 Arvid Lundmark

Yeah, so one big component of making it fast is speculative edits. So speculative edits are a variant of speculative decoding. And maybe it'd be helpful to briefly describe speculative decoding. With speculative decoding, what you do is you can kind of take advantage of the fact that most of the time, and I'll add the caveat that it would be when you're memory bound in language model generation.