Nvidia Part III: The Dawn of the AI Era (2022-2023)

3429.168 - 3454.337 Ben Gilbert

Three of our four clock cycles are taken up by loading and storing data to memory. Now, this is known as the von Neumann bottleneck, and it is one of the central constraints of AI, or at least it has been historically. Each step must happen in order and only one at a time. So in this simple example, it actually would not be helpful for us to add a bunch more memory to this computer.