Zico Colter

set of kind of mathematical equations that will learn to predict the words in the sequence that is given to them. If you see the quick brown fox as your starting phrase of a sentence, it will predict the word jumped. We train a big model on predicting words on the internet.

And then when it comes time to actually speak with an AI system, all we do is we use that model to predict what's the next word in a response. This is, to put it bluntly, a little bit absurd that this works. So there's sort of two philosophies of thought here. People often use this sort of mechanism of how these models work as a way to dismiss them.

And then when it comes time to actually speak with an AI system, all we do is we use that model to predict what's the next word in a response. This is, to put it bluntly, a little bit absurd that this works. So there's sort of two philosophies of thought here. People often use this sort of mechanism of how these models work as a way to dismiss them.

And then when it comes time to actually speak with an AI system, all we do is we use that model to predict what's the next word in a response. This is, to put it bluntly, a little bit absurd that this works. So there's sort of two philosophies of thought here. People often use this sort of mechanism of how these models work as a way to dismiss them.

Oftentimes, I know people say, oh, well, AI is, it's just predicting words. That's all it's doing. Therefore, it can't be intelligent. It can't be. And I think that's just demonstrably wrong. What I think is amazing, though, is the scientific fact

Oftentimes, I know people say, oh, well, AI is, it's just predicting words. That's all it's doing. Therefore, it can't be intelligent. It can't be. And I think that's just demonstrably wrong. What I think is amazing, though, is the scientific fact

Oftentimes, I know people say, oh, well, AI is, it's just predicting words. That's all it's doing. Therefore, it can't be intelligent. It can't be. And I think that's just demonstrably wrong. What I think is amazing, though, is the scientific fact

that when you build a model like this, when you build a model that predicts words, and then just turn this model loose, have it predict words one after the other, and then chain them all together, what comes out of that process is intelligent. And I think it's demonstrably intelligent, right? I really believe these systems are intelligent, definitely. And I would say that this fact is

that when you build a model like this, when you build a model that predicts words, and then just turn this model loose, have it predict words one after the other, and then chain them all together, what comes out of that process is intelligent. And I think it's demonstrably intelligent, right? I really believe these systems are intelligent, definitely. And I would say that this fact is

that when you build a model like this, when you build a model that predicts words, and then just turn this model loose, have it predict words one after the other, and then chain them all together, what comes out of that process is intelligent. And I think it's demonstrably intelligent, right? I really believe these systems are intelligent, definitely. And I would say that this fact is

You can train word predictors and they produce intelligent, coherent, long form responses. This is one of the most notable, if not the most notable scientific discovery of the past 10, 20 years, maybe much longer than that, right? Maybe it was much deeper than that, in fact. And so this is not oftentimes given its due as a scientific discovery because it is a scientific discovery.

You can train word predictors and they produce intelligent, coherent, long form responses. This is one of the most notable, if not the most notable scientific discovery of the past 10, 20 years, maybe much longer than that, right? Maybe it was much deeper than that, in fact. And so this is not oftentimes given its due as a scientific discovery because it is a scientific discovery.

You can train word predictors and they produce intelligent, coherent, long form responses. This is one of the most notable, if not the most notable scientific discovery of the past 10, 20 years, maybe much longer than that, right? Maybe it was much deeper than that, in fact. And so this is not oftentimes given its due as a scientific discovery because it is a scientific discovery.

two kind of answers to this question, which are diametrically opposed, as with many questions, right? Because you're exactly right. The thought is, because these models are built to basically predict text on the internet, if you run out of text, that would imply that they're kind of plateauing. I don't think this is actually true for several reasons, which I can get into.

two kind of answers to this question, which are diametrically opposed, as with many questions, right? Because you're exactly right. The thought is, because these models are built to basically predict text on the internet, if you run out of text, that would imply that they're kind of plateauing. I don't think this is actually true for several reasons, which I can get into.

two kind of answers to this question, which are diametrically opposed, as with many questions, right? Because you're exactly right. The thought is, because these models are built to basically predict text on the internet, if you run out of text, that would imply that they're kind of plateauing. I don't think this is actually true for several reasons, which I can get into.

But just from a raw standpoint of training these models, I mean, there's two ways in which this is sort of maybe true, maybe false. It is true that a lot of the easily available data, sort of the highest quality data that's out there on the internet has been consumed by these models. We have used this data. There is not another Wikipedia and things like this, right?

But just from a raw standpoint of training these models, I mean, there's two ways in which this is sort of maybe true, maybe false. It is true that a lot of the easily available data, sort of the highest quality data that's out there on the internet has been consumed by these models. We have used this data. There is not another Wikipedia and things like this, right?

But just from a raw standpoint of training these models, I mean, there's two ways in which this is sort of maybe true, maybe false. It is true that a lot of the easily available data, sort of the highest quality data that's out there on the internet has been consumed by these models. We have used this data. There is not another Wikipedia and things like this, right?

There's only so much really high quality, good text that's available out there. On the flip side, and this is the point I often make, first of all, we're only talking about text there. We're only talking about publicly available text.