Dario Amodei
Appearances
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And and, you know, that that the idea of meaning as as as kind of the only important thing is in some ways an artifact of of a small subset of people who have who have been economically fortunate. But I think all that said, I think a world is possible with powerful AI that not only has as much meaning for everyone, but that has more meaning for everyone, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
That can allow everyone to see worlds and experiences that it was either possible for no one to see or possible for very few people to experience. So I am optimistic about meaning. economics and the concentration of power. That's actually what I worry about more. I worry about how do we make sure that that fair world reaches everyone.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
When things have gone wrong for humans, they've often gone wrong because humans mistreat other humans. That is maybe in some ways even more than the autonomous risk of AI or the question of meaning. That is the thing I worry about most, the concentration of power, the abuse of power, structures like autocracies and dictatorships where a small number of people exploits a large number of people.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I'm very worried about that.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yes, it's very frightening. It's very frightening.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It's some kind of natural convergent distribution. And I think what it amounts to is that if you look at a lot of things that are produced by some natural process that has a lot of different scales, right? Not a Gaussian, which is kind of narrowly distributed.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
One, I realized it would be very long. And two, I'm very aware of and very much try to avoid You know, just being, I don't know what the term for it is, but one of these people who's kind of overconfident and has an opinion on everything and kind of says a bunch of stuff and isn't an expert.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I very much tried to avoid that, but I have to admit, once I got the biology sections, like I wasn't an expert. And so as much as I expressed uncertainty, probably I said a bunch of things that were embarrassing or wrong.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Thanks for having me. I just hope we can get it right and make it real. And if there's one message I want to send, it's that... To get all this stuff right, to make it real, we both need to build the technology, build the companies, the economy around using this technology positively. But we also need to address the risks because those risks are in our way.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
They're landmines on the way from here to there. And we have to defuse those landmines if we want to get there.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Like all things.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But, you know, if I look at kind of like large and small fluctuations that lead to electrical noise, they have this decaying 1 over X distribution. And so now I think of like parallelism. patterns in the physical world, right? Or in language. If I think about the patterns in language, there are some really simple patterns. Some words are much more common than others, like the.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Then there's basic noun-verb structure. Then there's the fact that nouns and verbs have to agree, they have to coordinate. And there's the higher level sentence structure. Then there's the thematic structure of paragraphs. And so the fact that there's this regressing structure, you can imagine that as you make the networks larger, for
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
First, they capture the really simple correlations, the really simple patterns, and there's this long tail of other patterns. And if that long tail of other patterns is really smooth, like it is with the 1 over F noise in physical processes like resistors, then you can imagine as you make the network larger, it's kind of capturing more and more of that distribution.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And so that smoothness gets reflected in how well the models are at predicting and how well they perform. Language is an evolved process, right? We've developed language. We have common words and less common words. We have common expressions and less common expressions. We have ideas, cliches that are expressed frequently, and we have novel ideas.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And that process has developed, has evolved with humans over millions of years. And so the guess, and this is pure speculation, would be that there's some kind of long tail distribution of the distribution of these ideas.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
If you have a small network, you only get the common stuff, right? If I take a tiny neural network, it's very good at understanding that, you know, a sentence has to have, you know, verb, adjective, noun, right? But it's terrible at deciding what those verb, adjective, and noun should be and whether they should make sense. If I make it just a little bigger, it gets good at that.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Then suddenly it's good at the sentences, but it's not good at the paragraphs. And so these rarer and more complex patterns get picked up as I add more capacity to the network.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I don't think any of us knows the answer to that question. My strong instinct would be that there's no ceiling below the level of humans, right? We humans are able to understand these various patterns. And so that makes me think that if we continue to scale up these models to kind of develop new methods for training them and scaling them up,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
that will at least get to the level that we've gotten to with humans. There's then a question of, you know, how much more is it possible to understand than humans do? How much is it possible to be smarter and more perceptive than humans? I would guess the answer has got to be domain dependent.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
If I look at an area like biology, and I wrote this essay, Machines of Loving Grace, it seems to me that humans are struggling to understand the complexity of biology, right? If you go to Stanford or to Harvard or to Berkeley, you have whole departments Of, you know, folks trying to study, you know, like the immune system or metabolic pathways and and each person understands only a tiny bit.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Part of it specializes and they're struggling to combine their knowledge with that of with that of other humans. And so I have an instinct that there's there's a lot of room at the top for A.I. to get smarter.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
if I think of something like materials in the physical world or, you know, like addressing, you know, conflicts between humans or something like that, I mean, you know, it may be there's only some of these problems are not intractable, but much harder. And it may be that there's only so well you can do with some of these things, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Just like with speech recognition, there's only so clear I can hear your speech. So I think In some areas, there may be ceilings that are very close to what humans have done. In other areas, those ceilings may be very far away. And I think we'll only find out when we build these systems. It's very hard to know in advance. We can speculate, but we can't be sure.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah. I think in many cases, you know, in theory, technology could change very fast. For example, all the things that we might invent with respect to biology are But remember, there's a clinical trial system that we have to go through to actually administer these things to humans.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think that's a mixture of things that are unnecessary and bureaucratic and things that kind of protect the integrity of society. And the whole challenge is that it's hard to tell. It's hard to tell what's going on. It's hard to tell which is which, right? My view is definitely... I think in terms of drug development, my view is that we're too slow and we're too conservative.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But certainly, if you get these things wrong, it's possible to risk people's lives by being too reckless. And so at least some of these human institutions are, in fact, protecting people. So it's all about finding the balance. I strongly suspect that balance is kind of more on the side of pushing to make things happen faster, but there is a balance. If we do hit a limit—
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Idea limited? So a few things. Now we're talking about hitting the limit before we get to the level of humans and the skill of humans. So I think one that's popular today and I think could be a limit that we run into, like most of the limits, I would bet against it, but it's definitely possible, is we simply run out of data. There's only so much data on the internet.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And there's issues with the quality of the data, right? You can get... hundreds of trillions of words on the internet, but a lot of it is repetitive or it's search engine optimization drivel, or maybe in the future, it'll even be text generated by AIs itself. And so I think there are limits to what can be produced in this way.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
That said, we, and I would guess other companies, are working on ways to make data synthetic. where you can use the model to generate more data of the type that you have already or even generate data from scratch.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
If you think about what was done with DeepMind's AlphaGo Zero, they managed to get a bot all the way from no ability to play Go whatsoever to above human level just by playing against itself. There was no example data from humans required in the AlphaGo Zero version of it.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
The other direction, of course, is these reasoning models that do chain of thought and stop to think and reflect on their own thinking. In a way, that's another kind of synthetic data coupled with reinforcement learning. So my guess is with one of those methods, we'll get around the data limitation or there may be other sources of data that are available.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We could just observe that even if there's no problem with data, as we start to scale models up, they just stop getting better. It seemed to be a reliable observation that they've gotten better. That could just stop at some point for a reason we don't understand. The answer could be that we need to invent some new architecture.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
There have been problems in the past with, say, numerical stability of models where it looked like things were leveling off, but actually when we found the right unblocker, they didn't end up doing so. So perhaps there's some new – optimization method or some new technique we need to unblock things.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I've seen no evidence of that so far, but if things were to slow down, that perhaps could be one reason.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So right now, I think most of the frontier model companies, I would guess, are operating roughly you know, $1 billion scale plus or minus a factor of three, right? Those are the models that exist now or are being trained now.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think next year we're going to go to a few billion and then 2026, we may go to, you know, above 10 billion and probably by 2027, their ambitions to build $100 billion clusters. And I think all of that actually will happen. There's a lot of determination to build the compute to do it within this country. And I would guess that it actually does happen.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Now, if we get to 100 billion, that's still not enough compute. That's still not enough scale. Then either we need even more scale or we need to develop some way of doing it more efficiently, of shifting the curve.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think between all of these, one of the reasons I'm bullish about powerful AI happening so fast is just that if you extrapolate the next few points on the curve, we're very quickly getting towards human level ability, right? Some of the new models that we developed, some reasoning models that have come from other companies,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
They're starting to get to what I would call the PhD or professional level, right? If you look at their coding ability, the latest model we released, Sonnet 3.5, the new or updated version, it gets something like 50% on Sweebench. And Sweebench is an example of a bunch of professional, real-world software engineering tasks. At the beginning of the year, I think the state of the art was 3% or 4%.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So in 10 months, we've gone from 3% to 50% on this task. And I think in another year, we'll probably be at 90%. I mean, I don't know, but might even be less than that. We've seen similar things in graduate level math, physics, and biology from models like OpenAI's 01.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So if we just continue to extrapolate this in terms of skill that we have, I think if we extrapolate the straight curve, within a few years, we will get to these models being above the highest professional level in terms of humans. Now, will that curve continue? You've pointed to and I've pointed to a lot of reasons why, you know, possible reasons why that might not happen.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But if the extrapolation curve continues, that is the trajectory we're on.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, so I want to separate out a couple things, right? So, you know, Anthropic's mission is to kind of try to make this all go well, right? And, you know, we have a theory of change called race to the top, right? Race to the top is about trying to push the other players to do the right thing by setting an example. It's not about being the good guy.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It's about setting things up so that all of us can be the good guy. I'll give a few examples of this. Early in the history of Anthropic, one of our co-founders, Chris Ola, who I believe you're interviewing soon, he's the co-founder of the field of mechanistic interpretability, which is an attempt to understand what's going on inside AI models.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So we had him and one of our early teams focus on this area of interpretability, which we think is good for making models safe and transparent. For three or four years, that had no commercial application whatsoever. It still doesn't today. We're doing some early betas with it, and probably it will eventually. But this is a very, very long research bed and one in which we've
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
built in public and shared our results publicly. And we did this because we think it's a way to make models safer. An interesting thing is that as we've done this, other companies have started doing it as well. In some cases, because they've been inspired by it. In some cases, because they're worried that,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
You know, if if other companies are doing this that look more responsible, they want to look more responsible, too. No one wants to look like the irresponsible actor. And so they adopt this. They adopt this as well. When folks come to Anthropic, interpretability is often a draw. And I tell them the other places you didn't go. Tell them why you came here. And then you.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
You see soon that there's interpretability teams elsewhere as well. And in a way, that takes away our competitive advantage because it's like, oh, now others are doing it as well, but it's good for the broader system. And so we have to invent some new thing that we're doing that others aren't doing as well in the hope is to basically bid up the importance of doing the right thing.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And it's not about us in particular, right? It's not about having one particular good guy. Other companies can do this as well. If they join the race to do this, that's the best news ever, right? It's about kind of shaping the incentives to point upward instead of shaping the incentives to point downward.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Trying to. I mean, I think we're still early in terms of our ability to see things, but I've been surprised at how much we've been able to look inside these systems and understand what we see, right? Unlike with the scaling laws, where it feels like there's some law that's deriving these models to perform better,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
On the inside, the models aren't, you know, there's no reason why they should be designed for us to understand them, right? They're designed to operate. They're designed to work, just like the human brain or human biochemistry. They're not designed for a human to open up the hatch, look inside and understand them.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But we have found, and, you know, you can talk in much more detail about this to Chris, that when we open them up, when we do look inside them, we find things that are surprisingly interesting.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I'm amazed at how clean it's been. I'm amazed at things like induction heads. I'm amazed at things like, you know, that we can, you know, use sparse autoencoders to find these directions within the networks and that the directions correspond to these very clear concepts, right? We demonstrated this a bit with the Golden Gate Bridge quad.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So this was an experiment where we found a direction inside one of the neural network's layers that corresponded to the Golden Gate Bridge. And we just turned that way up. And so we released this model as a demo. It was kind of half a joke for a couple of days, but it was illustrative of the method we developed.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And you could take the Golden Gate, you could take the model, you could ask it about anything, you know, it would be like, you could say, how was your day? And anything you asked, because this feature was activated, it would connect to the Golden Gate Bridge. So it would say, you know, I'm feeling relaxed and expansive, much like the arches of the Golden Gate Bridge, or, you know.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Somehow these interventions on the model where you kind of adjust its behavior somehow emotionally made it seem more human than any other version.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
version of the model strong personality strong strong personality it has these kind of like obsessive interests you know we can all think of someone who's like obsessed with something so it does make it feel somehow a bit more human let's talk about the present let's talk about Claude so this year Claude
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah. So let's go back to March when we first released these three models. So our thinking was different companies produce kind of large and small models, better and worse models.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We felt that there was demand both for a really powerful model, you know, that might be a little bit slower that you'd have to pay more for, and also for fast, cheap models that are as smart as they can be for how fast and cheap, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Whenever you want to do some kind of like, you know, difficult analysis, like if I, you know, I want to write code, for instance, or, you know, I want to brainstorm ideas or I want to do creative writing, I want the really powerful model. But then there's a lot of practical applications in a business sense where it's like, I'm interacting with a website.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I'm doing my taxes or I'm talking to a legal advisor and I want to analyze a contract. Or we have plenty of companies that are just like, I want to do autocomplete on my IDE or something. And for all of those things, you want to act fast and you want to use the model very broadly. So we wanted to serve... that whole spectrum of needs.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Um, so we ended up with this, uh, you know, this kind of poetry theme. And so what's a really short poem. It's a haiku. And so haiku is the small, fast, cheap model that is, you know, was at the time was really surprisingly, surprisingly, uh, intelligent for how fast and cheap it was. Uh, Sonnet is a medium-sized poem, right? A couple paragraphs. And so sonnet was the middle model.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It is smarter, but also a little bit slower, a little bit more expensive. And opus, like a magnum opus is a large work, opus was the largest, smartest model at the time. So that was the original kind of thinking behind it. Yeah. And our thinking then was, well, each new generation of models should shift that trade-off curve.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So when we released Sonnet 3.5, it has the same, roughly the same, you know, cost and speed as the Sonnet 3 model. Uh, but, uh, it, it increased its intelligence to the point where it was smarter than the original Opus 3 model, uh, especially for code, but, but also just in general.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And so now, you know, we've shown results for a Haiku 3.5 and I believe Haiku 3.5, the smallest new model is about as good as Opus 3, the largest old model. So basically, the aim here is to shift the curve, and then at some point, there's going to be an Opus 3.5. Now, every new generation of models has its own thing. They use new data.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Their personality changes in ways that we kind of try to steer but are not fully able to steer. And so there's never quite that exact equivalence where the only thing you're changing is intelligence. We always try and improve other things, and some things change without us knowing or measuring. So it's very much an inexact science.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
In many ways, the manner and personality of these models is more an art than it is a science.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, so there's different processes. There's pre-training, which is, you know, just kind of the normal language model training. And that takes a very long time. That uses, you know, these days, you know,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
tens, you know, tens of thousands, sometimes many tens of thousands of, uh, GPUs or TPUs or tranium, or, you know, what we use different platforms, but, you know, accelerator chips, um, often, often training for months.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Uh, there's then a kind of post-training phase where we do reinforcement learning from human feedback, as well as other kinds of reinforcement learning that, that phase is getting, uh, larger and larger now. And, you know, Often, that's less of an exact science. It often takes effort to get it right.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Models are then tested with some of our early partners to see how good they are, and they're then tested both internally and externally for their safety, particularly for catastrophic and autonomy risks. Uh, so, uh, we do internal testing according to our responsible scaling policy, which I, you know, could talk more about that in detail.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And then we have an agreement with the U S and the UK AI safety Institute, as well as other third-party testers in specific domains to test the models for what are called CBRN risk, chemical, biological, radiological, and nuclear, which are, you know, we don't think that models are
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
pose these risks seriously yet, but every new model we want to evaluate to see if we're starting to get close to some of these more dangerous capabilities. So those are the phases. And then it just takes some time to get the model working in terms of inference and launching it in the API. So there's just just a lot of steps to actually making a model work.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And of course, we're always trying to make the processes as streamlined as possible, right? We want our safety testing to be rigorous, but we want it to be rigorous and to be automatic, to happen as fast as it can without compromising on rigor. Same with our pre-training process and our post-training process. So it's just like building anything else. It's just like building airplanes.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
You want to make them You want to make them safe, but you want to make the process streamlined. And I think the creative tension between those is an important thing in making the models work.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
you would be surprised how much of the challenges of, you know, building these models comes down to, you know, software engineering, performance engineering, you know, you, you know, from the outside, you might think, oh man, we had this Eureka breakthrough, right? You know, this movie with the science, we discovered it, we figured it out.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But, but, but I think, I think all things, even, even, even, you know, incredible discoveries like, They almost always come down to the details and often super, super boring details. I can't speak to whether we have better tooling than other companies. I mean, you know, I haven't been at those other companies, at least not recently, but it's certainly something we give a lot of attention to.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, I think at any given stage, we're focused on improving everything at once. Okay. Um, just, just naturally like there are different teams. Each team makes progress in a particular area in, in, in making a particular, you know, their particular segment of the relay race better. And it's just natural that when we make a new model, we put, we put all of these things in at once.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, preference data from old models sometimes gets used for new models, although, of course, it performs somewhat better when it's, you know, trained on the new models. Note that we have this, you know, constitutional AI method such that we don't only use preference data, we kind of, there's also a post-training process where we train the model against itself.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And there's, you know, new types of post-training the model against itself that are used every day. So it's not just RLHF, it's a bunch of other methods as well. Post-training, I think, you know, is becoming more and more sophisticated.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We observed that as well, by the way. There were a couple very strong engineers here at Anthropic who all previous code models, both produced by us and produced by all the other companies, hadn't really been useful to them. They said, maybe this is useful to a beginner. It's not useful to me. But
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Sonnet 3.5, the original one for the first time, they said, oh my God, this helped me with something that it would have taken me hours to do. This is the first model that has actually saved me time. So again, the waterline is rising. And then I think the new Sonnet has been even better. In terms of what it takes, I mean, I'll just say it's been across the board.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It's in the pre-training, it's in the post-training, it's in various evaluations that we do. We've observed this as well. And if we go into the details of the benchmark, so SWE bench is basically, you know, since you're a programmer, you know, you'll be familiar with like pull requests and, you know, just pull requests are like, you know, like a sort of atomic unit of work.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
You know, you could say, you know, I'm implementing one, I'm implementing one thing. And so SweeBench actually gives you kind of a real world situation where the code base is in the current state and I'm trying to implement something that's described in language.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We have internal benchmarks where we measure the same thing and you say, just give the model free reign to like do anything, run anything, edit anything. How well is it able to complete these tasks? And it's that benchmark that's gone from it can do it 3% of the time to it can do it about 50% of the time.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So I actually do believe that if we get – you can gain benchmarks, but I think if we get to 100% on that benchmark in a way that isn't kind of like over-trained or – or game for that particular benchmark, probably represents a real and serious increase in kind of programming ability.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And I would suspect that if we can get to 90, 95%, that it will represent ability to autonomously do a significant fraction of software engineering tasks.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Not giving an exact date, but as far as we know, the plan is still to have a Cloud 3.5 Opus.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Like Duke Nukem Forever.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
You know, it's only been three months since we released the first Sonnet.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It just tells you about the pace. Yeah. The expectations for when things are going to come out.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Naming is actually an interesting challenge here, right? Because I think a year ago, most of the model was pre-training. And so you could start from the beginning and just say, okay, we're going to have models of different sizes. We're going to train them all together. And, you know, we'll have a family of naming schemes and then we'll put some new magic into them.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And then, you know, we'll have the next, the next generation. The trouble starts already when some of them take a lot longer than others to train, right? That already messes up your time, time a little bit, but yeah, As you make big improvements in pre-training, then you suddenly notice, oh, I can make better pre-trained model, and that doesn't take very long to do.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But, you know, clearly it has the same, you know, size and shape of previous models. Uh, uh, so I think those two together, as well as the timing, timing issues, any kind of scheme you come up with, uh, you know, the reality tends to kind of frustrate that scheme, right? It tends to kind of break out of the breakout of the scheme. It's not like software where you can say, oh, this is like,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
you know, 3.7, this is 3.8. No, you have models with different, different trade-offs. You can change some things in your models. You can train, you can change other things. Some are faster and slower at inference. Some have to be more expensive. Some have to be less expensive. And so I think all the companies have struggled with this.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think we did very, you know, I think, think we were in a good, good position in terms of naming when we had Haiku, Sonnet and Opus. Great start. We're trying to maintain it, but it's not perfect. So we'll try and get back to the simplicity, but just the nature of the field, I feel like no one's figured out naming. It's somehow a different paradigm from normal software. And so...
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
we just, none of the companies have been perfect at it. It's something we struggle with surprisingly much relative to how trivial it is for the grand science of training the models. So from the user side,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah. Yeah. I definitely think this question of there are lots of properties of the models that are not reflected in the benchmarks. I think I think that's that's definitely the case. And everyone agrees. And not all of them are capabilities. Some of them are, you know, models can be polite or brusque. They can be, you know, very reactive or they can ask you questions.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
They can have what feels like a warm personality or a cold personality. They can be boring or they can be very distinctive like Golden Gate Claude was. And we have a whole, you know, we have a whole team kind of focused on, I think we call it Claude character. Amanda leads that team and we'll talk to you about that. But it's still a very inexact science.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And often we find that models have properties that we're not aware of. The fact of the matter is that you can talk to a model 10,000 times and there are some behaviors you might not see. Just like with a human, right? I can know someone for a few months and not know that they have a certain skill or not know that there's a certain side to them. And so I think we just have to get used to this idea.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And we're always looking for better ways of testing our models to demonstrate these capabilities. And And also to decide which are the personality properties we want models to have and which we don't want to have. That itself, the normative question, is also super interesting.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
From Reddit. Oh, boy.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So this actually doesn't apply. This isn't just about Claude. I believe I've seen these complaints for every foundation model produced by a major company. People said this about GPT-4. They said it about GPT-4 Turbo. So a couple things. One, the actual weights of the model, right, the actual brain of the model, that does not change unless we introduce a new model.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
There are just a number of reasons why it would not make sense practically to be randomly substituting in substituting in new versions of the model. It's difficult from an inference perspective, and it's actually hard to control all the consequences of changing the weights of the model.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Let's say you wanted to fine tune the model to be like, I don't know, to like, to say certainly less, which, you know, an old version of Sonnet used to do. You actually end up changing a hundred things as well. So we have a whole process for it. And we have a whole process for
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
modifying the model we do a bunch of testing on it we do a bunch of um like we do a bunch of user testing and early customers so it we both have never changed the weights of the model without without telling anyone and it it wouldn't certainly in the current setup it would not make sense to do that now there are a couple things that we do occasionally do um one is sometimes we run ab tests um
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Um, but those are typically very close to when a model is being, is being, uh, released and for a very small fraction of time. Um, so, uh, you know, like the, you know, the, the day before the new sonnet 3.5, I agree. We should have had a better name. It's clunky to refer to it. Um, there were some comments from people that like, it's got, it's got, it's gotten a lot better.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And that's because, you know, a fraction were exposed to, to an AB test for, for those one or for those one or two days. Um, the other is that occasionally the system prompt will change, um, on the system prompt can have some effects, although it's on, it's unlikely to dumb down models. It's unlikely to make them dumber.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Um, and, and, and, and we've seen that while these two things, which I'm listing to be very complete, um, happen relatively, happen quite infrequently. The complaints for us and for other model companies about the model change, the model isn't good at this, the model got more censored, the model was dumbed down, those complaints are constant.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And so I don't want to say people are imagining it or anything, but the models are, for the most part, not changing. If I were to offer a theory, I think it actually relates to one of the things I said before, which is that Models are very complex and have many aspects to them.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And so often, if I ask the model a question, if I'm like, do task X versus can you do task X, the model might respond in different ways. And so there are all kinds of subtle things that you can change about the way you interact with the model that can give you very different results.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
To be clear, this itself is like a failing by us and by the other model providers that the models are just often sensitive to like small changes in wording. It's yet another way in which the science of how these models work is very poorly developed.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And so if I go to sleep one night and I was talking to the model in a certain way and I slightly change the phrasing of how I talk to the model, I could get different results. So that's one possible way. The other thing is, man, it's just hard to quantify this stuff. It's hard to quantify this stuff. I think people are very excited by new models when they come out.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And then as time goes on, they become very aware of the limitations. So that may be another effect. But that's all a very long-winded way of saying, for the most part, with some fairly narrow exceptions, the models are not changing.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And now I'm like, I can't get this thing to work. This is such a piece of crap.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah. So a couple points on this first. One is like things that people say on Reddit and Twitter or X or whatever it is. There's actually a huge distribution shift between like the stuff that people complain loudly about on social media and what actually kind of like statistically users care about and that drives people to use the models.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
People are frustrated with things like the model not writing out all the code or the model just not being as good at code as it could be, even though it's the best model in the world on code. I think the majority of things are about that, but certainly a kind of vocal minority are you know, kind of raise these concerns, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Are frustrated by the model, refusing things that it shouldn't refuse or like apologizing too much or just having these kind of like annoying verbal tics. The second caveat, and I just want to say this like super clearly because I think it's like, some people don't know it. Others like kind of know it, but forget it.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Like it is very difficult to control across the board how the models behave, right? You cannot just reach in there and say, oh, I want the model to like apologize less. Like you can do that. You can include trading data that says like, oh, the model should like apologize less.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But then in some other situation, they end up being like super rude or like overconfident in a way that's like misleading people. So there are all these trade-offs, right? For example, another thing is if there was a period during which models, ours and I think others as well, were too verbose, right? They would like repeat themselves. They would say too much.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
You can cut down on the verbosity by penalizing the models for just talking for too long. What happens when you do that, if you do it in a crude way, is when the models are coding, sometimes they'll say, rest of the code goes here, right? Because they've learned that that's a way to economize and that they see it.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So that leads the model to be so-called lazy in coding, where they're just like, ah, you can finish the rest of it. It's not because we want to save on compute or because the models are lazy during winter break or any of the other kind of conspiracy theories that have come up.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It's actually – it's just very hard to control the behavior of the model, to steer the behavior of the model in all circumstances at once. You can kind of – there's this whack-a-mole aspect where you push on one thing and like these – these other things start to move as well that you may not even notice or measure.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And so one of the reasons that I, that I care so much about, uh, you know, kind of grand alignment of these AI systems in the future is actually, these systems are actually quite unpredictable. They're actually quite hard to steer and control. Um, and this version we're seeing today of you make one thing better. It makes another thing worse. Uh,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think that's, that's like a present day analog of future control problems in AI systems that we can start to study today. Right. I think, I think that, that, that difficulty in, in steering the behavior and in making sure that if we push an AI system in one direction, it doesn't push it in another direction in some, in some other ways that we didn't want. Uh,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think that's, that's kind of an, that's kind of an early sign of things to come. And if we can do a good job of solving this problem, right. Of like, you ask the model to like, you know, to like make and distribute smallpox and it says no, but it's willing to like help you in your graduate level virology class. Like how do we get both of those things at once? It's hard.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It's very easy to go to one side or the other. And it's a multidimensional problem. And so, uh, I think these questions of shaping the model's personality, I think they're very hard. I think we haven't done perfectly on them. I think we've actually done the best of all the AI companies, but still so far from perfect.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And I think if we can get this right, if we can control the false positives and false negatives in this very kind of controlled present day environment will be much better at doing it for the future when our worry is, will the models be super autonomous? Will they be able to make very dangerous things? Will they be able to autonomously build whole companies and are those companies aligned?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So I think of this present task as both vexing, but also good practice for the future.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So typically we'll have internal model bashings where all of Anthropic, Anthropic is almost a thousand people. You know, people just try and break the model. They try and interact with it various ways. Um, uh, we have a suite of evals, uh, for, you know, oh, is the model refusing in ways that it couldn't?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think we even had a certainly eval because, you know, our, our model, again, one point model had this problem where like it had this annoying tick where it would like respond to a wide range of questions by saying, certainly I can help you with that. Certainly. I would be happy to do that. Certainly this is correct.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Um, uh, and so we had a, like, certainly eval, which is like, how, how often does the model say certainly? Yeah. Uh, uh, but, but look, this is just a whack-a-mole. Like, like what if it switches from certainly to definitely like, uh, uh, so, you know, every time we add a new eval and we're, we're always evaluating for all the old things.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So we have hundreds of these evaluations, but we find that there's no substitute for human interacting with it. And so it's very much like the ordinary product development process. We have like hundreds of people within Anthropic bash the model. Then we do, you know, then we do external AB tests. Sometimes we'll run tests with contractors. We pay contractors to interact with the model.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So you put all of these things together and it's still not perfect. You still see behaviors that you don't quite want to see, right? You know, you still see the model like refusing things that it just doesn't make sense to refuse, right? Um, but I, I, I think trying to, trying to solve this challenge, right.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Trying to stop the model from doing, you know, genuinely bad things that, you know, know what everyone agrees it shouldn't do. Right. You know, everyone, everyone, you know, everyone agrees that, you know, the model shouldn't talk about, you know, I, I don't know, child abuse material. Right. Like everyone agrees the model shouldn't do that.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Uh, but, but at the same time that it doesn't refuse in these dumb and stupid ways. I think drawing that line as finely as possible, approaching perfectly is still a challenge and we're getting better at it every day, but there's a lot to be solved. And again, I would point to that as an indicator of a challenge ahead in terms of steering much more powerful models. Yeah.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Do you think Claude 4.0 is ever coming out? I don't want to commit to any naming scheme because if I say here, we're going to have Claude 4 next year, and then we decide that we should start over because there's a new type of model. I don't want to commit to it. I would expect in a normal course of business that Claude 4 would come after Claude 3.5, but you never know in this wacky field, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Scaling is continuing. There will definitely be more powerful models coming from us than the models that exist today. That is certain. Or if there aren't, we've deeply failed as a company.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
As much as I'm excited about the benefits of these models, and we'll talk about that if we talk about machines of loving grace, I'm worried about the risks, and I continue to be worried about the risks. No one should think that machines of loving grace was me saying I'm no longer worried about the risks of these models. I think they're two sides of the same coin.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
The the power of the models and their ability to solve all these problems in biology, neuroscience, economic development, government, governance and peace, large parts of the economy. Those those come with risks as well. Right. With great power comes great responsibility. Right. That's the two are the two are paired. Things that are powerful can do good things and they can do bad things.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think of those risks as being in several different categories. Perhaps the two biggest risks that I think about, and that's not to say that there aren't risks today that are important, but when I think of the things that would happen on the grandest scale, one is what I call catastrophic misuse. These are misuse of the models in domains like cyber, bio, radiological, nuclear, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Things that could... harm or even kill thousands, even millions of people if they really, really go wrong. These are the number one priority to prevent. And here, I would just make a simple observation, which is that
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
The models, you know, if I look today at people who have done really bad things in the world, I think actually humanity has been protected by the fact that the overlap between really smart, well-educated people and people who want to do really horrific things has generally been small. Like, you know, let's say I'm someone who, you know, I have a PhD in this field. I have a well-paying job.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
There's so much to lose. Why do I want to like, even assuming I'm completely evil, which most people are not, why would such a person risk their life, risk their legacy, their reputation to do something like truly, truly evil? If we had a lot more people like that, the world would be a much more dangerous place. And so my worry is that by being a much more intelligent agent,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
AI could break that correlation. And so I do have serious worries about that. I believe we can prevent those worries, but I think as a counterpoint to machines of loving grace, I wanna say that there's still serious risks. And the second range of risks would be the autonomy risks,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
which is the idea that models might on their own, particularly as we give them more agency than they've had in the past, particularly as we give them supervision over wider tasks like writing whole code bases or someday even effectively operating entire companies, They're on a long enough leash. Are they doing what we really want them to do?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It's very difficult to even understand in detail what they're doing, let alone control it. And like I said, these early signs that it's hard to perfectly draw the boundary between things the model should do and things the model shouldn't do, that, you know, If you go to one side, you get things that are annoying and useless, and you go to the other side, you get other behaviors.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
If you fix one thing, it creates other problems. We're getting better and better at solving this. I don't think this is an unsolvable problem. I think this is a science, like the safety of airplanes or the safety of cars or the safety of drugs. I don't think there's any big thing we're missing. I just think we need to get better at controlling these models.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And so these are the two risks I'm worried about. And our responsible scaling plan, which I'll recognize is a very long-winded answer to your question. I love it. I love it. for its ability to do both of these bad things. So if I were to back up a little bit, I think we have an interesting dilemma with AI systems where they're not yet powerful enough to present these catastrophes.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I don't know that they'll ever prevent these catastrophes. It's possible they won't, but the case for worry, the case for risk is strong enough that we should act now. And they're getting better very, very fast, right? I testified in the Senate that we might have serious bio risks within two to three years. That was about a year ago. Things have preceded a pace.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So we have this thing where it's surprisingly hard to address these risks because they're not here today. They don't exist. They're like ghosts, but they're coming at us so fast because the models are improving so fast. So how do you deal with something that's not here today, doesn't exist, but is coming at us very fast.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So the solution we came up with for that in collaboration with people like the organization Meter and Paul Cristiano is, okay, what you need for that are you need tests to tell you when the risk is getting close. You need an early warning system.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And so every time we have a new model, we test it for its capability to do these CBRN tasks, as well as testing it for, you know, how capable it is of doing tasks autonomously on its own. And in the latest version of our RSP, which we released in the last in the last month or two,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
The way we test autonomy risks is the model, the AI model's ability to do aspects of AI research itself, which when the model, when the AI models can do AI research, they become kind of truly, truly autonomous. And that, you know, that threshold is important for a bunch of other ways. And so what do we then do with these tasks?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
The RSP basically develops what we've called an if-then structure, which is if the models pass a certain capability, then we impose a certain set of safety and security requirements on them. So today's models are what's called ASL2. Models that were, ASL 1 is for systems that manifestly don't pose any risk of autonomy or misuse. So for example, a chess playing bot, Deep Blue would be ASL 1.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It's just manifestly the case that you can't use Deep Blue for anything other than chess. It was just designed for chess. No one's going to use it to like, you know, to conduct a masterful cyber attack or to, you know, run wild and take over the world.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
ASL2 is today's AI systems where we've measured them and we think these systems are simply not smart enough to autonomously self-replicate or conduct a bunch of tasks and also not smart enough to provide meaningful information about CBRN risks and how to build CBRN weapons above and beyond what can be known from looking at Google.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Uh, in fact, sometimes they do provide information, but, but not above and beyond the search engine, but not in a way that can be stitched together. Um, not, not in a way that kind of end to end is dangerous enough. So ASL three is going to be the point at which, uh, the models are helpful enough to enhance the capabilities of non-state actors, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
State actors can already do a lot of, unfortunately, to a high level of proficiency, a lot of these very dangerous and destructive things. The difference is that non-state actors are not capable of it. And so when we get to ASL 3, we'll take special security precautions
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
designed to be sufficient to prevent theft of the model by non-state actors and misuse of the model as it's deployed, will have to have enhanced filters targeted at these particular areas. Cyber, bio, nuclear. Cyber, bio, nuclear, and model autonomy, which is less a misuse risk and more a risk of the model doing bad things itself.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So ASL 4 getting to the point where these models could enhance the capability of a already knowledgeable state actor and or become the main source of such a risk. Like if you wanted to engage in such a risk, the main way you would do it is through a model. And then I think ASL 4 on the autonomy side. It's some amount of acceleration in AI research capabilities with an AI model.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And then ASL 5 is where we would get to the models that are kind of truly capable, that could exceed humanity in their ability to do any of these tasks. And so the point of the if-then structure commitment is basically to say, look, I don't know. I've been working with these models for many years and I've been worried about risk for many years. It's actually kind of dangerous to cry wolf.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It's actually kind of dangerous to say this model is risky. And people look at it and they say this is manifestly not dangerous. Again, it's the... The delicacy of the risk isn't here today, but it's coming at us fast. How do you deal with that? It's really vexing to a risk planner to deal with it. And so this if then structure basically says, look, we don't want to antagonize a bunch of people.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We don't want to harm our own, you know, our kind of own ability to have a place in the conversation by imposing these these. very onerous burdens on models that are not dangerous today. So the if-then, the trigger commitment is basically a way to deal with this. It says you clamp down hard when you can show that the model is dangerous.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And of course, what has to come with that is enough of a buffer threshold that you can you know, you're not at high risk of kind of missing the danger. It's not a perfect framework.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We've had to change it every, you know, we came out with a new one just a few weeks ago and probably going forward, we might release new ones multiple times a year because it's hard to get these policies right, like technically, organizationally, from a research perspective. But that is the proposal. If then commitments and triggers in order to minimize burdens and false alarms now,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
but really react appropriately when the dangers are here.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, so that is hotly debated within the company. We are working actively to prepare ASL 3 security plans security measures as well as ASL 3 deployment measures. I'm not going to go into detail, but we've made a lot of progress on both and we're prepared to be, I think, ready quite soon. I would not be surprised at all if we hit ASL 3 next year.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
There was some concern that we might even hit it this year. That's still possible. That could still happen. It's very hard to say, but I would be very, very surprised if it was like 2030. I think it's much sooner than that.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, I think for ASL 3, it's primarily about security and about... you know, filters on the model relating to a very narrow set of areas when we deploy the model, because at ASL three, the model isn't autonomous yet. And so you don't have to worry about, you know, kind of the model itself behaving in a bad way, even when it's deployed internally.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So I think the ASL-3 measures are, I won't say straightforward, they're rigorous, but they're easier to reason about. I think once we get to ASL-4, we start to have worries about the models being smart enough that they might sandbag tests. They might not tell the truth about tests.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We had some results came out about like sleeper agents and there was a more recent paper about, you know, can the models mislead attempts to, you know, sandbag their own abilities, right? Show them, you know, present themselves as being less capable than they are. And so I think with ASL 4, there's going to be an important component of using other things than just interacting with the models.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
For example, interpretability or hidden chains of thought, where you have to look inside the model and verify via some other mechanism that is not as easily corrupted as what the model says, that the model indeed has some property. So we're still working on ASL 4. One of the
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
properties of the rsp is that we we don't specify asl4 until we've hit asl3 be and and i think that's proven to be a wise decision because even with asl3 it again it's hard to know this stuff in detail and and it we want to take as much time as we can possibly take to get these things right so for asl3 the bad actor will be the humans humans yes and so there's a little bit more um
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
For ASL 4, it's both, I think.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, I mean, of course you can hook up the mechanistic interpretability to the model itself, but then you've kind of lost it as a reliable indicator of the model state. There are a bunch of exotic ways you can think of that it might also not be reliable. Like if the model gets smart enough that it can like, jump computers and read the code where you're looking at its internal state.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We've thought about some of those. I think they're exotic enough. There are ways to render them unlikely. But yeah, generally, you want to preserve mechanistic interpretability as a kind of verification set or test set that's separate from the training process of the model.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Oh, yeah. Yeah. It's actually like, you know, we've seen lots of examples of demagoguery in our life from humans. And, you know, there's a concern that models could do that as well.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, it's actually relatively simple. So Claude has had for a long time since Claude 3 back in March, the ability to analyze images and respond to them with text. The only new thing we added is those images can be screenshots of a computer. And in response, we train the model to give a location on the screen where you can click and or buttons on the keyboard you can press in order to take action.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And it turns out that with actually not all that much additional training, the models can get quite good at that task. It's a good example of generalization. You know, people sometimes say if you get to low Earth orbit, you're like halfway to anywhere, right, because of how much it takes to escape the gravity.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Well, if you have a strong pre-trained model, I feel like you're halfway to anywhere in terms of the intelligence space. And so actually it didn't take all that much to get Claude to do this. And you can just set that in a loop, give the model a screenshot, tell it what to click on, give it the next screenshot, tell it what to click on.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And that turns into a full kind of almost 3D video interaction of the model. And it's able to do all of these tasks, right? You know, we, we showed these demos where it's able to like fill out spreadsheets. It's able to kind of like interact with a website.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It's able to, you know, um, you know, it's able to open all kinds of, you know, programs, different operating systems, windows, Linux, Mac, uh, uh, So, you know, I think all of that is very exciting. I will say, while in theory, there's nothing you could do there that you couldn't have done through just giving the model the API to drive the computer screen, this really lowers the barrier.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And, you know, there's a lot of folks who either, you know, kind of aren't in a position to interact with those APIs or it takes them a long time to do. It's just the screen is just a universal interface that's a lot easier to interact with. And so I expect... Over time, this is going to lower a bunch of barriers. Now, honestly, the current model has – it leaves a lot still to be desired.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And we were honest about that in the blog, right? It makes mistakes. It misclicks. And we were careful to warn people, hey, this thing isn't – you can't just leave this thing to run on your computer for minutes and minutes. You got to give this thing boundaries and guardrails.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And I think that's one of the reasons we released it first in an API form rather than kind of, you know, this kind of just hands it to the consumer and give it control of their computer. But, you know, I definitely feel that it's important to get these capabilities out there. As models get more powerful, we're going to have to grapple with, you know, how do we use these capabilities safely?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
How do we prevent them from being abused? And, you know, I think releasing the model while the capabilities are still limited is very helpful in terms of doing that. You know, I think since it's been released, a number of customers, I think Replit was maybe one of the most quickest to deploy things, have made use of it in various ways. People have hooked up demos for, you know, Windows desktops.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Macs, Linux machines. So yeah, it's been very exciting. I think as with anything else, it comes with new exciting abilities. And then with those new exciting abilities, we have to think about how to make the model safe, reliable, do what humans want them to do. I mean, it's the same story for everything, right? Same thing. It's that same tension.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, I think speaking at a high level, it's our intention to keep investing a lot in, you know, making the model better. Like, I think... We look at some of the benchmarks where previous models were like, oh, I could do it 6% of the time. And now our model would do it 14% or 22% of the time. And yeah, we want to get up to the human level reliability of 80%, 90%, just like anywhere else.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We're on the same curve that we were on with SweBench, where I think I would guess a year from now, the models can do this very, very reliably. But you got to start somewhere.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I mean, it depends what you mean by special and special in general, but I generally think The same kinds of techniques that we've been using to train the current model, I expect that doubling down on those techniques in the same way that we have for code, for models in general, for image input, for voice, I expect those same techniques will scale here as they have everywhere else.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, yeah, no. And we've been very aware of that. Look, my view actually is computer use isn't a fundamentally new capability like the CBRN or autonomy capabilities are. It's more like it kind of opens the aperture for the model to use and apply its existing abilities.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And so the way we think about it, going back to our RSP, is nothing that this model is doing inherently increases the risk from an RSP perspective. But as the models get more powerful, having this capability may make it scarier once it has the cognitive capability to...
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
you know, to do something at the ASL three and ASL four level, this, this, you know, this may be the thing that kind of unbounds it from doing so. So going forward, certainly this modality of interaction is something we have tested for and that we will continue to test for an RSP going forward.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think it's probably better to have, to learn and explore this capability before the model is super, you know, super capable.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, I mean, we've thought a lot about things like spam, CAPTCHA, you know, mass camp. There's all, you know, every, like, one secret I'll tell you, if you've invented a new technology, not necessarily the biggest misuse, but the first misuse you'll see, scams, just petty scams. Yeah. Just, just, just, it's, it's like, it's like a thing as old people scamming each other.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It's, it's this, it's this thing as old as time. And, and, and it's just every time you got to deal with it. It's almost like silly to say, but it's, it's true.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
there are a lot of, like, like I said, like there are a lot of petty criminals in the world and, and, and, you know, it's like every new technology is like a new way for petty, petty criminals to do something, you know, something stupid and malicious. Uh,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, we sandbox during training. So for example, during training, we didn't expose the model to the internet. I think that's probably a bad idea during training because the model can be changing its policy. It can be changing what it's doing and it's having an effect in the real world. You know, in terms of actually deploying the model, right, it kind of depends on the application.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Like, you know, sometimes you want the model to do something in the real world. But of course, you can always put guardrails on the outside, right? You can say, okay, well, you know, this model is not going to move data from my computer. you know, model's not going to move any files from my computer or my web server to anywhere else.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Now, when you talk about sandboxing, again, when we get to ASL 4, none of these precautions are going to make sense there, right? Where when you talk about ASL 4, you're then, the model is being kind of, you know, there's a theoretical worry the model could be smart enough to break it, to kind of break out of any box.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And so there we need to think about mechanistic interpretability, about, you know, if we're going to have a sandbox, it would need to be a mathematically provable sandbox. You know, that's a whole different world than what we're dealing with with the models today.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think it's probably not the right approach. I think the right approach, instead of having something, you know, unaligned that, that like you're trying to prevent it from escaping, I think it's, it's better to just design the model the right way or have a loop where you, you know, you look inside, you look inside the model and you're able to verify properties.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And that gives you a, an opportunity to like iterate and actually get it right. Um, I think, I think containing, uh, containing bad models is, is, is much worse solution than having good models. Yeah.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yes. We ended up making some suggestions to the bill, and then some of those were adopted. And, you know, we felt, I think, quite positively about the bill by the end of that. It did still have some downsides. And, you know, of course, of course, it got vetoed. I think at a high level, I think some of the key ideas behind the bill are, you know, I would say similar to ideas behind our RSPs.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And I think it's very important that some jurisdiction, whether it's California or the federal government and or other other countries and other states, passes some regulation like this. And I can talk through why I think that's so important. So I feel good about our RSP. It's not perfect.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It needs to be iterated on a lot, but it's been a good forcing function for getting the company to take these risks seriously, to put them into product planning, to really make them a central part of work at Anthropic and to make sure that all of a thousand people, and it's almost a thousand people now at Anthropic, understand that this is one of the highest priorities of the company, if not the highest priority.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But One, there are still some companies that don't have RSP-like mechanisms. Like OpenAI, Google did adopt these mechanisms a couple months after Anthropic did. But there are other companies out there that don't have these mechanisms at all. And so if some companies adopt these mechanisms and others don't,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
don't, it's really going to create a situation where some of these dangers have the property that it doesn't matter if three out of five of the companies are being safe. If the other two are being unsafe, it creates this negative externality. And I think the lack of uniformity is not fair to those of us who have put a lot of effort into being very thoughtful about these procedures.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
The second thing is I don't think you can trust these companies to adhere to these voluntary plans on their own, right? I like to think that Anthropic will. We do everything we can that we will. Our RSP is checked by our long-term benefit trust. So we do everything we can to adhere to our own RSP.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Um, but you know, you hear lots of things about various companies saying, oh, they said they would do, they said they would give this much compute and they didn't, they said they would do this thing and they didn't. Um, you know, I don't, I don't think it makes sense to, you know, to, to, to, you know, litigate particular things that companies have done.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But I think this, this broad principle that like, if there's nothing watching over them, there's nothing watching over us as an industry. there's no guarantee that we'll do the right thing. And the stakes are very high.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And so I think it's important to have a uniform standard that everyone follows and to make sure that simply that the industry does what a majority of the industry has already said is important and has already said that they definitely will do. Some people, I think there's a class of people who are against regulation on principle. I understand where that comes from.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
If you go to Europe and you see something like GDPR, you see some of the other stuff that they've done, some of it's good, but some of it is really unnecessarily burdensome. And I think it's fair to say really has slowed innovation. And so I understand where people are coming from on priors. I understand why people start from that position. But again, I think AI is different.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
If we go to the very serious risks of autonomy and misuse that I talked about just a few minutes ago, I think that those are unusual and they weren't an unusually strong response. And so I think it's very important. Again, we need something that everyone can get behind. You know, I think one of the issues with SB 1047, especially the original version of it, was it.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It had a bunch of the structure of RSPs, but it also had a bunch of stuff that was either clunky or that just would have created a bunch of burdens, a bunch of hassle, and might even have missed the target in terms of addressing the risks. You don't really hear about it on Twitter. You just hear about kind of, you know, people are, people are cheering for any regulation.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And then the folks who are against make up these often quite intellectually dishonest arguments about how, you know, it, you know, it'll make us move away from California. Bill, Bill doesn't apply.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
If you're headquartered in California, Bill only applies if you do business in California or that it would damage the open source ecosystem or that it would, you know, it would cause, cause all of these things. I, I think those were mostly nonsense, but there are better arguments against regulation.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
There's one guy, Dean Ball, who's really, you know, I think a very scholarly, scholarly analyst who looks at what happens when a regulation is put in place and ways that they can kind of get a life of their own or how they can be poorly designed. And so our interest has always been, we do think there should be regulation in this space, but
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We want to be an actor who makes sure that that regulation is something that's surgical, that's targeted at the serious risks, and is something people can actually comply with. Because something I think the advocates of regulation don't understand as well as they could is if we get something in place that is –
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
that's poorly targeted, that wastes a bunch of people's time, what's going to happen is people are going to say, see, these safety risks, this is nonsense. I just had to hire 10 lawyers to fill out all these forms. I had to run all these tests for something that was clearly not dangerous. And after six months of that, there will be a groundswell. And
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And we'll end up with a durable consensus against regulation. And so I think the worst enemy of those who want real accountability is badly designed regulation. We need to actually get it right. And if there's one thing I could say to the advocates, it would be that I want them to understand this dynamic better. And we need to be really careful. And we need to talk to people who actually have
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
who actually have experience seeing how regulations play out in practice. And the people who have seen that understand to be very careful. If this was some lesser issue, I might be against regulation at all. But what I want the opponents to understand is that the underlying issues are actually serious.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
They're not something that I or the other companies are just making up because of regulatory capture. They're not sci-fi fantasies. They're not any of these things. Every time we have a new model, every few months, we measure the behavior of these models, and they're getting better and better at these concerning tasks, just as they are getting better and better at...
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
good, valuable, economically useful tasks. And so I would just love it if some of the former, I think SB 1047 was very polarizing. I would love it if some of the most reasonable opponents and some of the most reasonable opponents
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
uh proponents uh would sit down together and you know i think i think that you know the different the different ai companies um you know anthropic was the the only ai company that you know felt positively in a very detailed way i think elon tweeted uh tweeted briefly something positive but you know some of the some of the big ones like google open ai meta microsoft were
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
were pretty staunchly against. So I would really like is if some of the key stakeholders, some of the most thoughtful proponents and some of the most thoughtful opponents would sit down and say, how do we solve this problem in a way that the proponents feel
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
brings a real reduction in risk and that the opponents feel that it is not hampering the industry or hampering innovation any more necessary than it needs to. And I think for whatever reason that things got too polarized and those two groups didn't get to sit down in the way that they should. And I feel urgency. I really think we need to do something in 2025.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
You know, if we get to the end of 2025 and we've still done nothing about this, then I'm going to be worried. I'm not worried yet because, again, the risks aren't here yet. But I think time is running short. And come up with something surgical, like you said. Yeah, yeah, yeah, exactly. And we need to get away from this crisis. This intense pro-safety versus intense anti-regulatory rhetoric, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It's turned into these flame wars on Twitter and nothing good is going to come of that.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, so I was at OpenAI for roughly five years. For the last, I think it was a couple of years, I was vice president of research there. Probably myself and Ilya Sutskiver were the ones who really kind of set the research direction.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
around 2016 or 2017, I first started to really believe in or at least confirm my belief in the scaling hypothesis when Ilya famously said to me, the thing you need to understand about these models is they just want to learn. The models just want to learn. And again, sometimes there are these one sentences, these Zen cones that you hear them and you're like, ah, That explains everything.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
That explains like a thousand things that I've seen. And then I, you know, ever after I had this visualization in my head of like, you optimize the models in the right way, you point the models in the right way. They just want to learn. They just want to solve the problem regardless of what the problem is.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Get out of their way. Yeah. Don't impose your own ideas about how they should learn. And, you know, this was the same thing as Rich Sutton put out in The Bitter Lesson or Gern put out in The Scaling Hypothesis. You know, I think generally the dynamic was, you know, I got this kind of inspiration from Ilya and from others, folks like Alec Radford, who did the original GPT-1. And then...
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Ran really hard with it, me and my collaborators on GPT-2, GPT-3, RL from human feedback, which was an attempt to kind of deal with the early safety and durability, things like debate and amplification, heavy on interpretability. So again, the combination of safety plus scaling, probably 2018, 2019, 2020, those were kind of the years when myself and my collaborators probably –
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
You know, many of whom became co-founders of Anthropic kind of really had a vision and like drove the direction. Why'd you leave?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah. So, look, I'm going to put things this way. And, you know, I think it ties to the race to the top. Right. Which is, you know, in my time at OpenAI, what I come to see as I'd come to appreciate the scaling hypothesis and as I come to appreciate kind of the importance of safety along with the scaling hypothesis. The first one, I think, you know, OpenAI was getting on board with.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
The second one, in a way, had always been part of OpenAI's messaging. But, you know, over many years of the time that I spent there, I think I had a particular vision of how these, how we should handle these things, how we should be brought out in the world, the kind of principles that the organization should have.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And look, I mean, there were like many, many discussions about like, you know, should the org do, should the company do this? Should the company do that? Like there's a bunch of misinformation out there. People say like, we left because we didn't like the deal with Microsoft. False.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Although, you know, it was like a lot of discussion, a lot of questions about exactly how we do the deal with Microsoft. We left because we didn't like commercialization. That's not true. We built GPT-3, which was the model that was commercialized. I was involved in commercialization. It's more, again, about how do you do it? Like... civilization is going down this path to very powerful AI.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
What's the way to do it that is cautious, straightforward, honest, that builds trust in the organization and in individuals? How do we get from here to there? And how do we have a real vision for how to get it right? How can safety not just be something we say because it helps with recruiting?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And, you know, I think at the end of the day, if you have a vision for that, forget about anyone else's vision. I don't want to talk about anyone else's vision. If you have a vision for how to do it, you should go off and you should do that vision. It is incredibly unproductive to try and argue with someone else's vision. You might think they're not doing it the right way.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
You might think they're dishonest. Who knows? Maybe you're right. Maybe you're not. Um, uh, but, uh, what, what you should do is you should take some people you trust and you should go off together and you should make your vision happen. And if your vision is compelling, if you can make it appeal to people, some, you know,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
If you can make a company that's a place people want to join, that engages in practices that people think are reasonable while managing to maintain its position in the ecosystem at the same time, if you do that, people will copy it.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And the fact that you are doing it, especially the fact that you're doing it better than they are, causes them to change their behavior in a much more compelling way than if they're your boss and you're arguing with them. I don't know how to be any more specific about it than that, but I think it's generally very unproductive to try and get someone else's vision to look like your vision.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It's much more productive to go off and do a clean experiment and say, this is our vision. This is how we're going to do things. Your choice is you can ignore us, you can reject what we're doing, or you can start to become more like us, and imitation is the sincerest form of flattery.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And, you know, that plays out in the behavior of customers, that pays out in the behavior of the public, that plays out in the behavior of where people choose to work. And again, again, at the end, it's not about one company winning or another company winning if...
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
If we or another company are engaging in some practice that, you know, people people find genuinely appealing and I want it to be in substance, not just not just in appearance. And, you know, I think I think researchers are sophisticated and they look at substance. and then other companies start copying that practice and they win because they copied that practice, that's great. That's success.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
That's like the race to the top. It doesn't matter who wins in the end, as long as everyone is copying everyone else's good practices, right? One way I think of it is like, the thing we're all afraid of is the race to the bottom, right? And the race to the bottom doesn't matter who wins because we all lose, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Like, you know, in the most extreme world, we make this autonomous AI that, you know, the robots enslave us or whatever, right? I mean, that's half joking, but you know, that is the most extreme thing that could happen. Then it doesn't matter which company was ahead. If instead you create a race to the top where people are competing to engage in good practices, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
then at the end of the day, it doesn't matter who ends up winning. It doesn't even matter who started the race to the top. The point isn't to be virtuous. The point is to get the system into a better equilibrium than it was before. And individual companies can play some role in doing this. Individual companies can help to start it, can help to accelerate it.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And frankly, I think individuals at other companies have done this as well, right? The individuals that when we put out an RSP, react by pushing harder to get something similar done at other companies. Sometimes other companies do something that's like, we're like, oh, it's a good practice. We think that's good. We should adopt it too. The only difference is, you know, I think we are
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We try to be more forward-leaning. We try and adopt more of these practices first and adopt them more quickly when others invent them. But I think this dynamic is what we should be pointing at. And I think it abstracts away the question of which company is winning, who trusts who. I think all these questions of drama are important. are profoundly uninteresting.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And the thing that matters is the ecosystem that we all operate in and how to make that ecosystem better because that constrains all the players.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Look, I'm sure we've made plenty of mistakes along the way. The perfect organization doesn't exist. It has to deal with the imperfection of a thousand employees. It has to deal with the imperfection of our leaders, including me. It has to deal with the imperfection of the people we've put to oversee the imperfection of the leaders, like the board and the long-term benefit trust.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So I can only describe it as it relates to kind of my own experience, but I've been in the AI field for about 10 years. And it was something I noticed very early on. So I first joined the AI world when I was working at Baidu with Andrew Ng in late 2014, which is almost exactly 10 years ago now. And the first thing we worked on was speech recognition systems.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It's all a set of imperfect people trying to aim imperfectly at some ideal that will never perfectly be achieved. Um, that's what you sign up for. That's what it will always be. But, uh, uh, imperfect doesn't mean you just give up. There's better and there's worse. And hopefully, hopefully we can begin to build.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We can do well enough that we can begin to build some practices that the whole industry engages in. And then, you know, my guess is that multiple of these companies will be successful. Anthropic will be successful. These other companies, like ones I've been at the past will also be successful because And some will be more successful than others.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
That's less important than, again, that we align the incentives of the industry. And that happens partly through the race to the top, partly through things like RSP, partly through, again, selected surgical regulation.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
This is one of these statements that's like more true every, every, every month. Every month I see this statement is more true than I did the month before. So if I were to do a thought experiment, let's say you have a team of 100 people that are super smart, motivated and aligned with the mission, and that's your company.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Or you can have a team of a thousand people where 200 people are super smart, super aligned with the mission. And then like 800 people are, let's just say you pick 800 like random big tech employees. Which would you rather have? The talent mass is greater in the group of 1,000 people. You have even a larger number of incredibly talented, incredibly aligned, incredibly smart people.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But the issue is just that if... Every time someone super talented looks around, they see someone else super talented and super dedicated. That sets the tone for everything, right? That sets the tone for everyone is super inspired to work at the same place. Everyone trusts everyone else. If you have a thousand or 10,000 people and things have really regressed, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
You are not able to do selection and you're choosing random people. What happens is then you need to put a lot of processes and a lot of guardrails in place, right? just because people don't fully trust each other. You have to adjudicate political battles. Like there are so many things that slow down the org's ability to operate. And so we're nearly a thousand people.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And, you know, we've, we've, we've tried to make it so that as large a fraction of those thousand people as possible are like, super talented, super skilled. It's one of the reasons we've slowed down hiring a lot in the last few months. We grew from 300 to 800, I believe, I think in the first seven, eight months of the year. And now we've slowed down.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We're at like, you know, the last three months we went from 800 to 900, 950, something like that. Don't quote me on the exact numbers, but I think there's an inflection point around a thousand and we want to be much more careful how we grow. Early on, and now as well, we've hired a lot of physicists. Theoretical physicists can learn things really fast.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Even more recently, as we've continued to hire that, we've really had a high bar on both the research side and the software engineering side, have hired a lot of senior people, including folks who used to be at other companies in this space. And we've just continued to be very selective.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It's very easy to go from a hundred to a thousand and a thousand to 10,000 without paying attention to making sure everyone has a unified purpose. It's so powerful. If your company consists of a lot of different fiefdoms that all want to do their own thing, they're all optimizing for their own thing. It's very hard to get anything done.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And in those days, I think deep learning was a new thing. It had made lots of progress, but everyone was always saying, we don't have the algorithms we need to succeed. You know, we're not, we're only matching a tiny, tiny fraction. There's so much we need to kind of discover algorithmically. We haven't found the picture of how to match the human brain.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But if everyone sees the broader purpose of the company, if there's trust and there's dedication to doing the right thing, that is a superpower. That in itself, I think, can overcome almost every other disadvantage.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah. I think the number one quality, especially on the research side, but really both, is open-mindedness. Sounds easy to be open-minded, right? You're just like, oh, I'm open to anything. But if I think about my own early history in the scaling hypothesis... Um, I was seeing the same data others were seeing.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I don't think I was like a better programmer or better at coming up with research ideas than any of the hundreds of people that I worked with. Um, in some ways, in some ways I was worse. Um, uh, you know, like I've, I've never liked, you know, precise programming of like, you know, finding the bug, writing the GPU kernels.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Like I could point you to a hundred people here who are better, who are better at that than I am. Um, but, but the, the thing that, that, that I think I did have that was different was that I was just willing to look at something with new eyes, right? People said, oh, you know, we don't have the right algorithms yet. We haven't come up with the right, the right way to do things.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And I was just like, oh, I don't know. Like, This neural net has 30 million parameters. What if we gave it 50 million instead? Let's plot some graphs. That basic scientific mindset of, oh man, I see some variable that I could change. What happens when it changes? Let's try these different things and create a graph. This was the simplest thing in the world.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Change the number of... This wasn't like... PhD level experimental design. This was like simple and stupid. Like anyone could have done this if you just told them that it was important. It's also not hard to understand. You didn't need to be brilliant to come up with this.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But you put the two things together and some tiny number of people, some single digit number of people have driven forward the whole field by realizing this.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
uh and and it's you know it's often like that if you look back at the discovery you know the discoveries in in history they're they're often like that and so this this open-mindedness and this willingness to see with new eyes that often comes from being newer to the field often experience is a disadvantage for this that is the most important thing it's very hard to look for and test for but i think i think it's the most important thing because when you when you find something
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
some really new way of thinking about things. When you have the initiative to do that, it's absolutely transformative.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Uh, and when, you know, in some ways it was fortunate. I was kind of, you know, you can have almost beginner's luck, right? I was like a newcomer to the field. And, you know, I looked at the neural net that we were using for speech, the recurrent neural networks. And I said, I don't know, what if you make them bigger and give them more layers and
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It's another example of this. Like some of the early work in mechanistic interpretability, so simple. It's just no one thought to care about this question before.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think my number one piece of advice is to just start playing with the models. This was actually, I worry a little. This seems like obvious advice now. I think three years ago, it wasn't obvious. And people started by, oh, let me read the latest reinforcement learning paper. Let me kind of... No, I mean, that was really the... I mean, you should do that as well.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But now, with wider availability of models and APIs, people are doing this more. But I think... I think just experiential knowledge. These models are new artifacts that no one really understands. And so getting experience playing with them. I would also say, again, in line with the, like, do something new, think in some new direction. Like, there are all these things that haven't been explored.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Like, for example, mechanistic interpretability is still very new. It's probably better to work on that than it is to work on new model architectures because it's, you know, it's more popular than it was before. There are probably like a hundred people working on it, but there aren't like 10,000 people working on it. And it's, it's just this, this, this fertile area for study. Like, like it.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
You know, there's so much like low-hanging fruit. You can just walk by and, you know, you can just walk by and you can pick things. And the only reason, for whatever reason, people aren't interested in it enough. I think there are some things around... long, long horizon learning and long horizon tasks where there's a lot to be done.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think evaluations are still, we're still very early in our ability to study evaluations, particularly for dynamic systems acting in the world. I think there's some stuff around multi-agent, um, skate where the puck is going is my, is my advice. And you don't have to be brilliant to think of it.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Like all the things that are going to be exciting in five years, like in people even mentioned them as like, you know, conventional wisdom, but like, it's, it's just somehow there's this barrier that people don't, people don't double down as much as they could, or they're afraid to do something. That's not the popular thing.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I don't know why it happens, but like getting over that barrier is that's the, my number one piece of advice. Yeah.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And what if you scale up the data along with this, right? I just saw these as like independent dials that you could turn. And I noticed that the model started to do better and better as you gave them more data, as you made the models larger, as you trained them for longer.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah. Um, I mean, uh, so first of all, we're not perfectly able to measure that ourselves. Um, uh, you know, when you see some, some great character ability, sometimes it's hard to tell whether it came from pre-training or post-training, uh, we've developed ways to try and distinguish between those two, but they're not perfect.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
You know, the second thing I would say is, you know, it's when there is an advantage and I think we've been pretty good at in general, in general at RL, perhaps, perhaps the best, although, although I don't know, cause I don't see what goes on inside other companies. Uh, Usually it isn't, oh my God, we have this secret magic method that others don't have, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Usually it's like, well, you know, we got better at the infrastructure so we could run it for longer, or, you know, we were able to get higher quality data, or we were able to filter our data better, or we were able to, you know, combine these methods in practice. It's usually some boring matter of kind of practice and tradecraft, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Um, so, you know, when I think about how to do something special in terms of how we train these models, both pre-training, but even more so post-training, um, you know, I, I really think of it a little more again as like designing airplanes or cars. Like, you know, it's not just like, oh man, I have the blueprint.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Like maybe that makes you make the next airplane, but like there's some, there's some cultural trade craft of how we think about the design process that I think is more important than, than, you know, than, than any particular gizmo we're able to invent.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
If I go back to like the scaling hypothesis, one of the ways to skate the scaling hypothesis is if you train for X and you throw enough compute at it, then you get X. And so RLHF is good at doing what humans want the model to do, or at least to state it more precisely, doing what humans who look at the model for a brief period of time and consider different possible responses, what they prefer as the response.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And I didn't measure things precisely in those days, but along with colleagues, we very much got the informal sense that the more data and the more compute and the more training you put into these models, the better they perform. And so initially my thinking was, hey, maybe that is just true for speech recognition systems, right? Maybe that's just one particular quirk, one particular area.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
which is not perfect from both the safety and capabilities perspective in that humans are often not able to perfectly identify what the model wants and what humans want in the moment may not be what they want in the long term. So there's a lot of subtlety there, but the models are good at producing what the humans in some shallow sense want.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And it actually turns out that you don't even have to throw that much compute at it because of another thing, which is this thing about a strong pre-trained model being halfway to anywhere. So once you have the pre-trained model, you have all the representations you need to get the model where you want it to go. So do you think our LHF
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I don't think it makes the model smarter. I don't think it just makes the model appear smarter. It's like RLHF bridges the gap between the human and the model, right? I could have something really smart that can't communicate at all, right? We all know people like this, people who are really smart, but you can't understand what they're saying. So I think RLHF just bridges that gap.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think it's not the only kind of RL we do. It's not the only kind of RL that will happen in the future. I think RL has the potential to make models smarter, to make them reason better, to make them operate better, to make them develop new skills even. And perhaps that could be done even in some cases with human feedback.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But the kind of RLHF we do today mostly doesn't do that yet, although we're very quickly starting to be able to.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It also increases, what was this word in Leopold's essay, unhobbling, where basically the models are hobbled and then you do various trainings to them to unhobble them. So I like that word because it's like a rare word. So I think RLHF unhobbles the models in some ways. And then there are other ways where a model hasn't yet been unhobbled and needs to unhobble.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
At the present moment, it is still the case that pre-training is the majority of the cost. I don't know what to expect in the future, but I could certainly anticipate a future where post-training is the majority of the cost.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I don't think you can scale up humans enough to get high quality. Any kind of method that relies on humans and uses a large amount of compute, it's going to have to rely on some scaled supervision method like debate or iterated amplification or something like that.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yes. So this was from two years ago. The basic idea is, so we describe what RLHF is. You have a model. And it, you know, spits out two, you know, like you just sample from it twice. It spits out two possible responses and you're like human, which response do you like better? Or another variant of it is rate this response on a scale of one to seven.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So that's hard because you need to scale up human interaction. And it's very implicit, right? I don't have a sense of what I want the model to do. I just have a sense of like what this average of a thousand humans wants the model to do. So two ideas. One is, could the AI system itself decide which response is better, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Could you show the AI system these two responses and ask which response is better? And then second, well, what criterion should the AI use? And so then there's this idea, could you have a single document, a constitution, if you will, that says, these are the principles the model should be using to respond. And the AI system reads those principles
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think it wasn't until 2017 when I first saw the results from GPT-1. that it clicked for me that language is probably the area in which we can do this. We can get trillions of words of language data. We can train on them. And the models we were trained in those days were tiny.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
it reads those principles as well as reading the environment and the response. And it says, well, how good did the AI model do? It's basically a form of self-play. You're kind of training the model against itself. And so the AI gives the response and then you feed that back into what's called the preference model, which in turn feeds the model to make it better.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So you have this triangle of like the AI, the preference model and the improvement of the AI itself.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, yeah. It's something both the human and the AI system can read. So it has this nice kind of translatability or symmetry. You know, in practice, we both use a model constitution and we use RLHF and we use some of these other methods. So it's turned into one tool in a toolkit that both reduces the need for RLHF and increases the value we get from using each data point of RLHF.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It also interacts in interesting ways with kind of future reasoning type RL methods. So it's one tool in the toolkit, but I think it is a very important tool.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah. So I'll give like a practical answer and a more abstract answer. I think the practical answer is like, look, in practice, models get used by all kinds of different like customers. Right. And so you can have this idea where, you know, the model can can have specialized rules or principles. You know, we fine tune versions of models.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We've talked about doing it explicitly, having special principles that people can build into the models. So from a practical perspective, the answer can be very different from different people. A customer service agent behaves very differently from a lawyer and obeys different principles. But I think at the base of it, there are specific principles that models, you know, have to obey.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think a lot of them are things that people would agree with. Everyone agrees that, you know, we don't, you know, we don't want models to present these CBRN risks. I think we can go a little further and agree with some basic principles of democracy and the rule of law. Beyond that, it gets, you know, very uncertain.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And there our goal is generally for the models to be more neutral, to not espouse a particular point of view and, you know, more just be kind of like wise agents or advisors that will help you think things through and will, you know, present present possible considerations. But, you know, don't express, you know, strong or specific opinions.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
You could train them on one to eight GPUs, whereas, you know, now we train jobs on tens of thousands, soon going to hundreds of thousands of GPUs. And so when I saw those two things together, and, you know, there were a few people like Ilya Sutskiver, who you've interviewed, who had somewhat similar views, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah. So I think that's a pretty useful direction. Again, it has a lot in common with constitutional AI. So again, another example of like a race to the top, right? We have something that's like, we think... a better and more responsible way of doing things. It's also a competitive advantage. Then others discover that it has advantages and then start to do that thing.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We then no longer have the competitive advantage, but it's good from the perspective that now everyone has adopted a
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
positive practice that others were not adopting and so our response to that as well looks like we need a new competitive advantage in order to keep driving this race upwards um so that's that's how i generally feel about that i also think every implementation of these things is different so you know there were some things in the model spec that were not in constitutional ai and so you know we you know we can always we can always adopt those things or you know at least learn from them um so again i think this is an example of like the positive dynamic that uh
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
that I think we should all want the field to have.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It is rather long.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Oh, yeah. I'm fully expecting to definitely be wrong about all the details. I might be just spectacularly wrong about the whole thing and people will laugh at me for years. That's just how the future works.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
vision of this essay and um what key takeaways that people have yeah i have spent a lot of time in anthropic i spent a lot of effort on like you know how do we address the risks of ai right how do we think about those risks like we're trying to do a race to the top you know what that requires us to build all these capabilities and the capabilities are cool but you know
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
you know, we're, we're, we're like a big part of what we're trying to do is like, is like address the risks and the justification for that is like, well, you know, all these positive things, you know, the market is this very healthy organism, right? It's going to produce all the positive things, the risks. I don't know. We might mitigate them. We might not.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And so we can have more impact by trying to mitigate the risks. But I noticed that one flaw in that way of thinking is, And it's not a change in how seriously I take the risks. It's maybe a change in how I talk about them. Is that, you know, no matter how kind of logical or rational that line of reasoning that I just gave might be.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
He might have been the first one, although I think a few people came to similar views around the same time, right? There was, you know, Rich Sutton's bitter lesson. There was, Goren wrote about the scaling hypothesis. But I think somewhere between 2014 and 2017 was when it really clicked for me, when I really got conviction that, hey, we're going to be able to do these incredible
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
if you kind of only talk about risks, your brain only thinks about risks. And so I think it's actually very important to understand what if things do go well? And the whole reason we're trying to prevent these risks is not because we're afraid of technology, not because we want to slow it down. It's because...
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
if we can get to the other side of these risks, right, if we can run the gauntlet successfully to, you know, to put it in stark terms, then on the other side of the gauntlet are all these great things. And these things are worth fighting for. And these things can really inspire people. And,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think I imagine because, look, you have all these investors, all these VCs, all these AI companies talking about all the positive benefits of AI. But as you point out, it's weird. There's actually a dearth of really getting specific about it. There's a lot of like...
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Random people on Twitter, like posting these kind of like gleaming cities and this, this just kind of like vibe of like grind, accelerate harder, like kick out the decel, you know, it's, it's just this very, this very like aggressive ideological. Then you're like, well, what are you – what are you actually excited about?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And so I figured that – I think it would be interesting and valuable for someone who's actually coming from the risk side to try and really – Make a try at at explaining, explaining, explain what the benefits are, both because I think it's something we can all get behind. And I want people to understand. I want them to really understand that this isn't this isn't doomers versus accelerationists.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
This this is. that if you have a true understanding of where things are going with AI, and maybe that's the more important axis, AI is moving fast versus AI is not moving fast, then you really appreciate the benefits and you really, you want humanity, our civilization to seize those benefits, but you also get very serious about anything that could derail them.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Maybe we're stuck with the terms and my efforts to change them are futile. It's admirable. I'll tell you what else I don't. This is like a pointless semantic point, but I, I keep talking about it. I'm just going to do it once more. Um, uh, I, I think it's a little like... Let's say it was like 1995 and Moore's Law is making the computers faster.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And for some reason, there had been this verbal tick that everyone was like, well, someday we're going to have supercomputers. And supercomputers are going to be able to do all these things that once we have supercomputers, we'll be able to sequence the genome. We'll be able to do other things. And so one, it's true. The computers are getting faster.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And as they get faster, they're going to be able to do all these great things. But there's like... There's no discrete point at which you had a supercomputer and previous computers were not. Supercomputer is a term we use, but it's a vague term to just describe computers that are faster than what we have today.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
There's no point at which you pass a threshold and you're like, oh my God, we're doing a totally new type of computation. I feel that way about AGI. There's just a smooth exponential. If by AGI you mean AI is getting better and better and gradually it's
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
incredibly wide cognitive tasks if we just scale up the models. And at every stage of scaling, there are always arguments. And when I first heard them, honestly, I thought, probably I'm the one who's wrong. And all these experts in the field are right. They know the situation better than I do. There's the Chomsky argument about you can get syntactics, but you can't get semantics.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
going to do more and more of what humans do until it's going to be smarter than humans, and then it's going to get smarter even from there, then yes, I believe in AGI. But if AGI is some discrete or separate thing, which is the way people often talk about it, then it's kind of a meaningless buzzword.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah. Yeah. I mean, you, you might imagine from outside the field that like, there's only one of these, right. That like you made it, you've only made one, but the truth is that like the scale up is very quick. Like we, we do this today. We make a model and then we deploy thousands, maybe tens of thousands of instances of it. I think by the time, uh,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
you know, certainly within two to three years, whether we have these super powerful AIs or not, clusters are going to get to the size where you'll be able to deploy millions of these and they'll be, you know, faster than humans. And so if your picture is, oh, we'll have one and it'll take a while to make them, my point there was, no, actually you have millions of them right away.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah. So yeah, let's, let's describe the extreme. So like one, one extreme would be, well, look, um, you know, uh, if we look at kind of evolutionary history, like there was this big acceleration where, you know, for hundreds of thousands of years, we just had like. single-celled organisms, and then we had mammals, and then we had apes, and then that quickly turned to humans.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Humans quickly built industrial civilization. And so this is going to keep speeding up. And there's no ceiling at the human level. Once models get much, much smarter than humans, they'll get really good at building the next models. And if you write down a simple differential equation, this is an exponential. And so what's going to happen is that
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
uh models will build faster models models will build faster models and those models will build you know nanobots that can like take over the world and produce much more energy than you could produce otherwise and and so if you just kind of like solve this abstract differential equation then like five days after we you know we build the first ai that's more powerful than humans then then uh you know like the world will be filled with these ais and every possible technology that could be invented like will be invented um i'm caricaturing this a little bit um
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But I, you know, I think that's one extreme. And the reason that I think that's not the case is that one, I think they just neglect like the laws of physics. Like it's only possible to do things so fast in the physical world. Like some of those loops go through, you know, producing faster hardware. It takes a long time to produce faster hardware. Things take a long time.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
There was this idea, oh, you can make a sentence make sense, but you can't make a paragraph make sense. You know, we're going to run out of data or the data isn't high quality enough or models can't reason. And each time, every time, we manage to either find a way around or scaling just is the way around. Sometimes it's one, sometimes it's the other.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
There's this issue of complexity. Like I think no matter how smart you are, like, People talk about, oh, we can make models of biological systems that'll do everything in biological systems. Look, I think computational modeling can do a lot. I did a lot of computational modeling when I worked in biology. But there are a lot of things that you can't predict.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
They're complex enough that just iterating, just running the experiment is going to beat any modeling, no matter how smart the system doing the modeling is.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, I think, well, the modeling's going to be hard, and getting the model to match the physical world is going to be hard. All right, so he does have to interrupt the physical world to verify. Yeah, yeah, yeah. But it's just, you know, you just look at even the simplest problems. Like, you know, I think I talk about, like, you know, the three-body problem or simple chaotic prediction, like...
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
you know, or, or like predicting the economy, it's really hard to predict the economy two years out. Like maybe the case is like, you know, normal, you know, humans can predict what's going to happen in the economy next quarter. Or they can't really do that.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Maybe, maybe a AI system that's, you know, a zillion times smarter can only predict it out a year or something instead of, instead of, you know, you have these kinds of exponential increase in computer intelligence for linear increase in, in, in ability to predict. Same with, again, like, you know, biological molecules interacting.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
You don't know what's going to happen when you perturb a complex system. You can find simple parts in it. If you're smarter, you're better at finding these simple parts. And then I think human institutions. Human institutions are just... are really difficult. It's been hard to get people.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I won't give specific examples, but it's been hard to get people to adopt even the technologies that we've developed, even ones where the case for their efficacy is very, very strong. People have concerns. They think things are conspiracy theories. It's been very difficult. It's also been very difficult to get you know, very simple things through the regulatory system, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think, you know, and, you know, I don't want to disparage anyone who, you know, works in regulatory systems of any technology. There are hard trade-offs they have to deal with. They have to save lives. But the system as a whole, I think, makes some obvious trade-offs that are very far from maximizing human welfare. And so if we bring AI systems into this, you know,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
into these human systems, often the level of intelligence may just not be the limiting factor, right? It just may be that it takes a long time to do something. Now, if the AI system circumvented all governments, if it just said, I'm dictator of the world and I'm going to do whatever, some of these things it could do. Again, the things have to do with complexity.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I still think a lot of things would take a while. I don't think it helps that the AI systems can produce a lot of energy or go to the moon. Like, Some people in comments responded to the essay saying the AI system can produce a lot of energy and smarter AI systems. That's missing the point. That kind of cycle doesn't solve the key problems that I'm talking about here.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So I think a bunch of people missed the point there. But even if it were completely unaligned and, you know, could get around all these human obstacles, it would have trouble. But again, if you want this to be an AI system that doesn't take over the world, that doesn't destroy humanity, then, then basically, you know, it's, it's, it's going to need to follow basic human laws, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And so I'm now at this point, I still think, you know, it's always quite uncertain. We have nothing but inductive inference to tell us that the next few years are going to be like the last 10 years. But I've seen the movie enough times.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
You know, if we want to have an actually good world, like we're going to have to have an AI system that, that interacts with humans, not one that kind of creates its own legal system or disregards all the laws or all of that. So as inefficient as these processes are, you know,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
we're going to have to deal with them because there needs to be some popular and democratic legitimacy in how these systems are rolled out. We can't have a small group of people who are developing these systems say, this is what's best for everyone, right? I think it's wrong. And I think in practice, it's not going to work anyway. So you put all those things together and we're not going to
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We're not going to change the world and upload everyone in five minutes. A, I don't think it's going to happen. And B, to the extent that it could happen, it's not the way to lead to a good world. So that's on one side. On the other side, there's another set of perspectives, which I have actually in some ways more sympathy for, which is, look, We've seen big productivity increases before, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Economists are familiar with studying the productivity increases that came from the computer revolution and internet revolution. And generally those productivity increases were underwhelming. They were less than you might imagine. There was a quote from Robert Solow, you see the computer revolution everywhere except the productivity statistics. So why is this the case?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
people point to the structure of firms, the structure of enterprises, how slow it's been to roll out our existing technology to very poor parts of the world, which I talk about in the essay, right? How do we get these technologies to the poorest parts of the world that are behind on cell phone technology, computers, medicine, let alone newfangled AI that hasn't been invented yet?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So you could have a perspective that's like, well, this is amazing technically, but it's all a nothing burger. I think Tyler Cowen, who wrote something in response to my essay, has that perspective. I think he thinks the radical change will happen eventually, but he thinks it'll take 50 or 100 years. And you could have even more static perspectives on the whole thing.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think there's some truth to it. I think the timescale is just too long. And I can see it. I can actually see both sides with today's AI. So, you know, a lot of our customers are large enterprises who are used to doing things a certain way. I've also seen it in talking to governments, right? Those are prototypical, you know, institutions, entities that are slow to change.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I've seen the story happen for enough times to really believe that probably the scaling is going to continue and that there's some magic to it that we haven't really explained on a theoretical basis yet.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But the dynamic I see over and over again is, yes, it takes a long time to move the ship. Yes, there's a lot of resistance and lack of understanding. But the thing that makes me feel that progress will in the end happen moderately fast is
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
not incredibly fast, but moderately fast, is that you talk to... What I find is I find over and over again, again, in large companies, even in governments, which have been actually surprisingly forward-leaning, you find two things. that move things forward.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
One, you find a small fraction of people within a company, within a government who really see the big picture, who see the whole scaling hypothesis, who understand where AI is going, or at least understand where it's going within their industry. And there are a few people like that within the current US government who really see the whole picture.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And those people see that this is the most important thing in the world, and so they agitate for it. And they alone are not enough to succeed because they're a small set of people within a large organization. But...
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
As the technology starts to roll out, as it succeeds in some places, in the folks who are most willing to adopt it, the specter of competition gives them a wind at their backs because they can point within their large organization. They can say, look, these other guys are doing this, right? You know, one bank can say, look, this newfangled hedge fund is doing this thing.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
They're going to eat our lunch. In the U.S., we can say we're afraid China is going to get there before we are. Uh, and that combination, the specter of competition, plus a few visionaries within these, you know, within these, the organizations that in many ways are, are sclerotic, you put those two things together and it actually makes something happen. I mean, it's interesting.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It's a balanced fight between the two because inertia is very powerful, but, but, but eventually over enough time, the innovative approach breaks through, uh, And I've seen that happen. I've seen the arc of that over and over again. And it's like the barriers are there. The barriers to progress, the complexity, not knowing how to use the model, how to deploy them are there.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And for a bit, it seems like they're going to last forever, like change doesn't happen. But then eventually change happens and always comes from a few people. I felt the same way when I was an advocate of the scaling hypothesis within the AI field itself and others didn't get it. It felt like no one would ever get it. Then it felt like we had a secret almost no one ever had.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And then a couple of years later, everyone has the secret. And so I think that's how it's going to go with deployment to AI in the world. It's going to – the barriers are going to fall apart gradually and then all at once. And so I think this is going to be more – and this is just an instinct. I could easily see how I'm wrong.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yes. All of those. In particular, linear scaling up of bigger networks bigger training times, and more data. So all of these things, almost like a chemical reaction. You have three ingredients in the chemical reaction, and you need to linearly scale up the three ingredients. If you scale up one, not the others, you run out of the other reagents and the reaction stops.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think it's going to be more like five or ten years, as I say in the essay, than it's going to be 50 or 100 years. I also think it's going to be five or ten years – more than it's going to be, you know, five or 10 hours. Uh, uh, because I've just, I've just seen how human systems work.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And I think a lot of these people who write down the differential equations who say AI is going to make more powerful AI who can't understand how it could possibly be the case that these things won't, won't change so fast. I think they don't understand these things.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It's almost... It's so... No, no. This was the feeling I had when I was writing it, that it's like, this would be such a beautiful future if we can just make it happen, right? If we can just get the landmines out of the way and make it happen. There's so much... There's so much... beauty and elegance and moral force behind it. And it's something we should all be able to agree on.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
As much as we fight about all these political questions, is this something that could actually bring us together?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
um but you were asking when when when when when do you think what's just so putting numbers on so you know this this is of course the thing i've been grappling with for many years and i'm not i'm not at all confident every time if i say 2026 or 2027 there will be like a zillion like people on twitter who will be like aico said 2026 2020 and it'll be repeated for like the next two years that like this is definitely when i think it's going to happen um so
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
whoever's exerting these clips will crop out the thing I just said and only say the thing I'm about to say. But I'll just say it anyway. Have fun with it. So if you extrapolate the curves that we've had so far, right? If you say, well, I don't know, we're starting to get to like PhD level. And last year we were at undergraduate level.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And the year before we were at like the level of a high school student. Again, you can quibble with at what tasks and for what. We're still missing modalities, but those are being added. Like computer use was added. Like image in was added. Like image generation has been added.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
If you just kind of like, and this is totally unscientific, but if you just kind of like eyeball the rate at which these capabilities are increasing, it does make you think that we'll get there by 2026 or 2027. Again, lots of things could derail it. We could run out of data. You know, we might not be able to scale clusters as much as we want.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Like, you know, maybe Taiwan gets blown up or something. And, you know, then we can't produce as many GPUs as we want. So there are all kinds of things that could derail the whole process. So I don't fully believe the straight line extrapolation. But if you believe the straight line extrapolation, we'll get there in 2026 or 2027.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think the most likely is that there is some mild delay relative to that. I don't know what that delay is, but I think it could happen on schedule. I think there could be a mild delay. I think there are still worlds where it doesn't happen in a hundred years. Those were the number of those worlds is rapidly decreasing.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But if you scale up everything in series, then the reaction can proceed.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We are rapidly running out of truly convincing Brockler's truly compelling reasons why this will not happen in the next few years. There were a lot more in 2020, um, Although my guess, my hunch at that time was that we'll make it through all those blockers.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So sitting as someone who has seen most of the blockers cleared out of the way, I kind of suspect, my hunch, my suspicion is that the rest of them will not block us. But, you know... Look, at the end of the day, I don't want to represent this as a scientific prediction. People call them scaling laws. That's a misnomer, like Moore's law is a misnomer.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Moore's law, scaling laws, they're not laws of the universe. They're empirical regularities. I am going to bed in favor of them continuing, but I'm not certain of that.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, yeah.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Well, let me start with your first questions and then I'll answer that. Claude wants to know what's in his future, right? Exactly. Who's it? Who am I going to be working with?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So I think one of the things I went hard on when I went hard on in the essay is, let me go back to this idea of, because it's really had an impact on me, this idea that within large organizations and systems, there end up being a few people or a few new ideas who kind of cause things to go in a different direction than they would have before, who kind of disproportionately affect the trajectory.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
There's a bunch of kind of the same thing going on, right? If you think about the health world, there's like trillions of dollars to pay out Medicare and other health insurance. And then the NIH is a hundred billion. And then if I think of like the few things that have really revolutionized anything, it could be encapsulated in a small fraction of that.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And so when I think of like, where will AI have an impact? I'm like, Can AI turn that small fraction into a much larger fraction and raise its quality? And within biology, my experience within biology is that the biggest problem of biology is that you can't see what's going on. You have very little ability to see what's going on and even less ability to change it, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
What you have is this, like... From this, you have to infer that there's a bunch of cells that within each cell is 3 billion base pairs of DNA built according to a genetic code. And there are all these processes that are just going on without any ability of us as unaugmented humans. to affect it. These cells are dividing.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Most of the time that's healthy, but sometimes that process goes wrong and that's cancer. The cells are aging. Your skin may change color, develop wrinkles as you age. And all of this is determined by these processes, all these proteins being produced, transported to various parts of the cells, binding to each other.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And in our initial state about biology, we didn't even know that these cells existed. We had to invent microscopes to observe the cells. We had to invent more powerful microscopes to see below the level of the cell to the level of molecules. We had to invent x-ray crystallography to see the DNA. We had to invent gene sequencing to read the DNA.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Now, we had to invent protein folding technology to predict how it would fold and how these things unfold.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
bind to each other uh you know we had to we had to invent various techniques for now we can edit the g the dna as of you know with crispr as of the last uh 12 years so the the whole history of biology a whole big part of the history is is basically our our our ability to read and understand what's going on and our ability to reach in and selectively change things
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Um, and, and my view is that there's so much more we can still do there, right? You can do CRISPR, but you can do it for your whole body. Um, let's say I want to do it for one particular type of cell and I want the rate of targeting the wrong cell to be very low. That's still a challenge. That's still things people are working on. That's what we might need for gene therapy for certain diseases.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And so the reason I'm saying all of this, and it goes beyond, you know, beyond this to, you know, to gene sequencing, to new types of nanomaterials for observing what's going on inside cells, for, you know, antibody drug conjugates. The reason I'm saying all this is that this could be a leverage point for the AI systems, right?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, we've documented scaling laws in lots of domains other than language, right? So initially, the paper we did that first showed it was in early 2020, where we first showed it for language. There was then some work late in 2020 where we showed the same thing for other modalities like images, video, text to image, image to text, math, that they all had the same pattern. And you're right.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
That the number of such inventions, it's in the mid-double digits or something, you know, mid-double digits, maybe low triple digits over the history of biology. Let's say I have a million of these AIs, like, you know, can they discover a thousand, you know, working together? Can they discover thousands of these very quickly?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And does that provide a huge lever instead of trying to leverage the, you know, two trillion a year we spend on, you know, Medicare or whatever? Can we leverage the one billion a year that's, you know, that's spent to discover, but with much higher quality? And so what is it like, you know, being a scientist that works with an AI system?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
The way I think about it actually is, well, so I think in the early stages, the AIs are going to be like grad students. You're going to give them a project. You're going to say, you know, I'm the experienced biologist. I've set up the lab. The biology professor or even the grad students themselves will say, Here's what you can do with an AI system. I'd like to study this.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And the AI system, it has all the tools. It can look up all the literature to decide what to do. It can look at all the equipment. It can go to a website and say, hey, I'm going to go to Thermo Fisher or whatever the lab equipment company is to the dominant lab equipment company is today. And my time was Thermo Fisher. I'm going to order this new equipment to do this.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I'm going to run my experiments. I'm going to write up a report about my experiments. I'm going to inspect the images for contamination. I'm going to decide what the next experiment is. I'm going to write some code and run a statistical analysis.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
All the things a grad student would do, there will be a computer with an AI that the professor talks to every once in a while and it says, this is what you're going to do today. The AI system comes to it with questions. When it's necessary to run the lab equipment, it may be limited in some ways. It may have to hire a human lab assistant to do the experiment and explain how to do it.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Or it could, you know, it could use advances in lab automation that are gradually being developed over, have been developed over the last decade or so and will continue to be developed. And so it'll look like there's a human professor and a thousand AI grad students.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And, you know, if you go to one of these Nobel Prize winning biologists or so, you'll say, okay, well, you know, you had like 50 grad students. Well, now you have a thousand and they're smarter than you are, by the way.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Um, uh, then I think at some point it'll flip around where the, you know, the AI systems will, you know, will, will be the PIs will be the leaders and, and, and, you know, they'll be, they'll be ordering humans or other AI systems around. So I think that's how it'll work on the research side. And they would be the inventors of a CRISPR type technology.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Um, and then I think, you know, as I say in the essay, we'll want to turn, turn, probably turning loose is the wrong, the wrong term, but we'll want to, we'll want to harness the AI systems, uh, to improve the clinical trial system as well. There's some amount of this that's regulatory. That's a matter of societal decisions and that'll be harder, but yeah.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Can we get better at predicting the results of clinical trials? Can we get better at statistical design so that clinical trials that used to require 5,000 people and therefore needed $100 million in a year to enroll them, now they need 500 people in two months to enroll them. That's where we should start.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Uh, and, and, you know, can we increase the success rate of clinical trials by doing things in animal trials that we used to do in clinical trials and doing things in simulations that we used to do in animal trials? Again, we won't be able to simulate it all. AI is not God. Um, uh, but, but, you know, can we, can we shift the curve substantially and radically?
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So I don't know, that would be my picture.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah, yeah, yeah. Can we just... One step at a time, and can that add up to a lot of steps? Even though we still need clinical trials, even though we still need laws, even though the FDA and other organizations will still not be perfect, can we just move everything in a positive direction? And when you add up all those positive directions, do you get...
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Everything that was going to happen from here to 2100 instead happens from 2027 to 2032 or something.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Now, there are other stages like post-training or there are new types of reasoning models. And in all of those cases that we've measured, we see similar types of scaling laws.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I think that's going to be one of the areas that changes fastest for two reasons. One, programming is a skill that's very close to the actual building of the AI. So the farther a skill is from the people who are building the AI, the longer it's going to take to get disrupted by the AI. I truly believe that AI will disrupt agriculture.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Maybe it already has in some ways, but that's just very distant from the folks who are building AI. And so I think it's going to take longer. But programming is the bread and butter of a large fraction of the employees who work at Anthropic and at the other companies. And so it's going to happen fast. The other reason it's going to happen fast is with programming, you close the loop.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Both when you're training the model and when you're applying the model, the idea that the model can write the code means that the model can then run the code and then see the results and interpret it back. And so it really has an ability, unlike hardware, unlike biology, which we just discussed, the model has an ability to close the loop.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Um, and, and so I think those two things are going to lead to the model getting good at programming very fast. As I saw on, you know, typical real world programming tasks, models have gone from 3% in January of this year to 50% in October of this year. So, you know, we're on that S curve, right? Where it It's going to start slowing down soon because you can only get 200%.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But I would guess that in another 10 months, we'll probably get pretty close. We'll be at least 90%. So again, I would guess, I don't know how long it'll take, but I would guess again, 2026, 2027, Twitter people who crop out these numbers and get rid of the caveats, like, I don't know, I don't like you, go away. Yeah.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I would guess that the kind of task that the vast majority of coders do, AI can probably, if we make the task very narrow, like just write code, AI systems will be able to do that. Now, that said, I think comparative advantage is powerful. We'll find that when AIs can do 80% of a coder's job, including most of it that's literally like write code with a given spec,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
we'll find that the remaining parts of the job become more leveraged for humans, right? Humans will, there'll be more about like high-level system design or, you know, looking at the app and like, is it architected well? And the design and UX aspects, and eventually AI will be able to do those as well, right? That's my vision of the, you know, powerful AI system.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But I think for much longer than we might expect, we will see that, uh, small parts of the job that humans still do will expand to fill their entire job in order for the overall productivity to go up. That's something we've seen. You know, it used to be that, you know, writing and editing letters was very difficult and like writing the print was difficult.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Well, as soon as you had word processors and then computers, and it became easy to produce work and easy to share it, then that became instant and all the focus was on the ideas. So this logic of comparative advantage that expands tiny parts of the tasks to large parts of the tasks and creates new tasks in order to expand productivity, I think that's going to be the case.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
someday AI will be better at everything and that logic won't apply. And then we all have, humanity will have to think about how to collectively deal with that. And we're thinking about that every day. And that's another one of the grand problems to deal with aside from misuse and autonomy. And we should take it very seriously.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But I think in the near term and maybe even in the medium term, like medium term, like two, three, four years, I expect that Humans will continue to have a huge role and the nature of programming will change, but programming as a role, programming as a job will not change. It'll just be less writing things line by line and it'll be more macroscopic.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So in my previous career as a biophysicist, so I did physics undergrad and then biophysics in grad school. So I think back to what I know as a physicist, which is actually much less than what some of my colleagues at Anthropic have in terms of expertise in physics.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I'm absolutely convinced that powerful IDEs, that there's so much low-hanging fruit to be grabbed there, that right now it's just like you talk to the model and it talks back. But look, I mean... IDs are great at lots of static analysis of so much as possible with static analysis, like many bugs you can find without even writing the code.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Then IDs are good for running particular things, organizing your code, measuring coverage of unit tests. There's so much that's been possible with the normal IDs. Now you add something like well, the model can now write code and run code.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I am absolutely convinced that over the next year or two, even if the quality of the models didn't improve, that there would be enormous opportunity to enhance people's productivity by catching a bunch of mistakes, doing a bunch of grunt work for people, and that we haven't even scratched the surface. Anthropic itself, I mean, you can't say no...
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
you know, it's hard to say what will happen in the future. Currently, we're not trying to make such IDs ourself. Rather, we're powering the companies like Cursor or like Cognition or some of the other, you know, Expo in the security space. You know, others that I can mention as well that are building such things themselves on top of our API. And our view has been let a thousand flowers bloom.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
We don't internally have the, you know, the resources to try all these different things, let's let our customers try it. And, you know, we'll see who succeeds and maybe different customers will succeed in different ways.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
So I both think this is super promising and, you know, it's not something, you know, Anthropic isn't eager to, at least right now, compete with all our companies in this space and maybe never.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
there's this concept called the 1 over f noise and 1 over x distributions, where often, you know, just like if you add up a bunch of natural processes, you get a Gaussian. If you add up a bunch of kind of differently distributed natural processes. If you like, take a probe and hook it up to a resistor. The distribution of the thermal noise in the resistor goes as one over the frequency.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
It is. It is really astounding. I feel like, you know, as a CEO, I don't get to program that much. And I feel like if six months from now I go back, it'll be completely unrecognizable to me. Exactly.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Yeah.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
This is something that I've written about a little bit in the essay, although I actually, I give it a bit short shrift, not for any principled reason, but this essay, if you believe it, was originally going to be two or three pages. I was going to talk about it at all hands. And the reason I realized it was an important underexplored topic is that I just kept writing things.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
And I was just like, oh man, I can't do this justice. And so the thing ballooned to like 40 or 50 pages. And then when I got to the work and meaning section, I'm like, oh man, this isn't going to be a hundred pages. Like I'm going to have to write a whole other essay about that.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
But meaning is actually interesting because you think about like the life that someone lives or something, or like, you know, like, you know, let's say you were to put me in like a, I don't know, like a simulated environment or something where like,
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
um you know like i have a job and i'm trying to accomplish things and i don't know i like do that for 60 years and then then you're like oh oh like oops this was this was actually all a game right does that really kind of rob you of the meaning of the whole thing you know like i still made important choices including moral choices i still sacrificed i still had to kind of gain all these skills or or or just like a similar exercise you know think back to like you know one of the historical figures who you know discovered electromagnetism or relativity or something
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
If you told them, well, actually, 20,000 years ago, some alien on this planet discovered this before you did, does that rob the meaning of the discovery? It doesn't really seem like it to me, right? It seems like the process is what matters and how it shows who you are as a person along the way and how you relate to other people and the decisions that you make along the way.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Those are consequential, right? I could imagine if we handle things badly in an AI world, we could set things up where people don't have any long-term source of meaning or any, but that's more a set of choices we make. That's more a set of the architecture of a society with these powerful models. If we design it badly and for shallow things, then that might happen.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
I would also say that most people's lives today while admirably, you know, they work very hard to find meaning in those lives. Like, look, you know, we who are privileged and who are developing these technologies, we should have empathy for people, not just here, but in the rest of the world who, you know, spend a lot of their time kind of scraping by to like survive.
Lex Fridman Podcast
#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Assuming we can distribute the benefits of this technology to everywhere, like their lives are going to get a hell of a lot better. And, you know, meaning will be important to them as it is important to them now. But but, you know, we should not forget the importance of that.