Michael Truell

Don't you think this gets at a little bit some of the stuff you were talking about earlier with the difficulty of specifying intent for what you want with software? Where sometimes it might be because the intent is really hard to specify, it's also then going to be really hard to prove that it's actually matching whatever your intent is.

Yeah, or just for a given spec, maybe you can... I think there is a question of can you actually do the formal verification? Is that possible? I think that there's more to dig into there.

Yeah, or just for a given spec, maybe you can... I think there is a question of can you actually do the formal verification? Is that possible? I think that there's more to dig into there.

Yeah, or just for a given spec, maybe you can... I think there is a question of can you actually do the formal verification? Is that possible? I think that there's more to dig into there.

Even if you have the spec. Is the spec written in natural language? Yeah, how do you map the spec?

Even if you have the spec. Is the spec written in natural language? Yeah, how do you map the spec?

Even if you have the spec. Is the spec written in natural language? Yeah, how do you map the spec?

But how easy would that be to draw? So then I think that you care about things that are not going to be easily well-specified in the spec language.

But how easy would that be to draw? So then I think that you care about things that are not going to be easily well-specified in the spec language.

But how easy would that be to draw? So then I think that you care about things that are not going to be easily well-specified in the spec language.

Yeah. Maybe an argument against formal verification is all you need.

Yeah. Maybe an argument against formal verification is all you need.

Yeah. Maybe an argument against formal verification is all you need.

Yeah. And I think that it's also important to note that this is having good bug finding models feels necessary to get to the highest reaches of having AI do more and more programming for you, where you're going to, you know, if the AI is building more and more of the system for you, you need to not just generate, but also verify.

Yeah. And I think that it's also important to note that this is having good bug finding models feels necessary to get to the highest reaches of having AI do more and more programming for you, where you're going to, you know, if the AI is building more and more of the system for you, you need to not just generate, but also verify.

Yeah. And I think that it's also important to note that this is having good bug finding models feels necessary to get to the highest reaches of having AI do more and more programming for you, where you're going to, you know, if the AI is building more and more of the system for you, you need to not just generate, but also verify.

And without that, some of the problems that we've talked about before with programming with these models will just become untenable. So it's not just for humans, like you write a bug, I write a bug, find the bug for me, but it's also being able to verify the AI's code and check it is really important.

And without that, some of the problems that we've talked about before with programming with these models will just become untenable. So it's not just for humans, like you write a bug, I write a bug, find the bug for me, but it's also being able to verify the AI's code and check it is really important.

And without that, some of the problems that we've talked about before with programming with these models will just become untenable. So it's not just for humans, like you write a bug, I write a bug, find the bug for me, but it's also being able to verify the AI's code and check it is really important.

You can also do a bunch of work, not even at the model level, of taking the biggest models and then maybe giving them access to a lot of information that's not just the code. it's kind of a hard problem to stare at a file and be like, where's the bug? And that's hard for humans often, right? And so often you have to run the code and being able to see things like traces and step through a debugger.