Swale Asif

Yes, that is very important. That is very important. And it's actually sort of an underrated aspect of how we decide what to build. Like a lot of the things that we build and then we try them out, we do an experiment and then we actually throw them out because they're not fun. And so a big part of being fun is like being fast a lot of the time. Fast is fun.

And I think actually one of the underrated aspects of GitHub Copilot is that even when it's wrong, it's like a little bit annoying, but it's not that bad because you just type another character and then maybe then it gets you or you type another character and then it gets you. So even when it's wrong, it's not that bad.

That's what the- Well, it was technically not.

Yeah, I think one thing that I think helps us is that we're sort of doing it all in one, where we're developing the UX and the way you interact with the model at the same time as we're developing how we actually make the model give better answers. So we're like... how you build up the prompt or like how do you find the context and for a cursor tab, like how do you train the model?

So I think that helps us to have all of it like sort of like the same people working on the entire experience end-to-end.

The full generalization is like next action prediction. Sometimes you need to run a command in the terminal, and it should be able to suggest the command based on the code that you wrote to. Or sometimes you actually need to... Like it suggests something, but it's hard for you to know if it's correct because you actually need some more information to learn.

Like you need to know the type to be able to verify that it's correct. And so maybe it should actually take you to a place that's like the definition of something and then take you back so that you have all the requisite knowledge to be able to accept the next completion.

I am personally very excited for... making a lot of improvements in this area. We often talk about it as the verification problem, where these diffs are great for small edits. For large edits, or when it's multiple files or something, it's actually a little bit prohibitive to review these diffs. And so there are a couple of different ideas here.

One idea that we have is, okay, parts of the diffs are important. They have a lot of information. And then parts of the diff are just very low entropy. They're the same thing over and over again. And so maybe you can highlight the important pieces and then gray out the not so important pieces. Or maybe you can have a model that

looks at the diff and sees, oh, there's a likely bug here, I will mark this with a little red squiggly and say, you should probably review this part of the diff. And ideas in that vein, I think, are exciting.

Yeah.

Yeah, and you want an intelligent model to do it. Like currently, diff algorithms are, they're like... like they're just like normal algorithms. There is no intelligence. There's like intelligence that went into designing the algorithm, but then there's no, like, you don't care if it's about this thing or this thing, as you want a model to do this.

Just one idea there is I think ordering matters. Generally, when you review a PR, you have this list of files and you're reviewing them from top to bottom. But actually, you actually want to understand this part first, because that came logically first. And then you want to understand the next part. And you don't want to have to figure out that yourself.

You want a model to guide you through the thing.

I think sometimes. I don't think it's going to be the case that all of programming will be natural language. And the reason for that is, you know, if I'm pair programming with Swala and Swala is at the computer and the keyboard. And sometimes, if I'm driving, I want to say to Swallow, hey, implement this function. And that works.

And then sometimes it's just so annoying to explain to Swallow what I want him to do. And so I actually take over the keyboard and I show him. I write part of the example. And then... it makes sense. And that's the easiest way to communicate. And so I think that's also the case for AI.

Sometimes the easiest way to communicate with the AI will be to show an example, and then it goes and does the thing everywhere else. Or sometimes if you're making a website, for example, the easiest way to show to the AI what you want is not to tell it what to do, but drag things around or draw things. And Yeah.

And like maybe eventually we will get to like brain machine interfaces or whatever and kind of like understand what you're thinking. And so I think natural language will have a place. I think it will not definitely not be the way most people program most of the time.

Yeah, I think it depends on which model you're using. And all of them are slightly different and they respond differently to different prompts. But I think the original GPT-4 and the original sort of pre-double models last year, they were quite sensitive to the prompts. And they also had a very small context window.

And so we have all of these pieces of information around the code base that would maybe be relevant in the prompt. Like you have the docs, you have the files that you add, you have the conversation history. And then there's a problem like how do you decide what you actually put in the prompt and when you have a limited space.