Conrad Irwin
Appearances
Rust in Production
Zed with Conrad Irwin
a graphical app so there's kind of a specific thread that coco knows okay like regardless of how busy the system is like we're going to keep this thread running because it's visible to the user and so we wanted to be able to hook into that system and so we have our own async runtime that's not tokyo but but kind of works works in that more ui based way of like if you want ui updates you have to be on the main thread otherwise you have to be on a background thread and we have a bunch of helpers to make sure that your code is always running on the thread you expect
Rust in Production
Zed with Conrad Irwin
We don't use the runtimes for Tokyo or Small. I think we use a bunch of the helper stuff from them. But I know that was a push to get rid of Small because it's like, well, we don't really need anything from here. But that's kind of where the Rust ecosystem stuff comes in. It's a very big ecosystem with a whole bunch of stuff in it.
Rust in Production
Zed with Conrad Irwin
And so it's very easy to end up with a small dependency on these things through someone else, even if we're not using the main parts of their libraries.
Rust in Production
Zed with Conrad Irwin
And if you compare it to something like a VS Code, you can take the browser emulator and the extensions APIs and all of that stuff that makes VS Code kind of slow and clunky and rebuild it natively in Rust and using an actual fast GPU native rendering. So taking more of kind of like the React approach and like a game would be built.
Rust in Production
Zed with Conrad Irwin
I think we actually did. I was trying to look it up. It was something we were trying to do. And so we took that piece of GPUI and split it out into a separate crate. But I'm now looking, and I can't see it within 10 seconds. So I'll have to look it up after this and find it.
Rust in Production
Zed with Conrad Irwin
A little side story. We're using a really old HTTP client right now called ISHC, which was very popular four or five years ago. And the reason we're using it is it's the only one that lets you plug and play the async runtime.
Rust in Production
Zed with Conrad Irwin
And so I really like the idea of having this kind of diverse set of async runtimes, but there seems to be some kind of... The abstraction is not in quite the right place, because it seems like if you're building something, you can't just say, oh, give me any async runtime. That's not as easy to do as it is to say, okay, we'll just use Tokyo. So that one seems to be landing on right now.
Rust in Production
Zed with Conrad Irwin
And so in the abstract, it's good for people to kind of solidify around Tokyo. That seems to be where most of the energy is, because it gives you more batteries included. But... Suddenly, you're in this situation where there's tools that we want to use, like Tokyo's HTTP stuff, which we can't because we don't use Tokyo's runtime.
Rust in Production
Zed with Conrad Irwin
And so, yeah, it's on the back burner right now, but we need to upgrade the HTTP client to upgrade LibSSL, which we don't really want to link, but there's no alternative async HTTP client that isn't Tokyo-based. And I don't want to build my own, please. So we'll have to kind of figure that out. And that's kind of been my experience with it.
Rust in Production
Zed with Conrad Irwin
You know, there's the day-to-day async pain points in the language itself, but the ecosystem is just very either Tokyo or kind of you're on your own a little bit.
Rust in Production
Zed with Conrad Irwin
I don't know the answer. I think it's, as with all of these things, it's part cultural, part technical. It's hard to write an async runtime because it's such an abstract piece of thing to do. The end result's only a few hundred lines of code, but the right few hundred lines of code, if that makes sense.
Rust in Production
Zed with Conrad Irwin
And so most people don't think to do that, which means kind of like, okay, culturally, sure, let's just use Tokyo. It's the most common one. And so I think
Rust in Production
Zed with Conrad Irwin
if we go that way it's i don't know maybe that's okay but if they don't support the async runtimes that everyone wants to use then then you kind of end up with these problems i don't know and it'll it'll be interesting to see how the community kind of like responds to it i think the ross community kind of likes rebuilding things over and over again so i'm sure we'll be fine it'll be interesting to see what comes out if you compared it with ross's arrow handling story
Rust in Production
Zed with Conrad Irwin
On the other side, you have things like Vim, which are fast and people love them for the speed and the ease, but they don't work with any of the modern tools. So you spend all your time configuring language servers and like breaking plugins and fixing plugins and breaking plugins.
Rust in Production
Zed with Conrad Irwin
Yeah, I mean, I definitely see the problem, given the examples that we have. So one of the things that I'm hopeful for is that as the language evolves, you know, AsyncRust is still very beta, as I think they called it in the latest Rust planning blog post.
Rust in Production
Zed with Conrad Irwin
As they evolve the language support for it, what I hope is that it becomes easier to do both sides of things, both easier to use as a consumer, but also easier to create as a kind of like, hey, here's my new Async library for you.
Rust in Production
Zed with Conrad Irwin
And I think if you look at something like jQuery to use a very different example, it was something that started out very dominant or started out small, became very dominant very quickly. And then as the underlying browser APIs improved, became kind of less relevant.
Rust in Production
Zed with Conrad Irwin
So that's kind of the hope I'd see, if that makes sense, of as the APIs get more sensible and people get more used to all of the concepts involved, it becomes easier to do both sides of the coin. So I have nothing against Tokyo. It's a great piece of software. But I also, to your point earlier, I really like the diversity of being able to do things in multiple different ways.
Rust in Production
Zed with Conrad Irwin
And so we wanted to make something that had this kind of trio of collaborative, extremely fast and just works out of the box. So really helping people get their work done, like not spending time configuring your editor, if that makes sense.
Rust in Production
Zed with Conrad Irwin
But even things like async traits and stuff like that, there's so much obvious stuff that needs improvement first that I think there's definitely time to figure it out, even if it feels like it's not figured out yet. Come on. What percentage of Z is async? I would guess 50-50, but I don't know. Here we could have a look. But pretty much everything that requires accessing the disk is async.
Rust in Production
Zed with Conrad Irwin
The network is async, and text editors do a lot of disk access and lots of language server access. And so the stuff that's not async is really just, oh, you typed a key? Okay, left. One move the cursor one position to the left. We can do that synchronously.
Rust in Production
Zed with Conrad Irwin
But even things like search, if you search for a character, we kick that off to an async background thread so that we can keep rendering UI while we search. And so... Pretty much anything that uses a lot of CPU is not on the main thread.
Rust in Production
Zed with Conrad Irwin
It's something that evolved naturally. So back to GPUI again, it provides contexts. And contexts are roughly a way of storing global state, but not in a global. But what that lets you do is that code that is sync takes an app context, and code that is async takes an async context. And so if you build something slow, you make it take an async context. You can't call it on the main thread anymore.
Rust in Production
Zed with Conrad Irwin
And so it gets easy to manage it kind of at a type level. But it does require when you're building a new feature that is CPU bound to notice and put that in the background.
Rust in Production
Zed with Conrad Irwin
Yeah, exactly. I hadn't thought about it that way, but you're right. It's kind of nice. This is async. Go away.
Rust in Production
Zed with Conrad Irwin
Well, I mean, beyond the complexity of the editor piece, one of the things that is tricky with Rust, and we kind of touched on this a bit of time, is the ownership rules make it challenging because the text editor is fairly self-referential. You have all of the... All of the things want to know about the editor and the editor needs to be able to tell lots of things stuff.
Rust in Production
Zed with Conrad Irwin
And so we have several different solutions for, okay, here's the editor, we'll send events this way, or you can observe it and get things out. But it's not as easy as it would be in a language like JavaScript to just, okay, we'll plonk this thing on the side and the editor could talk to it and it could talk to the editor and we don't have to worry about it.
Rust in Production
Zed with Conrad Irwin
Well, before it did exist, I kept switching between NeoVim and VS Code. VS Code, because I like the language servers and all of that stuff, but it would just frustrate me too often. So I'd go back to NeoVim where I know how to be productive. And so I've spent a lot of my time making Zed's Vim mode work for me and for the other people who use Vim. So that's been fun.
Rust in Production
Zed with Conrad Irwin
And so that's probably the main piece that Rust makes it tricky. Other than that, the hard bit is just building a text editor. There's so many features that people expect to just work.
Rust in Production
Zed with Conrad Irwin
Why is that? Because the people who want Vim Mode can't live without it, and everyone else is like, I don't care about this. So that's true for not every feature we build, but a number of them. We've been working on Jupyter Notebooks, for example, and people who need Jupyter Notebooks, they love that feature. And everyone else is like, yeah, whatever. I don't use that.
Rust in Production
Zed with Conrad Irwin
And so kind of trying to navigate the trade-offs of which features do we build and who do we make happy in what order is a big problem. But that's definitely a text editor problem, not a Rust problem.
Rust in Production
Zed with Conrad Irwin
Yeah, definitely agree. So if you think about a programming text editor, kind of one of the first features you want to build is syntax highlighting. And if you look at really all editors, it's a hand-coded parser for each language that does it. Not going to fly. We don't have time to build a hand-coded parser for every language.
Rust in Production
Zed with Conrad Irwin
Then maybe a decade or two ago, people started using regular expressions. Like, cool, here's a regular expression that does it. And in some cases, like the KDE text editor was kind of influential early in this. It's like a half XML language, half regular expression. So you get a bit of recursion, a bit of regular expression, and kind of a mix in there.
Rust in Production
Zed with Conrad Irwin
And these things are all fine, and they work for what they work for. But they only solve the syntax highlighting problem. And so if you want to be able to understand a little bit more about, okay, so this text on the screen is just an array of Unicode bytes. But what does it mean? You need something that can not just look at it byte by byte, but really divide it up into syntax.
Rust in Production
Zed with Conrad Irwin
And one of the Z founders, Max, built TreeSitter to do this in Atom. But it's like, okay, we get syntax highlighting for this for free because it understands the language. But we also get things like jumping to matching brackets or If you want to look at a file and say, what's in this file? We have a thing called the outline view. And so you can just see all the things that are defined in there.
Rust in Production
Zed with Conrad Irwin
And that's all powered, again, by TreeSitter. And so it's fundamental to the way that we do programming language stuff, which can all be done instead of having to do it by byte, by tree traversal instead, which is orders of magnitude faster.
Rust in Production
Zed with Conrad Irwin
So TreeSitter, each language in TreeSitter has its own kind of definitions. And then Zed has a couple of things that map from those definitions to our definitions. And so each supported language has a mapping of like, OK, in the TreeSitter grammar, there's a thing called comment. In the Z code highlighting, there's a thing called comment. Those are the same thing.
Rust in Production
Zed with Conrad Irwin
So we have that mapping for each language. Similarly, it's like, OK, if you want to extract all the function definitions from a file, this is the TreeSitter query used to get that. And those queries can all run on the background thread. And TreeSitter itself is kind of crazy. It's a bunch of C libraries written for each language.
Rust in Production
Zed with Conrad Irwin
So we run those inside a WebAssembly module to avoid the obvious problems with running C libraries. And that's been good for reliability.
Rust in Production
Zed with Conrad Irwin
No idea. I do know that kind of the key feature of it as opposed to like a pauser for a compiler is that it's error tolerant. So if you have a trailing quote mark, it's not going to throw it off. It can always do something. And so it has optimized small edits in the text lead to small changes in the tree. So I guess it monitors or babysits your tree. Maybe that's where it comes from. I don't know.
Rust in Production
Zed with Conrad Irwin
Do you know Semgrep? I know the name, but you'd have to remind me.
Rust in Production
Zed with Conrad Irwin
Yes, is kind of the answer. So yeah, we don't have much built on that right now, but it's kind of there in the background. One of the most obvious things we can do, there are bindings, select more and select less, and they work on the tree setter definition. So you can kind of, okay, I start in this quote, then I expand more until I have the whole function or less until I'm back down to the thing.
Rust in Production
Zed with Conrad Irwin
But we don't really have many go edit via syntax tree stuff right now. One small example of something we do have in Vim, we have an argument object. So you can select an argument with VIA, you know, select inside an argument. And that uses the tree set of grammar to find the argument that you're in.
Rust in Production
Zed with Conrad Irwin
Keeping it simple and building it ourselves, I guess the two parts of that. So internally, we have a bunch of branches, one for each kind of version. So right now we're on 149 stable, 150 preview. Any commits that get added to those branches don't do anything until you run a script, which is script trigger release, and you give it a table of preview. That kicks off a build that uploads to GitHub.
Rust in Production
Zed with Conrad Irwin
And there's a lot involved in making a build of something like this because you have x86, you have ARM64, you have Mac and Linux. And so you end up with four or five different binary downloads that it creates. It uploads them all to GitHub, and then it marks releases preview.
Rust in Production
Zed with Conrad Irwin
And then Joseph, usually, but it could be anyone, goes in, takes all the commit messages, and formats out the release notes. And we have some tooling to kind of help with most of that work. But to your point, if you want to make them readable and nicely formatted, there's no auto, like, oh, yeah, we just pull it in from the PR call of the day. It doesn't work well enough.
Rust in Production
Zed with Conrad Irwin
And so we want to make sure that it's easy to understand how that is changing. And so we spend time on, spend manual time on that. And then, yeah, then go from there. And then the auto-updater on the client side is just a loop. Sleep for an hour. Is there an update? If there is, download it, copy it into place, and then reboot.
Rust in Production
Zed with Conrad Irwin
No, we use something that's out there. I don't remember which one off the top of my head, but we've, yeah, we use something that's out there. We have fixed some bugs in there because we saw some crashes from it. So, you know, it's kind of fun.
Rust in Production
Zed with Conrad Irwin
Extensions can register tree set of grammars, and that's kind of the main interaction there. And then extensions themselves, they also have the ability to run some code in WebAssembly. Right now, it's pretty limited. So the most common use for an extension today is a language server. And trying to download the correct version of a language server requires a little bit of code.
Rust in Production
Zed with Conrad Irwin
And so you're kind of the ideal target user, someone who wants better things from their tools. And so building that the right way has been very fun.
Rust in Production
Zed with Conrad Irwin
So kind of one of the things that we really care about is the speed and the performance piece of it. And so when we were first talking about extensions, it's like, well, we could have like a JavaScript runtime and run JavaScript extensions. But I don't think that's going to happen anymore. There are enough languages that are easy to compile down to a WebAssembly thing that will do it there.
Rust in Production
Zed with Conrad Irwin
The second piece that's tricky is we have a completely custom UI framework that doesn't have any bindings for any other language. And so how I imagine things going is that we kind of continue down the approach we have today, which is that we expose simple things that you can extend, one of which is the AI features have some hook points where you can pull in more context for the model. That's one.
Rust in Production
Zed with Conrad Irwin
Another is language server. A third is themes. But it's not like, hey, you can run arbitrary code in our process. Thank you very much. I think we'll keep it pretty tight for now. The next piece I really want to build is being able to kind of bind keyboard shortcuts and then run some code that modifies the editor.
Rust in Production
Zed with Conrad Irwin
And I think that that's kind of a solvable piece that we could kind of ship by yourself. And so let's say you want, you know, let's say SHA-256 support, right? You could imagine an extension that registers a new command and a keyboard shortcut to SHA-256 the file and gives you the answer. But trying to build something that allows you to render UI is a long way off, I think. Yeah.
Rust in Production
Zed with Conrad Irwin
Yeah, if you're doing a little bit of user research and talking to people working on very large code bases in VS Code, they really, really try not to restart their computer or turn off VS Code ever because they turn it back on and it takes five or six minutes to update all the extensions and spin around and show up banners until it's ready to use. And it's like, oh... We have to avoid that.
Rust in Production
Zed with Conrad Irwin
And it's hard because all those things do something useful for someone, but really try to make sure that they don't get in the way, I think is really important.
Rust in Production
Zed with Conrad Irwin
Yep. I'm Conrad. I work at Zed, which is trying to build the next generation of text editor. Coding at the speed of thought is our motto. Prior to Zed, I built Superhuman, which is also focused on speed, building the fastest email client in the world. So I really like building tools that make people get what they need to get done faster and easier. And that's how I found myself working on Zed.
Rust in Production
Zed with Conrad Irwin
Rust definitely helps a lot. It's not JavaScript, which is really nice. But one of the things that we do that is very different is that we render like a game renders. So each frame, so every 8 milliseconds or 16 milliseconds, we redraw the entire screen to the GPU.
Rust in Production
Zed with Conrad Irwin
Yeah. So the primary input to the create structure is compile time because 500,000 lines of Rust takes a long time to compile. And so really the question is, what code gets recompiled a lot, and how do we reduce the amount of it? And so that's where the create structure comes from.
Rust in Production
Zed with Conrad Irwin
This is obviously a little bit around abstractions and not leaking between things, but really primarily it's the speed thing. So we don't really want a kind of a deeply nested thing where visibility is tightly controlled and we have lots of, you know, that's not important. We trust the whole code base. And so it's making sure that it's somewhat easy to find what you're looking for.
Rust in Production
Zed with Conrad Irwin
And when you rebuild, you know, if you make a change to the editor, you don't have to rebuild too much else of the code base to get into testing.
Rust in Production
Zed with Conrad Irwin
Right. Yeah, and I like the pragmatism there. We do have some crates that are too big, but mostly it's pretty well factored out, I think.
Rust in Production
Zed with Conrad Irwin
So one thing I learned, we did a rewrite of GPUI, the graphics framework, that was fairly heavy on use of generics and ended up exporting a bunch of things that were instantiated multiple times with multiple different types. And the way that Rust compiles generics is to kind of copy-paste the code each time for each type.
Rust in Production
Zed with Conrad Irwin
And so we had some kind of big compile time regressions at that point just because of the way the code was structured. And so one thing to look out for is as often as you can avoid having a generic type exported from a crate.
Rust in Production
Zed with Conrad Irwin
So if you have a function that takes generic arguments, you kind of want to keep that internal because otherwise every time someone uses it, you get another copy of that whole thing out. And obviously, like everything, there's a trade-off. There are some things where it is worth the pain, but there are other things where if you can avoid that kind of type boilerplate exploding, you should.
Rust in Production
Zed with Conrad Irwin
and that means that we are not CPU-bound when it comes to rendering stuff, unlike, say, a VS Code or an HTML that has to do a lot of CPU work to get stuff onto the screen. Ours is all GPU out there. And so that's kind of the biggest difference, I would say. Also, because of the time we started it, language servers are built in in VS Code. Each language server is wrapped in an extension.
Rust in Production
Zed with Conrad Irwin
So for things like the rendering pipeline, there's two frames that are just big static chunks of memory that we switch between, a fairly common approach there. And Rust is actually kind of helpful for that. It tells you if you mess up because you know which one is live and what you have access to every given time.
Rust in Production
Zed with Conrad Irwin
For most of the rest, not yet, because if you think about the way it works, so let's say you're on a 120-hertz screen, you have eight milliseconds to render every frame. So rendering a frame needs to be really fast. But the average person can only type a couple of characters every second.
Rust in Production
Zed with Conrad Irwin
So it's kind of fine if it's slow to respond to a key press, where slow means you have 8 milliseconds and you're not doing that much work. And so we use a fair amount of ref counted pointers just to maintain all of this to keep the code sane, even though that wouldn't strictly be optimal, just because we're not using them often enough for it to be a problem.
Rust in Production
Zed with Conrad Irwin
Mostly the instruments, tools from Xcode have been super helpful on Mac. Linux is a little newer and there are some tools there, but I'm not as familiar with them. One of the really interesting things that's kind of on the back burner for me is that deallocating a frame in Linux can take nearly a millisecond or two, which we're like, that shouldn't be the case.
Rust in Production
Zed with Conrad Irwin
And so if anyone listening is a good Linux performance person, figuring that out would be great. But if we're running a profiler on it, it's like, why is dropping the frame taking so much time? Because, you know, you only have eight milliseconds. And if you're using two of them doing nothing, it's a complete waste of time.
Rust in Production
Zed with Conrad Irwin
Relatively few. We have a couple that are used a lot, but for most things, we just write the code out.
Rust in Production
Zed with Conrad Irwin
Mostly stylistic, I think. That's the way the code base is. But again, I mean, macros can be a performance problem. They haven't been for us. Is that because we got lucky by choosing the style or did, you know, before my time, someone chose that style and now we all copy it.
Rust in Production
Zed with Conrad Irwin
Yes, quite a lot. So macOS is actually a really nice target to develop against because similar to their reputation on iPhone, there's only really one platform you need to support. And sure, the APIs shift a little bit as time goes on, but you can look at one number that's like, this is the version of Cocoa that you have, and you know all the libraries that you have.
Rust in Production
Zed with Conrad Irwin
Linux is not like that at all, right down to the fact of about half of Linux users use X11, the old Windows server, half of them using Wayland, the new Windows server. They both work differently, quite fundamentally differently. And so we have two graphics pipelines on Linux, one for Wayland, one for X11. And that kind of fragmentation hits us at every layer of the stack.
Rust in Production
Zed with Conrad Irwin
In Z, we do have some extensions that provide that, but language server is really kind of like the thing it's based on. And then the other big piece of tech that we use a lot of is TreeSitter. So when you're trying to do things like jumping to matching brackets, we don't have to pause the entire file.
Rust in Production
Zed with Conrad Irwin
So on macOS, you want to choose a file, just open the system file chooser. On Linux, well, they might not even have a system file chooser installed. Now what are you going to do? And so that was kind of the most surprising thing for me is just how... just how customized everyone's look setup is.
Rust in Production
Zed with Conrad Irwin
Like even, even what I would consider like, surely this is just provided like a file picker isn't there. And so trying to navigate those trade-offs of like making it work for as many people as possible without going truly insane has been hard. Another good example is GPUs. Mac OS has a GPU. It works always. You just do it.
Rust in Production
Zed with Conrad Irwin
Linux has a GPU, but maybe the drivers are out of date or the drivers that are the wrong version or the closed source or the crash or whatever. And so we have a whole bunch of people who have tried to use ZLX and then it just hasn't worked. And it's like, well, when we try and talk to your GPU, it crashes. So is that our problem? Maybe. Is it your problem? Maybe. I don't know.
Rust in Production
Zed with Conrad Irwin
We have to try and find more people who know more about how GPUs work under the hood and why they might not be working.
Rust in Production
Zed with Conrad Irwin
We have a dedicated... A dedicated team of volunteers. There's three or four people who I see regularly doing Windows fixes and ports. We need a breath after Linux before we dump into the next platform. But it is something we'd like to have. Windows is going to be fun for different reasons than Linux. Some of the same problems. It's a little bit more fragmented, though less so.
Rust in Production
Zed with Conrad Irwin
But the big one is the file path separator is the wrong way around. And we use Rust's path buff extensively internally. But if we allow collaboration between Linux and Windows, we can't represent a path in a path buff because it might be a Windows UTF-16 path or it might be a Linux UTF-8 path.
Rust in Production
Zed with Conrad Irwin
So we need some kind of new file path abstraction that is not tied to the current system, which is one of the downsides of the way Rust does that.
Rust in Production
Zed with Conrad Irwin
We already have a tree of what the syntax looks like, and we can just jump you to the other end of the node. And so building on sensible foundations means that most operations are faster. We're not trying to iterate over the whole file at one time or that kind of stuff.
Rust in Production
Zed with Conrad Irwin
Cross-platform is a little manual, to be honest. So the way that the app is set up, you have kind of a platform-specific layer, and then everything else is Rust. We have a test implementation of the platform-specific layer, so we can very easily test all the stuff that's not platform-specific. And it mostly just works.
Rust in Production
Zed with Conrad Irwin
And sure, there are a couple of if statements that depend on what platform you're on, but mostly the code is the same for everyone. And that is one of the nice things about Rust. It is just Rust. When it comes to testing platform integrations, like back to keyboard shortcut handling, like when you type these keys on this keyboard layout on Mac OS, it should do this instead.
Rust in Production
Zed with Conrad Irwin
I have not figured out a better way than just getting yourself into that setup and trying it. And so to be determined.
Rust in Production
Zed with Conrad Irwin
pretty much integration tests for the most part. So we have, as you know, we have collaboration. And because the server piece is also written in Rust and also part of the same repository, we boot up the server, we boot up Zeds, and we talk through both of them. And so we have full integration tests. And I kind of like that approach because A, it lets you test interesting ordering properties.
Rust in Production
Zed with Conrad Irwin
So the test platform will reorder different async events that happen simultaneously so that you get more test coverage, for example. It also means you can refactor the code and it doesn't break the tests. That's always been my gripe with unit tests. You change the code and then you have to change the tests. What's the point?
Rust in Production
Zed with Conrad Irwin
Yeah, Protobuf is kind of like the classic solution to this. We're actually thinking about changing off them because they don't integrate with Rust super well. And as all of our code is in Rust, it'd be nice to have something that integrates better. But one of the main challenges of a distributed system is you have to be able to deal with messages sent from the past. So like, you know, forward .
Rust in Production
Zed with Conrad Irwin
And then you also need to be able to deal with messages sent from the future. So if someone is on a newer version of the set than you, they could send you a message. And you need to be able to do something in that case that isn't this crash. And so there's not much that handles that. There's protobufs and kind of a couple other big, heavy solutions.
Rust in Production
Zed with Conrad Irwin
But there seems to be kind of a missing niche for a Rust-based thing that can solve this. Because one of the downsides of protobufs is it generates a whole bunch of struct definitions. It's like, well, we have all the structs defined in our code base, and we have to map between the two.
Rust in Production
Zed with Conrad Irwin
It would be nice if we could, more like Serde, just say, and make the struct forward, backward compatible over the wire, please. But I haven't found anyone who's built that yet.
Rust in Production
Zed with Conrad Irwin
We use one, I'm trying to remember what it's called in a different part of the code base. It's like MessagePack or something like that, which works fine, but it doesn't have any versioning support. Yeah.
Rust in Production
Zed with Conrad Irwin
Definitely. And we have a build step that does it for you. But for example, if you have an error in the protobuf file, it breaks everything because the build step fails. And then it's like, oh, you can't build. And you have to really dig in and find out why that is. But yeah, thanks for the postcard link. And I will look into those. Thank you.
Rust in Production
Zed with Conrad Irwin
Yeah, there's still some CPU work involved in rendering the screen, like when you change the UI. But the CPU creates this kind of tree of nodes, very similar to HTML. And then the GPU is responsible for taking that tree of nodes and flushing that to pixels.
Rust in Production
Zed with Conrad Irwin
I guess what makes it so complicated is that there are 50 different programming languages that we're trying to support. The other thing that makes it complicated is it's actually a very fiddly piece of UI and UX. And obviously, there are lots of existing ones, so we can kind of copy them. But it's not just a copy-paste from VS Code or something like that.
Rust in Production
Zed with Conrad Irwin
There's a lot to think about and a lot to build so that it not only works well, but it feels intuitive and you can actually understand how to use it. So one of the things that's really interesting, there's kind of a debugger protocol that's beginning to feel somewhat standard, which is the one that Chrome uses for its dev tools. That's the one the VS Code builds on.
Rust in Production
Zed with Conrad Irwin
There are obviously other implementations, like some go directly to the debuggers. But what I imagine we'll do first is kind of support the debug protocol, kind of punt a little bit on the languages that don't work with that and make it a language problem. But I hope if we do that, we can kind of like language servers. We get most of the benefit with a tenth of the work.
Rust in Production
Zed with Conrad Irwin
But they're still building all the UI, so you can look at local variables, job list lines. If you start to think about all the things that a debugger can do, it's definitely a lot more than just play and pause.
Rust in Production
Zed with Conrad Irwin
So JavaScript definitely works well with the protocol because it was written for that. I know that Go's debugger, Delve, also has support for it. One thing I'm not sure about is does LLPP, which is the Rust debugger, or NEC-based language, I don't know if it supports that protocol yet, but that's definitely a debugger that we would like to have support for.
Rust in Production
Zed with Conrad Irwin
A particularly unique bug, I guess. So Zed allows you to edit files of any size. And we had a bug where if you had a file that was over about 100,000 lines long, and you scrolled down, the line numbers would not be at the right position. You'd have the line of text, and the line number would be plus or minus a couple of pixels.
Rust in Production
Zed with Conrad Irwin
And we looked into it, and it turned out that because our graphics coordinates float 32s, when we were multiplying the line number by the float 32 to try and figure out the distance from the very top of the file, it just didn't work out at all. And so we ended up having to first subtract from the first visible line and then do the offset. And then that just fixed it.
Rust in Production
Zed with Conrad Irwin
But it's really interesting to think about, how do you have a file that's so long and you could just edit it without having to rewrite the entire file every time?
Rust in Production
Zed with Conrad Irwin
Well, you can still open files that are big enough that Z will crash. We don't do anything super clever around paging things in yet. But one of the things that we do do is when you load a file, we break it up into a bunch of chunks.
Rust in Production
Zed with Conrad Irwin
And so as you're editing a file, we're not having to say, you know, take a evacuate, which is the Rust underlying string type, insert one in the middle and reallocate the rest. That would be way too slow. So when you insert into the middle, we use our CRDT to say, oh, we're just the first end characters from here, then this character, then these end characters.
Rust in Production
Zed with Conrad Irwin
And so because we're representing it as a tree, kind of like a rope, if you've heard of that data structure, that means that everything is kind of quick. And because it's a CRDT, it also works for collaboration natively. So we know for each chunk who added it and in which order.
Rust in Production
Zed with Conrad Irwin
And so even as multiple people are concurrently editing the string, everyone ends up with the same representation without having to copy and paste everything across.
Rust in Production
Zed with Conrad Irwin
Exactly. And then one of the things that wraps that that I think is pretty cool is we have to maintain where the line breaks are. And we don't want to scan through the string and figure out where all the line breaks are. And so we maintain a couple of indexes on top so that we know, OK, in this half of the file, there's 1,500 line breaks. In this half of the file, there's more.
Rust in Production
Zed with Conrad Irwin
So that as you scroll, we can quickly jump to the right part of the file without having to, from the beginning, scan and count the new lines. And so using indexes like that to help us navigate the strings means that we're basically never doing anything that's order n in the size of the file, which is nice.
Rust in Production
Zed with Conrad Irwin
Well, we've talked about one of them, which is TreeSitter. It's like syntax highlighting for arbitrary languages is a very hard problem. But once you've solved it with something like TreeSitter, you only have to solve it once.
Rust in Production
Zed with Conrad Irwin
It doesn't happen automatically. A lot of the code in Zed is built to handle those things. And so where Rust really helps is things like lifetimes. You close a file that should be deallocated, and Rust makes really good guarantees about that kind of stuff.
Rust in Production
Zed with Conrad Irwin
Yeah. So kind of back to the beginning, one of the things that I love working on is tools that help people. And I believe that collaboration in programming is way behind where it should be. And so I have kind of a secret goal, which is to get more people trying it out instead and trying to get more converts that way.
Rust in Production
Zed with Conrad Irwin
But what I found is that it's very slow comparatively to have a discussion about how to fix an issue in GitHub issues, because it's kind of like email. You send a bang, maybe you wait a few days, someone replies back. You don't understand what they said, but another few days to clarify. If you can say, hey, let's work on this together. Some people opt out.
Rust in Production
Zed with Conrad Irwin
Either they don't feel like they could be useful in Rust or they don't feel like the English is good enough. But for the people who say, sure, and join in, we can, in half an hour of working together, solve an issue that would have taken us each an hour or so back and forth on GitHub. So I have on Calendly link two hours on Tuesdays, two hours on Fridays that are just like compare with me time.
Rust in Production
Zed with Conrad Irwin
And it's kind of fun both to work on issues that other people feel are important. You know, sometimes, particularly Vim, there are so many features in Vim where I'm like, I never even knew this existed and I don't understand why you need it. But you really, really want it. Great. Let's build it together. Because then I'm getting you into Z and getting more people kind of helping out.
Rust in Production
Zed with Conrad Irwin
And you feel amazing because you've got to implement the thing that you wanted to build. And so it's been really helpful to do that. In general, obviously, the whole code base is open source. We intend to keep it that way. We want people to make Z the editor that they want. And so really encouraging people to send in changes. And they can be pretty major.
Rust in Production
Zed with Conrad Irwin
There's been one contributor working tirelessly on what he calls Zen mode, which is how do you get rid of all the UI? And so setting by setting, he's adding them all in so he can do that. But a lot of people just come in, they're like, hey, I hit this bug, here's a fix, let's do that.
Rust in Production
Zed with Conrad Irwin
For things like the GPU pipelines, because Rust is good interoperability with C, we can just call straight into the graphics APIs and just dump, here's the buffers that we need you to render GPU. Go do that. And so there's a whole framework, GPUI, which is the framework that Zed uses for all of that. And a couple other apps are using it, but mostly we built that to build Zed.
Rust in Production
Zed with Conrad Irwin
And so really trying to make sure that the community is kind of making Zed the editor they want to see, it helps everyone. And so I would strongly encourage you if you are using Zed or if you're not yet using Zed to use it and then fix the bugs that you find.
Rust in Production
Zed with Conrad Irwin
There's about 2000 issues. Some of them are tagged with, I can't remember the name, but it's like a first issue tag. Otherwise, my suggestion to anyone trying to get into anything is find something that irks you or something that you miss from the editor you were in before and build it or file an issue about it and discuss it. We're very happy as a team to pair with people. It saves a lot of time.
Rust in Production
Zed with Conrad Irwin
And so, you know, filing an issue and trying to talk to someone in Discord helps a lot.
Rust in Production
Zed with Conrad Irwin
I think what it would be is remember to keep things simple for newcomers. I've been doing Rust for about a year now and I just about feel like a beginner. So if we can make it simpler for people joining in, I think that will help with everything.
Rust in Production
Zed with Conrad Irwin
Well, thank you for organizing this and great to spend time.
Rust in Production
Zed with Conrad Irwin
And so once you've built that, then you can build an editor on top of it.
Rust in Production
Zed with Conrad Irwin
Primarily speed of fixing things. And so Zed moves very quickly compared to any other software team I've worked on. I've never been moved along so quickly. And so we're very careful to avoid putting things in the way that will move slowly, and particularly unnecessary abstractions. So we spend a lot of time trying to make the macOS integration feel right.
Rust in Production
Zed with Conrad Irwin
And if we had to each time go through someone else's crate, fix it there, then pipe it all through, the overall development time is slower. It also means that it's very concretely designed for Zed. There's no extra APIs that we don't need or stuff that we don't want. And that comes important. One good example of that is keyboard shortcut handling.
Rust in Production
Zed with Conrad Irwin
It's very unlikely that someone else would have a keyboard shortcut thing that can support FIM mode and all of the other things that we want to do with Zed. And so skipping all of that and doing it ourselves makes some things easier, less obstructions to get in the way.
Rust in Production
Zed with Conrad Irwin
And some really interesting design decisions we have, like in Vim mode, the colon key brings up the command palette. And one of the nice things about that is we can provide autocompletion. In Vim, there's no space to do that because you don't have a proper graphical UI. You only have lines in the command panel. And so, yeah, it's definitely freeing to not be in a terminal emulator.
Rust in Production
Zed with Conrad Irwin
One that JetBrains users love, you can double tap shift to bring up the, if you enable the JetBrains keymap, double tap shift brings up the command palette. And that was a fun one that we added recently just to kind of appease those people. I'm actually working on a PR with a contributor right now to try and automatically support non-Latin keyboards.
Rust in Production
Zed with Conrad Irwin
So for example, in a macOS, if you do command A on a Russian keyboard, it's obviously going to do the same, select all. But in Vim mode, A in Vim normal mode, it sees the Russian character instead and doesn't do anything. And so I've been trying to figure out how to make that work as well.
Rust in Production
Zed with Conrad Irwin
Exactly. And actually, one thing that helps us avoid planning is because we own the whole thing from the very top level Swift layer all the way down to the GPI layer, we can control how that works. But handling things like international input and making that work sensibly with keyboard shortcuts, it's not an obvious problem. And so I look forward to fixing all the remaining edge cases.
Rust in Production
Zed with Conrad Irwin
Yeah, so starting kind of from the top, macOS gives us an event-based API where they say, hey, someone pressed a key down at this point. We then have to say to them, okay, well, is this part of a multi-key key? So on a US keyboard, if you do option backtick, you get into the mode where you can type A with an acute accent on top. So we then have to integrate with that.
Rust in Production
Zed with Conrad Irwin
And that's just a weird edge case of it has a reentrant callback system. So it gives you the event, then you call back into it, then it calls back into you, which Rust does not like. It's very, very unsafe as far as Rust is concerned. But once we've gone through a few rounds of that, we know, OK, the user pressed this key. This is the character it's going to generate.
Rust in Production
Zed with Conrad Irwin
This is the key that was pressed. We then send that to the matcher. And we're like, OK, did either the character that was generated or the key that was pressed with the Modifiers or Shift Control command Does that match any of the defined bindings? If it matches the binding, we do the action. If it doesn't, then we input the text.
Rust in Production
Zed with Conrad Irwin
And one of the things that we do to try and make the keyboard bindings easier to program is that each component in the app, so the editor is one big component, has a bunch of key bindings. And then the pane, which contains the editor, has a bunch of key bindings. And so you can set them at the right level.
Rust in Production
Zed with Conrad Irwin
And that means you can have multiple different meanings for a given key binding, depending on where you're focused right now. which is really important for something that's complicated as that.
Rust in Production
Zed with Conrad Irwin
I think there's kind of two approaches to think about. One of which is, as you look at the world today, everything is collaborative by default. People are designing in Figma, docs are in Google Docs, and then programmers are still stuck, kind of each person editing their own file and then git commit, git push. Oh no, what happened to my git commits?
Rust in Production
Zed with Conrad Irwin
So as you mentioned, we have the ability to do multi-key keystrokes. So if you do a G and then an R, it goes to all references in that mode. And so that piece is a state machine of like, okay, is there a key? Like, could it be a pending match? Is it something else? But once it's happened, it really just happens. And it's kind of like a callback into the rest of the code.
Rust in Production
Zed with Conrad Irwin
And so you hit GR and it goes, okay, run the final references code. That's going to probably, in that case, kick off an async task. So we don't want to run that on the main thread because we're going to communicate back to the language server and ask it questions. So we kick it to the background, wait for the response, and then update the UI in response to that.
Rust in Production
Zed with Conrad Irwin
So using a lot of the ASIC Rust stuff to make that not happen on the main thread.
Rust in Production
Zed with Conrad Irwin
The, I'm not sure exactly if AsyncRust solves that problem, but the GPUI framework has some tools for it. So mostly the UI is a tree. It's really the only data structure you can have in Rust where you have the root view that maintains handles to all of the subviews.
Rust in Production
Zed with Conrad Irwin
But obviously often you want to be able to refer back up the tree of like, okay, I'm an editor, but I know I'm rendered inside a pane or I know I'm rendered inside something else. And so we have a system of kind of strong pointers and weak pointers. And that works pretty well for avoiding it.
Rust in Production
Zed with Conrad Irwin
But it is one area where I wish the compiler was better because it can't tell you, hey, you have a smart pointer cycle, right? It's like you have two strong pointers in each direction. But as long as you're somewhat careful to maintain the order in your head, it's mostly fine. And we don't really have that many problems with memory leaks, at least none that we found yet.
Rust in Production
Zed with Conrad Irwin
Yes. So we have a whole bunch of smart pointers for view components because we need to be able to reference them from the renderer and also from the view tree. And so they need to be kind of like an arc reference from multiple places. We don't use an arc directly. Instead, we have our own wrapper around it that's a little bit more type safe for some things.
Rust in Production
Zed with Conrad Irwin
And so if you do have things like a double bar, you get a panic instead of a compile time check. And so it would be nice if the compiler supported that, but I understand why it doesn't.
Rust in Production
Zed with Conrad Irwin
So the framework gives you, it calls them a view, and you can read it or update it, just like kind of an ArcMutex. What's nice about the view is it supports a bunch of other things. So a view can be rendered, a view can be converted into kind of like a black box, any view that the renderer can deal with.
Rust in Production
Zed with Conrad Irwin
It seems ludicrous that we don't have real-time collaboration for code. And so that was one of the key kind of design things is how do we build real-time collaboration incorrectly? But we knew that if we wanted to persuade people that they wanted to use that, it also needed to be better than what's out there.
Rust in Production
Zed with Conrad Irwin
And so, you know, when you're given a view handle, like there are some things you can do with it. It's not just a knock around an arbitrary struct.
Rust in Production
Zed with Conrad Irwin
Yeah, I mean, yes, but. So if you're using GPUI to build an app, you definitely would use it. And it gives you the whole suite of tools. If you were just trying to pull out bits of GPUI on its own, that's probably not a bit I would pull out, just because it's so tied into the rendering system, if that makes sense.
Rust in Production
Zed with Conrad Irwin
I would say we very much are on the rebuild side of the spectrum. So there have been a lot of things where we haven't used what's out there. I think the other big one is Tokyo. So Tokyo has its own async runtime. But on macOS, it's not main thread aware. And macOS has a whole bunch of cool scheduling stuff to do when you have...