Menu
Sign In Pricing Add Podcast

Glauber Costa

Appearances

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1003.35

And you have to put it in context as well, that Pekka and I grew up in the Linux kernel community, right? And a lot of people say this about SQLite, so SQLite has to, and with that, I disagree, SQLite being what it is, has to be kept as this very small thing, because that's the whole beauty of SQLite.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1020.501

And again, I disagree with that because if you look at Linux, Linux started as this project that would only run on x86 computers and et cetera. And today it is an operating system that runs everywhere. It runs very well on the server, you know,

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1037.523

Pekka and I worked together in memory management for very big box machines, like how to make sure that when you have two terabytes of memory in 2010, which was a lot of memory back then, it's still a lot of memory today, but like those algorithms to manage memory don't take three minutes and hang up the system. It's a problem that doesn't happen in embedded devices.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1057.362

And Linux still runs on embedded devices. So through this big community, you actually can create something that runs well everywhere and don't necessarily have to make those compromises. So we don't agree that SQLite has to be protected and has to be maintained by just a single individual to be the success it is. But look, who knows? Maybe I'm wrong. Maybe I'm right.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1079.139

We act based on what we believe. But we don't believe there is anything wrong with what they're doing. We just think that it could benefit with a different direction.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1094.175

100%, and we will. And I want to clarify that now, because with LeapSQL, it became very clear to us in maybe a month that that was not going to happen. So with LibSQL, I joked about it today. We had this goal of replacing SQLite for 10 minutes because it's a dream, et cetera. It's obviously a big prize. It's obviously this big prize and et cetera.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1118.573

As I wrote in my blog post, were we bummed about it? No, because we built a pretty successful business on top of LibSQL, which is today our company, Turso. So for us, it was it was this exercise of like if you heard the expression shoot for the stars, because even if you miss, you're going to hit the moon.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1139.229

For us, it was like, yeah, you know, it turns out I don't think it's going to be possible for us to replace the light with the fork. But that's us hitting for the stars. We didn't hit the stars. We hit the moon. The moon's great. Again, we built a super great project. We're having fun.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1153.077

Leap SQL brought something to SQLite that wasn't there, which was like the serverless, running this on serverless environments on the cloud platform, syncing databases. One of the things that we do with Torso is that you can sync your databases between different devices and backups and all of that. And this is going great. I mean, this is going fantastically for us. And so we were never...

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1176.076

sad or disappointed that our, you know, that this thing that crossed our mind, imagine how amazing it will be if, you know, people just start using our fork instead of SQLite. We're never disappointed at that. But when we put the rewrite out, it became very clear. And I've been using this analogy from chemistry that the reaction was right. Like LibSQL was something that the market was saying,

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1202.453

clamoring for. I think what it became very clear seeing the reaction of Limbo now is that it's just that the activation energy is not enough to make the reaction happen. And after talking to a lot of people, and I want to tell, there's a whole story of what happened after we announced this. I talked to a lot of the people who came, started contributing to the project,

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1227.321

And what it became clear is that they all wanted, like, they all agree with us that, like, we need a better version of SQLite. I mean, it would be fantastic if you have something like SQLite, but there's this, all our original vision. But the fork just wasn't differentiated enough. It just wasn't, you know, ambitious enough. It just wasn't bold enough to get people to come and contribute.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1251.622

Furthermore, it is actually very hard to do because one of the things you did mention, like the SQLite's test suit is fully proprietary. So once you start making changes that are very deep into the core, you start hitting a bunch of issues.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1266.009

And some of them is just like yourself saying, I know that the best way to solve this problem is this, but I'm going to solve it in this other way, which is almost as good, but not great. but it's less risky. So you start having this on your mind.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1280.872

So the fork never really unshackles you fully to go pursue this, which is why when we announced the rewrite, what we saw, like, again, was something I've never seen before. We got 8,000 GitHub stars in a week, essentially.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1293.784

We had for the first, what I consider to be for the first time in the history of computer science, a Hacker News comment section that was mostly positive with, you know, just a... I think maybe one or two comments only that were trying to denigrate us, but the rest very supportive. We had this fantastic, inspiring story of this individual who is in prison at the moment.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1317.3

And he was one of the first people in the United States to be granted access to the internet in prison. He can access Discord, but not X, by the way. Maybe say something about it. But he is now the fourth top contributor to the Limbo project.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1332.188

And many, many other things. I mean, the level of contributions that we started to get, we started getting people running this on the browser. A week after we announced, it was running on the browser already, just like this compiled to WebAssembly. all from contributions from third-party people.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1350.347

Again, Leap SQL, we did get a lot of contributors, but they were all contributing to the drivers, to the server, you know, things at the margin, which they're important as well for an open-source project. But with Limbo, we actually managed to see our dream, like the 10-minute dream come true. Like, we're going to build a community of people coming here and writing this database with us, right?

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1372.578

So it... We revisit it. Now it's no longer a 10-minute dream that went away. We truly believe now that once we get the momentum going, we can replace SQLite. And that's the goal.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1427.345

Your intuition is correct, by the way, which is one of the things that all things considered, we went with the fork. Because at least there is some level of guardrails that come with the fact that at least you know that this code that you're importing was tested by this proprietary tester.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1446.015

I don't think your intuition is incorrect, but something amazing happened. And by the way, the way I look into the situation, another thing that I want to make it clear, I don't actually think that we made a mistake with the fork because if we were rewriting back then, there were many decisions that we made today that we would not have made But now we did because we had the experience.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1468.828

We've been running the Thurso platform and we learn a lot of things. So it actually allows... I almost see it as a prototype and it actually allows us to make tremendously good decisions with the Limbo project. And one of them is exactly the testing. What happened is that... Becca and I got hooked into something called deterministic simulation testing.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1488.75

And deterministic simulation testing is a very niche thing that most people have never heard about. I had heard about it a couple of times, but I didn't truly quite understand what it was until I met Yoran. So Yoran is the founder of of, and I can say this, I mean, even though I am the founder of a database company, Yoran is the founder of the most amazing database company I've ever seen.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1510.824

Although now I think with Limbo, we have a chance to reclaim that title. Yoran and I are going to fight about it. But, you know, ask me a month ago, I would say, man, I have to give it to him. This is the most impressive database company of all, Tiger Beetle. Tiger Beetle is a database, just to keep it very brief, that is designed for financial transactions. So it's completely written in Zig.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1531.331

And their goal is to replace entire systems, the bespoke systems that most financial institutions have to process transactions. So if you're selling to banks and financial institutions, if your goal is to replace the backbone of world's commerce, It's pretty hard to do, right? And one of the challenges that you have is just this, like, how do we trust this thing?

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1556.775

And you're unfound the solution for that. The solution that you're unfound was to write Tiger Beetle entirely with a technique called deterministic simulation testing. Deterministic simulation testing is essentially a fuzzer. So it will generate a bunch of inputs. It randomly generates as many inputs as it can on the space of possible inputs. And this is the disadvantage.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1581.153

You kind of have to write software in a special way to lend itself to deterministic simulation testing. It's very hard to boat it to an existing code base, right? But you write it in a way that every single operation that you do on IO, on thread scheduling, everything that happens goes through a special interface that abstracts all IO, and then you plug a simulator.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1605.802

When you're testing this, you plug this into a simulator, and the simulator is included in the code base. So what the simulator does is that it's going to explore the space of possibilities, create the most arcane, impossible situations ever, And then when something breaks, it gives you the exact steps deterministically that everything in the system had up until that point.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1628.039

So debugging those problems become very easy. So in record time, Tiger Beetle managed to create a system based on deterministic simulation testing, which is their database. And the stories that Yoram would tell is like, look, we found this bug that would only happen if you would call fsync on a disk, and then fsync would return an incorrect result.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1649.331

And at the same time, a packet would come from the network. I mean, he would describe the most complicated scenarios. Oh, and the end result of that, the simulator gives you a seed, and you type that seed into the simulator. Now you have every single step that happens to make that happen, right? So when we... This is one of the things that led us to believe, what if we try the rewrite?

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1669.901

Because we knew that we could now try the rewrite using deterministic simulation testing. And then we also partnered with a company called Lantitis that offers like a...

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1680.817

almost like the integration test analogy for deterministic simulation testing, which is a full system version of that, that will simulate things between machines, it will simulate the network, it will simulate hardware failures and stuff like that. So whatever bugs our simulator does not catch, Usually we just, you give it to antithesis the next day, antithesis catches the bug.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1703.2

So we knew that with that, that was the missing part of the puzzle. And then with that, we will be able to actually create something that probably even surpasses the level of testing the SQLite has. But here's the catch. It is easier to do this on the rewrite because you have to write the system with this in mind from the get-go. It's not something that is easy to bolt upon.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1732.338

All of it. Yeah. So for people who are interested in DST, but don't want to go through the pains of just rewriting all sorts of things like that, I truly recommend taking a look at antithesis. Because, I mean, antithesis is amazing. It's the second... is the second next best thing. We use it in conjunction, it's not an either or.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1752.653

The analogy that I have is that our own DST is like unit tests, like we can run centuries of possibilities in two days. And it's very fast, all things considered. And Antithesis is like integration tests. So you wanna have both and we do have both. And Antithesis has been a great partner for us. But the problem with deterministic simulation testing

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1776.781

is that once you start importing other crates in Rust, for example, you have no idea what those crates are doing. Those crates are calling IO. So you kind of have to write everything. So it's not even that you have a crate for deterministic simulation testing. You might not even want to import something super simple 100%, but you don't want to import anything that does IO, that has a timer.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1800.529

You want to write those things. So it's something very hard to do for a general purpose testing for a database like Tiger Beetle, for a database like Limbo. You know, it's worth it. And the scope is quite limited. Right. And you don't want to be importing crates all the time anyway. You know, Tiger Beetle is way more insane than we are.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1819.06

I mean, they have a policy that they just don't have dependencies. That's it. They write every single piece of code that we try to be a little bit more flexible. But we will not import a crate anymore. And again, it also depends because SQLite comes with a CLI.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1836.111

If you want to import a crate that does whatever crazy stuff to implement callers in the CLI, that's fine. But for the core of the database, we try not to import anything that could potentially do I.O. because we want to make sure that everything goes through the simulator.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1982.053

It does.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

1999.499

Yeah, so again, the simulator itself, you just write a couple of scenarios. And again, it's not truly magic because you do have to write the simulator, right? So the simulator just writes, imagine, for example, a workload, and the workload is like generate a couple of queries and then write, etc. But the simulator will include a fuzzing element to that.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2020.636

So we will, instead of like generate this query that I wrote in the unit test, you give it almost like a query generator and then it starts generating random queries. Then you start in the simulator itself. What happens, imagine this, if you want to write to a file in any software, you would call a operating system API, write.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2040.388

If we're talking about Rust, it's like a FS write, and then you write to the operating system. When you're writing software for DST, you don't do this. You have your own IO interface abstraction, and then all of your IO goes through this. So when you're running this in production mode, your abstraction for write just called the operating system write.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2062.651

But when you're running in deterministic simulation testing mode, your abstraction for write runs the simulator code that will start injecting failures into this. And again, injecting failures in a deterministic way, because then if the query fails, because you injected a IO failure at that moment, you will be able to replay that session piece by piece.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2092.743

Is that the guy that, is that the guy that. It's barely inconvenient. Very easy, barely inconvenient. That's right. Super easy. I love that guy, man. Wow, wow, wow. Super easy, barely inconvenient. Wow, wow, wow.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2104.83

So how do you rewrite SQLite with the confidence that it will actually have the level of trust that SQLite acquired? It's super easy, barely an inconvenience, right? All you have to do is write the determinants.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2139.295

First of all, compatibility in a project like this, and this is the experience that we gained at Scylla when we were rewriting Apache Cassandra. Compatibility has to be like a one-way street, right? You don't want to shackle yourself and say, I will always... Because sometimes, for example, to implement a new feature, you have to do it in a different way.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2157.304

So it's usually something, hey, we're going to offer you the same feature set as SQLite. We're going to read your SQLite files. We're going to execute your SQLite code. And if you're not using any of the specific features that we have, we can generate SQLite files as well. But the moment you start using new features that are only present in your implementation, it's impossible to be compatible.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2184.39

It's just not possible. So that's it. It's limbo so far, and this is something that we want to keep. The language is the same. Again, we want to keep the language. We want to keep the ABI. We want to be able to load SQLite extensions. The file format, obviously, it's the thing that defines SQLite. So obviously, again, we might add to it in the future, but we'll be reading SQLite files.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2207.751

Normally, and we are bytecode compatible as well. So one of the other ways in which we test Limbo is by now out of the simulator, just generating random SQL statements, and then seeing that the bytecode that is generated by Limbo is the same bytecode set of instructions that is generated by SQLite.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2229.734

So that doesn't catch bugs in the implementation of the bytecode, but at least you know that the query plan is the same and et cetera. So this is yet another way in which we've been testing to make sure that it's up to standards.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2254.568

Yeah, so async is something that a lot of people misunderstand. Async doesn't mean, for the rust-minded people in the audience, to be clear, async does not mean that we're going to use an async runtime like Tokyo or like others. I actually personally wrote an async runtime when I was a data dog that's still around called Glomio. We don't use any of that.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2278.334

And again, part of the reason we don't use any of that is that we want all of that to go through our simulator. So if you look at the limbo code, it's just really sync rust. It's not async rust, right? So all the async means is that when you call an operation, if that operation is not ready, it will return to you instead of blocking.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2298.105

And the main, now I'm getting a little bit too technical, but let me just give you the full context. The SQLite C API has a bunch of functions, but the most important of them is called SQLite step. And SQLite step is essentially take another step, take another step in processing this query for me. And this is a blocking function.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2320.233

So what that means is that if you call SQLite step, it will block until it resolves. let's say in that step you want to execute 4, 5, 10, however many bytecode instructions, until it does that, it will block. In limbo, if you detect that you're not ready to execute those bytecode instructions at that time, you just return something saying no. Call me back later.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2343.596

So that is essentially what async means. And then with that, it becomes very easy to plug something like Tokyo on top in Rust or run it on the browser or whatever, because you can call those async functions. And if it's not ready, it will not block. That's all that means. But it doesn't mean that we have to use the async runtime internally. In fact, quite the opposite way.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2373.997

Well, it isn't that fast. And I ran a query yesterday, for example, that took 10 minutes to run. There was a query with five joints looking at all the user data about users who are signed up to the Turso platform. it takes a long time just because the query is complicated. SQLite is very fast for CRUD style, just like, hey, here's the key, give me the value. Sure.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2399.556

But it's not super fast for analytical stuff. The first thing that it unlocks is just this, queries that are much more complex that you can run in dashboards and things like that. Also, for the serverless environments, Durso being one example, and for clarification, our product will be renamed to Durso Cloud. And we have all the intention in the world to rename Limbo to just Durso.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2423.954

Limbo is just not a great name, just a parenthesis here. And it just, you know, the story, it's all in the blog post, but we never expected this level of support. So we just came up with a made up name for it. That means nothing, right? But now we'll address that. But our product, the Durso Cloud, allows you to do like serverless SQLite on the cloud.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2443.726

You have a bunch of HTTP requests there in the middle. It allows you, for example, to host your data on S3 because if you have your data hosted on S3, your query now is not necessarily super fast because you may be hitting a page that is not local, right? So it allows you, for example, to run SQLite with partial storage, like with most of your data on S3 and some of the data locally.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2465.35

It allows us to run on the browser because the browser, as you know, is a very... sync unfriendly environment. Like if you block, you block everything. The page just doesn't load. So all you have in the browser, you have to be async. So there is a lot of things that it unlocks in terms. In fact, the one thing that Limbo doesn't even support transactions.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2485.88

There is, you know, it's very, very early, which for a lot of people who came to contribute, they actually saw as an advantage, you know, just because you come with this energy and there's so much to do. But we're not even yet at the point there is support transaction. And the one thing that is already different and is resonating a lot is the fact that it is async.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2518.262

Technically, we put a lot of thought on the product. And the full story there is that, first of all, we consider, just revisiting, especially if people are joining now, we consider rewriting SQLite from the get-go anyway. So this was always on our mind. This was always on our mind. And... We added to Leap SQL.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2535.902

So one of the things, if you download Leap SQL today in the fork, Leap SQL comes with vector search out of the box. You don't have to install any extension. You can do your rag pipelines of SQLite out of the box with vector search. It was very, very hard to do. It was a ordeal to actually get vector search working. And there are lots of things in the syntax

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2557.99

earlier when i was saying you might do it one way but then you end up doing this other way which is more conservative because you don't vector search was the thing that i had in mind like many things in the syntax they're okay but they're not great because we had to be a little bit more conservative and then pekka started thinking and this is all on pekka he started thinking okay how would that look like how would that look like if we were to rewrite it and

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2584.469

Our goal was something like, hey, look, if this project keeps going and it goes well and it's an experimental thing, maybe in two years, in three years, we can make something out of it. So we were very thoughtful about the technical decisions that we were making. But everything around the presentation was just like, yeah, whatever, man, whatever. So, again, this was not on our company's GitHub.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2612.996

This was on Pekka's personal GitHub. He spent 10 seconds thinking about the name and then just wrote on and said, it's going to be Limbo because it's a state of confusion. Like, we don't know what we're going to make of this. Like, the whole story of the name Limbo was essentially Pekka. What? What are we going to make with this? I have no idea. I just want to experiment with this concept.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2633.528

The logo, he just asked Chat2PT to generate. We took another five seconds. So there was not a lot of thought on the presentation and publishing, but there was a lot of thought about the technical side because we thought that maybe in two years, maybe in three years… There is a play for us once we are a much more established company to tackle rewriting C. coli.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2655.866

What we do not expect is that what we decided to do is just make it an official torso experiment. But if you read the posts in which we were announcing this, we were very clear. We don't have a roadmap for this. Don't ask us what our intentions are. There is nothing that we want to do with this one on the short term. This was an experiment on Pekka's GitHub that actually did pretty well.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2683.326

It did pretty well based on two metrics. We got 1,000 GitHub stars and we never talked about it. We never talked about it. And we got two incredibly good engineers that started, one from Red Hat, who is now with us. I mean, so we hired those two engineers. We got two incredibly good engineers that started contributing to the project. Again, it was just a project on Pekka's GitHub. Mm-hmm.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2705.387

So we knew that we knew there was something there and we decided, let's just now publish it as a official torso experiment. We wrote in the blog post, maybe if this goes, if that goes well in six months, we can start pouring some of our resources and interest and then we're going to see what happened. So we had. We had some idea that this will be useful at some point.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2726.965

We just did not know it will be so well received to the point that it was to the point that we changed our entire company strategy now to be able to pour our whole weight of our resources behind the project. Right. And that's what we're going to do.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2742.02

We're all in. In fact, we wrote a blog post last week telling the story, a lot of what we're discussing here. Hey, I mean, this is – look, in my wildest dreams, in my wildest dreams, I would expect maybe this to gain like 2,000 more stars in a month or so than from 1,000 goes to 3,000.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2761.378

Maybe a couple of other engineers that would come and contribute as well and start slowly, but we would see some potential on it. That was my definition of success. And every single metric that we thought Limbo could be successful at, we saw three times more, four times more than what we anticipated. So we decided to go all in.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2779.806

And there is a blog post that we wrote recently with all the changes that we're going to make to the platform to allow us to do this. With all the changes that we're doing to the company, we had a lot of reorganizations internally. And this is really something that we decided in a couple of weeks in January. Because we just like, how can we ignore this?

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2798.451

I mean, it sounds, it seems very clear to us now that the world at large really wants a evolution of SQLite. Like the signals are very strong. So I think we just need to get behind it, right?

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2825.146

Yes. And again, those two individuals in particular, they are now working for us. And we hired them already last year because, you know, they're great. And it's back and I was like, man, if Limbo has no other value, it's a great hiring tool for a company, right? Because it's attracting this kind of people. But those are people, of course, that we have direct access to.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2842.298

So we talked to them extensively and all the others that came as well after. We had 32 contributors to Limbo the day we announced it as a torso experiment. As you know, Days later, we had already 60 and we're reaching 70 now. And again, a lot of that very core contributors doing a lot of great work. So Perry and Yusei, those two contributors are the first ones that we talked to.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2870.773

And the story is always the same. The story is always the same. And it's exactly the thing I said. They were very excited about the prospect of a better SQLite. They knew about LibSQL, but LibSQL never enticed them because it just wasn't... In UC's word, it just wasn't crazy enough, which I translate to bold. But it just wasn't...

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2893.955

it kind of caps where you can go because of all, you know, you still shackle by the fork. So when they saw this thing that, and I think you add this sprinkle to that as well too, man, there's so, this is a new project. So there's so many things to do, which engineers get attracted to, to some extent as well. But if you talk to them again, it's always the same story.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2913.292

I love the vision and I already loved the vision when I heard of Leap Sequel, but this thing here, is just the right amount of ambitious that I want to start devoting my time to that and I want to be a part of that as well.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2923.838

So that's exactly, and we wanted to, one of the things that we said a lot in the early Leapsicle days is that we said that a lot because we heard from a couple of people, a couple of companies even, we love the idea of what you're doing because it gives us a seat at the table.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

293.714

Thank you. It's great to be here.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2938.402

And then my interpretation of that is that it turns out it's actually pretty hard to give people a seat at the table if you don't own the table. So Limbo gave us this. I mean, it's our table now. Now, it's a table that is modeled after another table. But we own the table and now we can really truly give those people a seat at the table.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2958.468

I mean, they're coming, they're writing code, they're contributing, they're reviewing code, they're helping with the direction, they come out with ideas, things that we never consider, things that we never consider, things that will not be a priority. Like the thing, SQL already limbo runs on the browser. Somebody showed up today, today.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2974.124

Oh, I got LibSQL to compile, run the limbo CLI in WebAssembly and now I'm distributing the CLI. It's not even something we would have done, but the person took an interest because it doesn't have a ceiling. You essentially can take this anywhere.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

2994.548

Leap SQL came to be a little bit, and a part of that because, again, as I said, after we had this dream of replacing SQLite for 10 minutes, and then it became clear that it wouldn't happen. So we had this something that was very valuable. We knew it was valuable, but we had to struggle a little bit more with what do we do with that. So the Leap SQL project today is, again, it's two things.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

301.274

It sounds great, though, Adam. You sound great. Thank you. You mentioned you're on the tail end of the flu, but you don't sound like it.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3019.429

It's a fork of SQLite. with some of those changes like vector search, but it is also, and it's all in the same repository. It's all part of the same project. It is also a server implementation to do serverless SQLite over the wire. So that is what, that is what lib SQL is. This will, it's, it's an open source project. So we're still going to be maintaining that, but the, the,

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3042.219

The client side part is going to be full Limbo. We intend to eventually get all of those things we've done, like vector search, all of that. Leap SQL is still maintained for the time being because, again, we have a business around it. We have features that we depend on, but our goal is to eventually port all of those features to Limbo, which, again, we plan to rename to Turso.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3064.897

And then the client is that, and Leap SQL can become just the server implementation as an open source alternative to do that.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3227.485

There's so much. There's so much. Because one of the things that we heard as we launched Leap Sequel, for some reason I'll never forget that individual. So we create this Discord community. We announced Leap Sequel. People came and one person in particular, but others said the same thing as well. Give me better schema changes and I'm switching to this tomorrow.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3249.642

So one of the areas, for example, the SQLite is really bad at is schema changes. You can make schema changes, but you cannot alter the type of a column. There's all sorts of limitations like that, which is something, by the way, that we made better in LibSQL. So LibSQL does better schema changes than SQLite, but it's not better enough. Same story, because we were limited in what we can do.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

326.221

Oh, happy to do that. And the story, in a sense, it's still the same story. So I think you hit the nail on the head there. It's just really how it started. I don't feel this is a completely different thing, what we're doing now. The story there is that we were using SQLite in our company. So Pekka and I, Pekka is my co-founder.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3273.073

So we want to run... replication, like native replication to the browser, which is something like people have been asking us for a long time. Imagine you have this SQLite database running on your browser that can then sync with an external server or whatever and just get pages on demand. That's one of the things we want to do. We want to tackle the problem of schema changes.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3292.003

We want to tackle the problem of write throughput because SQLite is a very bad database for write operations. We want to make SQLite much better for analytical workloads. So there's just so much. that we believe we can do, right? But together with all of that, there is also the boring work of, you know, we don't even support transactions yet.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3312.658

And again, some of that you can do in parallel, like a sync, like browser, but some of them is just time in the saddle, right?

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3322.987

Nine months to a year. Nine months to a year. For two reasons. First of all, because of the deterministic simulation testing. Because it allows you to move with a lot more confidence. Imagine trying to get into the thought process of a database writer.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3339.705

It's always like I want to make this change, but I don't fully understand the impact that this change will have in all of those environments that I don't control, like running on an embedded device here, running this or so making changes to systems like that again. And those are the kind of systems that we dealt with at Linux.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3357.535

sometimes Linux has a lot of weird stuff in the process of development around the idea of like, we know that this thing can break on a processor that only three people have in the world and it's not supported anymore. And we don't want to break those things. So we always move very, very, very, very carefully. And the deterministic simulation testing just allows it to make a lot faster. Like,

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3378.891

paired with antithesis, which is our integration testing in this analogy, it just allows you to move so much faster. The second reason for why we believe like a year is the reasonable timeframe here is a SQLite, as it turns out, that's why one of the reasons we believe And forking and rewriting is not the biggest code base in the world. SQLite is not that complex.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3402.678

If we were rewriting Postgres, it's a completely different story. But rewriting SQLite, it's actually doable in a year. And also, as I said, we're not going to do this in Q1 because we're treating our first quarter of the year as a transition quarter. to allow us to finish other work so that we can be free to do this.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3423.018

But we're going to have at least seven, eight engineers just working on it full time. So imagine that you have seven people working on a code base that is less than 200,000 lines big for nine months with a deterministic simulator that catches all possible bugs that you can muster. We believe it's a very reasonable timeline, like nine months to a year.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

343.751

We founded a company for, if you happen to stumble upon it, just mentioning here by name called ChiselStrike. And it was in the API space. We were doing a bunch of things and we were using SQLite very heavily in that company. And the reasons we were using SQLite were reasons I think that a lot of your audience will understand and relate to.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3449.052

Yeah. We want to, we want to release a 1.0 much earlier than that, much earlier than that, because we want to, you know, we'll see, like we still believing in, in like release, release often. Just, just put it out there.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3463.443

A lot of people are going to be early adopters. A lot of people, I think, like our thesis that, you know, it doesn't even support even without transactions. There's a lot of people that with a little bit more support on the read side, they can already use it because a lot of workloads for SQLite, like you get the file, like you don't write to the file.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3479.151

You get the SQLite file from somewhere and then you just run a bunch of stuff on it. So even without writes being fully supported, we believe there's a lot of use cases that this unlocks. So we want to be very aggressive with like making releases. But the moment we're going to say, hey, like this is stable.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3495.981

And look, we can actually even give us more time because it's a great commitment to say something is stable. I mean, you know, people trust you that this is going to work. So we can take even more time than that to say, hey, this now has our seal of approval. We're going to support all of that. But much earlier than that, it can already be production ready. Yeah.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3525.436

So our business is going to change. We announced recently a lot of what we need to do is that we need to simplify our product a lot. So we had a very... Very hard decisions to make. So again, it was a tough time for us to go through those decisions. But we knew we kept the mission in front of us.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3545.386

We kept this idea like we need to make those changes because I think the community now trusts us to rewrite SQLite, which is like the biggest prize we believe. So we have to make those changes. A couple of features, a couple of features that we have. Some of them that I love. A lot of users, unfortunately, got quite upset with that, obviously. I mean, the features that they use and came to trust.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3570.301

We will discontinue. So the way we're doing this is that if you're a paid user, after a certain cutout date, you're going to be allowed to keep using those features. But new users, new signups, anybody else who is not a paid user at that point will no longer have access to those features. With that, we believe our platform will become a lot simpler. We're still going to maintain our platform.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3592.271

But again, we think that having a single person just running the platform will be enough. And we're not going to be investing in having new features into the platform. So the platform will essentially go, which is the Turso Cloud, will essentially be a place where you can still run. Like if what we do today is good for you, great.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3611.782

And what we do today is two things very well, especially after those features will be discontinued. You can access SQLite over the wire from serverless environments. So you have a serverless managed SQLite database. And we have our features, point-in-time restore for backups, branching. We have all of that that a serverless database needs.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3629.247

And you have syncing of databases between devices, servers, et cetera. So you can start with your SQLite file and you can upload that file to Torso and then replicate that to other SQLite files that you own. That's the platform. We're not going to be investing too much in new features, in front-end features, in quality of life features. The platform will essentially go into a freeze for a year.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3652.499

And the platform is pretty good. It's at a position that we can afford to do this. And then for one year, we're going to go all in in getting Limbo to replace SQLite, right? And that's it. And then at that point, Limbo then becomes the thing that we run on the platform. And we have the money. We have the running to do this. I mean, we It's all accounted for.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3671.032

So if we managed to, which I believe it's very doable, to get limbo in a production-ready state in 15 months, our plan still works. But 9 to 12 is what we're thinking here.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

368.537

Look, you would just do npm install, the name of the package, and then everything's there. Your database is there. Everything is there. Everything works. You don't have to install anything. So SQLite for us always had this appeal of a database that is just always working, and then you can build stuff around it, and it's just always there. So there was always something that was very appealing to us.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3690.938

Terso Cloud doesn't go away. What is Terso today becomes, it's called Terso Cloud. And then Terso, the embedded database just becomes Terso. And... Again, this is a lesson that we learn. Not that we haven't heard this before, but I think we're just stubborn. People kept telling us it's very hard to create two brands. It's very hard to create one brand. Two brands is even harder.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3717.186

But the reason we kept those things separate is And the reason LibSQL wasn't called Durso was exactly because we wanted to give people the sense of, you know, this is a welcoming community. We have our business, but your interests will be heard, right? Like Linux. We model a lot of that on the experiences we had at Linux. Very few people care about this.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3739.341

And that's the lesson that we learned, right? Just almost nobody. Of all the reasons that we heard for people not contributing to Leap Seek, this was never something that showed up. Never, ever, ever, ever.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3752.392

And then when we announced our changes to the Thurso platform, we had many questions from many people about, hey, this feature, they're going to discontinue this other thing they're not going to support. And what happens is... Nobody asked about the name. Nobody said, well, with the name Turso, nobody cares.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3769.642

So this idea that we had that we have to keep things very separate to create the welcoming community, we learned that nobody cares. So we're going to take the opportunity here just to fix it. Because again, it is true what we heard from many of our advisors. Creating a brand is already incredibly hard. Creating two brands is very...

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3789.5

very hard and the people in the know, they know, but people who are just hearing about it, now you have to explain that this is this and this is that. We're going to just consolidate the names. Turso today, again, is the Cloud offering that is going to be renamed to Turso Cloud. Then what is Limbo, which is the client offering, will just be renamed to Turso and that's it.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3811.113

We're still welcoming, we still want people to come and build this with us. We just learned that the name doesn't matter.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3835.252

It does. And that's why what we decided to do is never port the server code to Limbo. Because then the server can be kept completely separate. And again, I think the reason we had the server and the client together was exactly because it was clear to us after a month or so that we would not, with LibSQL, with the fork,

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3858.544

have achieved our goal of really just replacing SQLite, this goal that we had briefly. So the best strategy then became have the server code here as well, because now there are other things you can do with this open source project. But with the success of Limbo, again, it's back on the table and it's a very realistic goal that this will replace SQLite.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3879.73

So we just revisited the strategy and said, this has to be just the client-side library exactly because of this, because we don't want to be in a position. And the name was a part of that. We just learned that it doesn't matter, right? But we don't want to ever be in a position. We want to design things in a way that we are never, ever, ever in a position to even think about

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

389.133

However, we also stumble upon this thing a little bit about SQLite not being open source. So let me address that. SQLite is a public domain software, which is technically a difference that doesn't matter. I think only a lawyer would be concerned with what is the difference between public domain and open source.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3899.75

We would like to, you know, there's a contributor coming with this code and maybe we don't want to merge it because, you know, this might affect our business. You never want to be in this position. In fact, we want to make people in the community maintainers as well. We want to we want to have people hopefully like Preston, the person that I mentioned is in prison and others.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3917.837

They're doing fantastic work. I mean, if they keep working for another six months. or a year, and you see that they're committed to the project, they should be maintainers as well. And they should have the ability to just merge code without talking to us. That's what we want to see. So the way we're going to structure things to make that happen is that our business now has to be just the cloud.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3938.354

And then all the code for the cloud become, that is a separate project. That's going to be its own thing. And then the client, SQLite, for example, they don't have encryption at rest. They don't have a bunch of things because they do try to build a lifestyle business around it, as far as I understand, by selling those things. They're very specific. We want all of this in the open source for us.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3959.624

We want all of this. We want encryption at rest. We want all sorts of extensions. We want everything that runs on the client to be 100% open source.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3981.396

I think, yeah, I think if this was a consumer business is when your mom calls you to say that she's using it without knowing that this is the software they work with. I don't think my mom's going to use Limbo or Turso or SQLite or whatever. So I can't use that.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

3994.548

So the criteria for us is really when we can see somehow through some fuzzy metric that we've got, like we have a billion databases out there, right? So, you know, SQLite has a trillion databases. And I think that we don't think we're going to replace SQLite in a year, right? That's not the goal, right? But we think that in a year, we can get to this point where, hey, we got our first billion.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4016.201

This is clearly working. You do this for another two years, you do this for another three years, you get there. So the first milestone that we have in mind is that when we can, through some proxy, because a lot of that you cannot measure, but we have a good level of confidence that we have a billion databases in the open. Then I think, okay, that's success, right?

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4040.436

A lot of the read stuff works. A lot of the read stuff works. And one of the things that Pekka, and this was entirely Pekka's idea, that he did very well, and he was very praised by a lot of people, including those contributors. Some of them said, this is one of the reasons I contributed, is that he wrote on day one, because technically, this was very well thought.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4062.355

We thought about everything very well on the technical level, just on the presentation that we didn't have a lot of time. But there is a compatibility matrix. So if you go to the repository, there is a file there, compat.md, that is linked in the readme. And there is a full compatibility matrix with everything.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4079.806

So what you're going to see there is like the read stuff from SQLite already works pretty well. So you can, in a lot of ways, if your queries are not the one I run today with 10 joins and very complicated use cases, the basics of like reading from a SQLite file, they work already.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

410.188

For all intents and purposes, it's the same difference between the BSD license and the MIT license. It's minor differences. Nobody cares. I don't care. And I think nobody else does either. But SQLite is, according to their own website, by the way, this is something that they put in big bold letters in their website, open source. And again, I'm not claiming that they're wrong to say that.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4128.05

And if you want to self-host... More or less. Just to add to that, it wasn't something very relevant, so I didn't bring it up, but we already had, for other reasons, we already had a proof of concept of a new server implementation that is not based on libSQL. And the reason we did this is that we also want our server to have deterministic simulation testing.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4150.29

So that new server will be closed source. LibSQL, so that's one of the changes in strategy that we're making. And again, we understand that in a perfect world, everything will be open source. I would love that. But the reason we're doing this is that we want to have this very clear separation without concerns of what goes where.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4168.797

And we think the best way to do this is if the server is fully proprietary. And the client is fully open source. So Leap SQL is open source. The protocols are all the same. So you would be able to run your Thurso Cloud databases on Leap SQL. That's the goal.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4184.866

But the new server that runs deterministic simulation testing, that has a lot of additional features, that is designed to be multi-tenant, the design goal that we had is that we will be able to run queries to a billion SQLite databases in a single box. So all of that, like that level of scalability, we're going to keep it closed source. So that's going to be the dividing line.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4206

And then LibSQL still exists as a reference implementation. If you truly want to run an open source version, I think it's more accurate to say if you want to run an open source implementation of our cloud platform, LibSQL will do this for you with limitations. But our new server,

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4227.352

We decided not to, you know, seeing the success of Limbo, we decided not to release that as open source to keep things clear. And then Turso, the embedded database is 100%, 200% if we can, open source.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4252.557

Yes, or write their own implementation. Or use Leap SQL again, which is still going to be there, but we believe our implementation is going to be better. That's why we didn't.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4274.423

No, we have runway for more than that. Everybody needs money.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4280.788

We don't need to raise more money now. Yeah, that's right. For more than two years. I mean, I've been talking about 15 months, not as a cutoff point for that. It was more about like, when do we believe that this can become, but we have capital for a lot longer than that.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4307.305

Send me an invitation today for a year in the future.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

431.791

It's all good. Open source, but not open contribution. So when people think of an open source project, there are two things that comes to your mind. And the first one is like the code is open and SQLite is that. But you also think about, you know, instinctively, well, you know, the code is going to be on GitHub somehow. And then if I find an issue, I can go contribute to this project.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4311.948

Same time, same time, same date. How about that? Yeah, exactly. Let's do it.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4481.059

Yeah, he wrote a blog about his situation. I actually mentioned that on X. The Primogen, which is this big streamer that I'm sure a lot of your audience knows, read his article on stream based on my tweet. So there's a lot. If you want to see a long account and hear from him, I do recommend you look into his blog. I'm happy to give you the link here.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4501.775

But long story short, as far as I understand, I mean, it's this guy that was imprisoned for nonviolent drug offenses. So he's in prison somewhere in Maine as part of a pilot program. He was allowed to have a remote job. And for his remote job, he has access to the internet monitor and with restrictions, of course. But when Limbo was announced, he took a tremendous interest

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4527.345

And I actually spoke to him. I called him the other day. And the story that he told me is that it just so happened. We announced Limbo in December, right, December 2025. So he was on a break. So he was on vacation from his work. And when you and I go on vacation, you know, maybe you go travel somewhere or I go play with my kids, et cetera.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

453.693

And I have some source of ownership to the code. It's not just they can see the code. They can also modify the code. So that's usually classically one of the tenets of open source. Now, SQLite is not like that. The maintainers of SQLite, the creators of SQLite do not take contributions. This is not something that we are saying. This is something that they're saying. And

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

4548.474

Preston's vacation is just like I sit here in prison for 12 hours. Change love plus plus.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

477.356

Yes, I mean, I think it happened in the past that people managed to contribute to SQLite in very special occasions. But it's not a project that is designed to take contributions from other people. So that is how SQLite is. We started running into a couple of. you know, hurdles with SQLite. So we wanted... Part of that is that we wanted to deploy our SQLite databases to the edge.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

503.368

We wanted to deploy it in a way that was replicated. We wanted to do things like read replicas, and we wanted to put it to run on Cloudflare Workers and all things like that. And we considered this a good thing for our project. So, you know, if... And... Technically, we knew we could do it. And for context here, like Pekka and I, we've been working together. Pekka is my co-founder.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

523.91

We've been working together for 15 years now. We met each other when we were both working on the Linux kernel. Pekka was one of the maintainers. We work independently. Each one of us worked in Linux for almost 10 years, the first five without knowing each other. I think we knew off each other, but we never met.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

540.224

And in the last five years, very closely to each other, where we developed a good friendship. After that, we actually joined a startup that also became a database company. We spent eight years doing that. So we felt like technically, SQLite is all things considered pretty simple, given the scope of things that we used to do. So technically, we can make the changes that we believe we can.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

561.309

And so let's just come up with our own version. And we saw at the time a lot of people dancing around the subject. That's what we saw. So there were some projects like Light at Fast that were super interesting projects, Dequal Light. We saw lots of people trying to get SQLite to work as a distributed system.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

580.146

The problem that we saw with all of those projects is that they were all, again, dancing around the subject that SQLite cannot be changed. So we're going to create layers and layers and layers around SQLite to help with that. And for us, it was like, technically, this is actually a pretty simple problem. You just have to change SQLite here, here, and here, and you're good to go.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

602.178

And I think the last piece of the puzzle, or maybe the two last pieces of the puzzle, number one is that we were very concerned with who's going to want to run our SQLite plus patches, because SQLite is a very trusted project. So you make random modifications to SQLite, everybody starts distrusting what you're doing. So we knew that, look,

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

627.574

The solution to this problem is that we should just create an open contribution version of SQLite because now it's not like SQLite plus Fing. It's its own project. And then people come to trust that there's one project. And with that, the diagnosis that we had at the time is like, hey, SQLite is great code-wise.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

647.887

The thing that is holding it back in our reading is exactly the fact that it cannot take advantage of the dynamism of the creativity of a truly open source community. So let's create that. And the last piece of the puzzle is that at the time we were actually discussing how do we do this? And one of the options was like, maybe we should just rewrite SQLite.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

672.097

And we had a lot of experience in our previous company, Scylla. Scylla was a re-implementation of Apache Cassandra and C++, fully compatible. And maybe we over-indexed in the fears that we had going that route. We say, hey, the problem with Scylla is that it took a long time for us to put in the market. SQLite is much smaller, so maybe we can do it. But

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

692.419

maybe because we had just come from an experience where we wrote a database, we figured that forking was a better strategy.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

700.821

At the time, our decision after much deliberation was, we're not going to rewrite SQLite, we are going to fork it instead because the main advantage is that you have something tomorrow because you start from a code base that is already working, and then we're going to start making changes to that. So that was Leap Sequel. Leap Sequel was essentially at the time.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

722.629

And the story of Limbo, if I had to tell the sanitized version in which the fork never existed, it would be the same up until this point. Because all of those things, all of that is still valid. It's just that at the time, a year and a half ago, we decided the forking was the best alternative. And

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

741.667

Now, and I'm happy to go into the why, but now we decided to try, okay, what if we had done that, one of those options that we consider, what if we just rewrote it? And then when we put it out there, the results were, you know, I've never seen anything like that before. You know, so just that's the story.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

817.054

It has trillions of databases deployed, right?

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

853.723

This was one of the big contributors for us to revisit that decision was exactly the test suits. And again, it's not that we didn't know about it. It's just that we thought things would go one way. But look, I always love to, if I may, to use opportunities like this to clarify something because...

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

871.697

When you're telling a story, when you're saying something, it's easy for people to misunderstand you. So I always want to make this 100% clear. There's nothing wrong with SQLite and the way they manage their community, right? So sometimes people assume that because we went this direction, we believe that what they're doing is wrong. And I want to clarify that that's not the case.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

892.781

I think the beauty of having different people with different points of view coming from different backgrounds and doing things differently is exactly that you can experiment with a lot of models. And those models have advantages and disadvantages, and it's fine. So we that's not the way we we believe we truly believe. And the goal we have with Limbo now is really to replace SQLite.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

915.921

So we believe that we can build something much better if we're able to tap into the creativity and the dynamism of a modern open source community. We believe that, which does not mean that what they're doing is wrong. And it's just the choice that they made. And we make a different choice. It's OK, right?

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

942.364

Exactly.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

964.993

Exactly. And we spent a lot of, we did spend a lot of time, by the way, crafting that message. Yeah. Regardless of how much, there's always some room for misunderstanding, which is why I always appreciate. Now, I also don't want to run from it. I want to make something clear. We disagree technically with that decision, right? It doesn't mean we think it's wrong.

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

984.961

There's nothing like we don't think badly of them. They clearly created something very successful. But the way Pekka and I always run open source communities and the way we always participated in open source communities, for us, this... this component of like being able to tap into the creativity of the community was always something very important.