Menu
Sign In Pricing Add Podcast
Podcast Image

The Changelog: Software Development, Open Source

Turso is rewriting SQLite in Rust (Interview)

Thu, 30 Jan 2025

Description

Glauber Costa, co-founder and CEO of Turso, joins us to discuss libSQL, Limbo, and how they're rewriting SQLite in Rust. We discuss their efforts with libSQL, the challenge of SQLite being in the public domain but not being open for contribution, their choice to rewrite everything with Limbo, how this all plays into the future of the Turso platform, how they test Limbo with Deterministic Simulation Testing (DST), and their plan to replace SQLite.

Audio
Transcription

6.522 - 27.719 Adam Stachowiak

What's up, welcome back. This is The Change Log. We feature the hackers, the leaders, and those replacing SQLite. Yes, today we're joined by Glaber Costa, co-founder and CEO of Terso. Terso is an open source distributed database powered by LibSQL. which is described as the open contribution fork of SQLite.

0
💬 0

28.059 - 50.146 Adam Stachowiak

We discussed their efforts with LibSQL, the challenge of SQLite being in the public domain and not having an opportunity for contribution, their choice to rewrite everything with Limbo, how this all plays into the future of the Terzo platform, how they test Limbo with deterministic simulation testing, also known as DST, and their plan to replace SQLite.

0
💬 0

50.746 - 72.546 Adam Stachowiak

A massive thank you to our friends and our partners over at fly.io. Fly is the public cloud built for developers who ship. That's you. That's me. That's us. Over 3 million apps have launched on fly and you can too. Deploy your app in five minutes. Learn more at fly.io. Okay, let's see what it takes to replace SQLite.

0
💬 0

83.51 - 98.828 Adam Stachowiak

Well, friends, before the show, I'm here with my good friend, David Shu, over at Retool. Now, David, I've known about Retool for a very long time. You've been working with us for many, many years. And speaking of many, many years, Brex is one of your oldest customers. You've been in business almost seven years.

0
💬 0

98.988 - 109.738 Adam Stachowiak

I think they've been a customer of yours for almost all those seven years, to my knowledge. But share the story. What do you do for Brex? How does Brex leverage Retool? And why have they stayed with you all these years?

0
💬 0

110.179 - 129.214 David Shu

So what's really interesting about Brex is that they are an extremely operational heavy company. And so for them, the quality of the internal tools is so important because you can imagine they have to deal with fraud, they have to deal with underwriting, they have to deal with so many problems basically. They have a giant team internally basically just using internal tools day in and day out.

0
💬 0

129.234 - 143.362 David Shu

And so they have a very high bar for internal tools. And when they first started, we were in the same YC batch, actually. We were both at Winter 17. And they were, yeah, I think maybe customer number five or something like that for us. I think DoorDash was a little bit before them, but they were pretty early.

0
💬 0

143.843 - 162.016 David Shu

And the problem they had was they had so many internal tools they needed to go and build, but not enough time or engineers to go build all of them. And even if they did have the timer engineers, they wanted their engineers focused on building external facing software because that is what would drive the business forward. Brex mobile app, for example, is awesome.

0
💬 0

162.216 - 179.183 David Shu

The Brex website, for example, is awesome. The Brex expense flow, all really, you know, really great external facing software. So they wanted their engineers focused on that as opposed to building internal CRUD UIs. And so that's why they came to us. And it was awesome. Honestly, a wonderful partnership. It has been for seven, eight years now.

0
💬 0

179.423 - 197.707 David Shu

Today, I think Brex has probably around a thousand Retool apps they use in production. I want to say every week, which is awesome. And their whole business effectively runs now on Retool. And we are so, so privileged to be a part of their journey. And to me, I think what's really cool about all this is that we've managed to allow them to move so fast.

0
💬 0

198.187 - 214.86 David Shu

So whether it's launching new product lines, whether it's responding to customers faster, whatever it is, if they need an app for that, they can get an app for it in a day, which is a lot better than, you know, in six months or a year, for example, having to schlep through spreadsheets, etc. So I'm really, really proud of our partnership with Brex.

0
💬 0

215.74 - 239.543 Adam Stachowiak

Okay, Retool is the best way to build, maintain, and deploy internal software, seamlessly connect to databases, build with elegant components, and customize with code, accelerate mundane tasks, and free up time for the work that really matters for you and your team. Learn more at retool.com. Start for free. Book a demo. Again, retool.com.

0
💬 0

272.099 - 293.314 Unknown

So what would happen if you took SQLite, the most widely deployed database in the entire world, and you rewrote it in Rust with some new twists and new ideas? We are all going to find out because Globber, Costa, and his team at Terso are working on it. It's called Limbo. Globber, welcome to the changelog.

0
💬 0

293.714 - 295.135 Glauber Costa

Thank you. It's great to be here.

0
💬 0

295.691 - 301.153 Unknown

It's great to have Adam here, despite flu season hitting hard in the Stachowiak house. Oh, man.

0
💬 0

301.274 - 305.876 Glauber Costa

It sounds great, though, Adam. You sound great. Thank you. You mentioned you're on the tail end of the flu, but you don't sound like it.

0
💬 0

305.896 - 308.837 Adam Stachowiak

I've been trying so hard to sound better, so thank you.

0
💬 0

309.497 - 325.821 Unknown

Where should we start? I think this story goes back quite a ways. I remember... Lib SQL or Lib SQL, which was a couple of years ago now, your guys' effort to create a different open source SQLite. I thought SQLite was open source and public domain. Can you tell that story?

0
💬 0

326.221 - 343.731 Glauber Costa

Oh, happy to do that. And the story, in a sense, it's still the same story. So I think you hit the nail on the head there. It's just really how it started. I don't feel this is a completely different thing, what we're doing now. The story there is that we were using SQLite in our company. So Pekka and I, Pekka is my co-founder.

0
💬 0

343.751 - 367.676 Glauber Costa

We founded a company for, if you happen to stumble upon it, just mentioning here by name called ChiselStrike. And it was in the API space. We were doing a bunch of things and we were using SQLite very heavily in that company. And the reasons we were using SQLite were reasons I think that a lot of your audience will understand and relate to.

0
💬 0

368.537 - 388.175 Glauber Costa

Look, you would just do npm install, the name of the package, and then everything's there. Your database is there. Everything is there. Everything works. You don't have to install anything. So SQLite for us always had this appeal of a database that is just always working, and then you can build stuff around it, and it's just always there. So there was always something that was very appealing to us.

0
💬 0

389.133 - 410.148 Glauber Costa

However, we also stumble upon this thing a little bit about SQLite not being open source. So let me address that. SQLite is a public domain software, which is technically a difference that doesn't matter. I think only a lawyer would be concerned with what is the difference between public domain and open source.

0
💬 0

410.188 - 431.711 Glauber Costa

For all intents and purposes, it's the same difference between the BSD license and the MIT license. It's minor differences. Nobody cares. I don't care. And I think nobody else does either. But SQLite is, according to their own website, by the way, this is something that they put in big bold letters in their website, open source. And again, I'm not claiming that they're wrong to say that.

0
💬 0

431.791 - 453.592 Glauber Costa

It's all good. Open source, but not open contribution. So when people think of an open source project, there are two things that comes to your mind. And the first one is like the code is open and SQLite is that. But you also think about, you know, instinctively, well, you know, the code is going to be on GitHub somehow. And then if I find an issue, I can go contribute to this project.

0
💬 0

453.693 - 476.435 Glauber Costa

And I have some source of ownership to the code. It's not just they can see the code. They can also modify the code. So that's usually classically one of the tenets of open source. Now, SQLite is not like that. The maintainers of SQLite, the creators of SQLite do not take contributions. This is not something that we are saying. This is something that they're saying. And

0
💬 0

477.356 - 502.707 Glauber Costa

Yes, I mean, I think it happened in the past that people managed to contribute to SQLite in very special occasions. But it's not a project that is designed to take contributions from other people. So that is how SQLite is. We started running into a couple of. you know, hurdles with SQLite. So we wanted... Part of that is that we wanted to deploy our SQLite databases to the edge.

0
💬 0

503.368 - 523.87 Glauber Costa

We wanted to deploy it in a way that was replicated. We wanted to do things like read replicas, and we wanted to put it to run on Cloudflare Workers and all things like that. And we considered this a good thing for our project. So, you know, if... And... Technically, we knew we could do it. And for context here, like Pekka and I, we've been working together. Pekka is my co-founder.

0
💬 0

523.91 - 540.164 Glauber Costa

We've been working together for 15 years now. We met each other when we were both working on the Linux kernel. Pekka was one of the maintainers. We work independently. Each one of us worked in Linux for almost 10 years, the first five without knowing each other. I think we knew off each other, but we never met.

0
💬 0

540.224 - 560.549 Glauber Costa

And in the last five years, very closely to each other, where we developed a good friendship. After that, we actually joined a startup that also became a database company. We spent eight years doing that. So we felt like technically, SQLite is all things considered pretty simple, given the scope of things that we used to do. So technically, we can make the changes that we believe we can.

0
💬 0

561.309 - 578.885 Glauber Costa

And so let's just come up with our own version. And we saw at the time a lot of people dancing around the subject. That's what we saw. So there were some projects like Light at Fast that were super interesting projects, Dequal Light. We saw lots of people trying to get SQLite to work as a distributed system.

0
💬 0

580.146 - 600.617 Glauber Costa

The problem that we saw with all of those projects is that they were all, again, dancing around the subject that SQLite cannot be changed. So we're going to create layers and layers and layers around SQLite to help with that. And for us, it was like, technically, this is actually a pretty simple problem. You just have to change SQLite here, here, and here, and you're good to go.

0
💬 0

602.178 - 625.77 Glauber Costa

And I think the last piece of the puzzle, or maybe the two last pieces of the puzzle, number one is that we were very concerned with who's going to want to run our SQLite plus patches, because SQLite is a very trusted project. So you make random modifications to SQLite, everybody starts distrusting what you're doing. So we knew that, look,

0
💬 0

627.574 - 646.93 Glauber Costa

The solution to this problem is that we should just create an open contribution version of SQLite because now it's not like SQLite plus Fing. It's its own project. And then people come to trust that there's one project. And with that, the diagnosis that we had at the time is like, hey, SQLite is great code-wise.

0
💬 0

647.887 - 670.896 Glauber Costa

The thing that is holding it back in our reading is exactly the fact that it cannot take advantage of the dynamism of the creativity of a truly open source community. So let's create that. And the last piece of the puzzle is that at the time we were actually discussing how do we do this? And one of the options was like, maybe we should just rewrite SQLite.

0
💬 0

672.097 - 691.739 Glauber Costa

And we had a lot of experience in our previous company, Scylla. Scylla was a re-implementation of Apache Cassandra and C++, fully compatible. And maybe we over-indexed in the fears that we had going that route. We say, hey, the problem with Scylla is that it took a long time for us to put in the market. SQLite is much smaller, so maybe we can do it. But

0
💬 0

692.419 - 700.121 Glauber Costa

maybe because we had just come from an experience where we wrote a database, we figured that forking was a better strategy.

0
💬 0

700.821 - 721.948 Glauber Costa

At the time, our decision after much deliberation was, we're not going to rewrite SQLite, we are going to fork it instead because the main advantage is that you have something tomorrow because you start from a code base that is already working, and then we're going to start making changes to that. So that was Leap Sequel. Leap Sequel was essentially at the time.

0
💬 0

722.629 - 741.487 Glauber Costa

And the story of Limbo, if I had to tell the sanitized version in which the fork never existed, it would be the same up until this point. Because all of those things, all of that is still valid. It's just that at the time, a year and a half ago, we decided the forking was the best alternative. And

0
💬 0

741.667 - 760.158 Glauber Costa

Now, and I'm happy to go into the why, but now we decided to try, okay, what if we had done that, one of those options that we consider, what if we just rewrote it? And then when we put it out there, the results were, you know, I've never seen anything like that before. You know, so just that's the story.

0
💬 0

760.698 - 781.179 Adam Stachowiak

I think the public domain-ness of SQLite is unique. I think the not open to contribution is also quite unique. Very valid in the, in quotes, open source world. And even... they say on this public domain page they have, where they say warranty of title.

0
💬 0

781.199 - 801.931 Adam Stachowiak

So if you are a company that needed something where you needed to be identified against the copyright infringement or anything happening inside this code base, it gets to be a little bit hairy trying to use SQLite in unique ways beyond. So it kind of makes sense, some of the challenges you had, and maybe the fork made the most sense, but ultimately the rewrite seems to make sense.

0
💬 0

802.593 - 817.034 Adam Stachowiak

the most sense because what you want is the attributes of what SQL offers the world but not the And I don't want to say it like a negative thing because it's great, but it's dominated very well in the marketplace. I mean, it's used on Mars, I believe, right?

0
💬 0

817.054 - 820.195 Glauber Costa

It has trillions of databases deployed, right?

0
💬 0

820.355 - 844.5 Adam Stachowiak

Yeah, I mean, like it's not... And Dr. Richard Hipp is one of our friends. We love him. We think he's an amazing person. But I think you are right, though, that the public domain-ness and the not open to contribution makes it challenging because that... And Dr. Richard Hipp even said on this podcast in the past that the test suite is proprietary in the fact that it's not in the open.

0
💬 0

844.54 - 853.423 Adam Stachowiak

You can't see it. So making these large scale changes. So to build on top of what is SQLite, if that's your desire, becomes just problematic.

0
💬 0

853.723 - 870.737 Glauber Costa

This was one of the big contributors for us to revisit that decision was exactly the test suits. And again, it's not that we didn't know about it. It's just that we thought things would go one way. But look, I always love to, if I may, to use opportunities like this to clarify something because...

0
💬 0

871.697 - 892.741 Glauber Costa

When you're telling a story, when you're saying something, it's easy for people to misunderstand you. So I always want to make this 100% clear. There's nothing wrong with SQLite and the way they manage their community, right? So sometimes people assume that because we went this direction, we believe that what they're doing is wrong. And I want to clarify that that's not the case.

0
💬 0

892.781 - 915.901 Glauber Costa

I think the beauty of having different people with different points of view coming from different backgrounds and doing things differently is exactly that you can experiment with a lot of models. And those models have advantages and disadvantages, and it's fine. So we that's not the way we we believe we truly believe. And the goal we have with Limbo now is really to replace SQLite.

0
💬 0

915.921 - 930.011 Glauber Costa

So we believe that we can build something much better if we're able to tap into the creativity and the dynamism of a modern open source community. We believe that, which does not mean that what they're doing is wrong. And it's just the choice that they made. And we make a different choice. It's OK, right?

0
💬 0

931.24 - 941.964 Unknown

Well, I'm glad you make that explicit because as an observer, I remember your initial announcement. And maybe it's because of maybe the social baggage of what a fork implies.

0
💬 0

942.364 - 942.905 Glauber Costa

Exactly.

0
💬 0

943.325 - 956.53 Unknown

My initial impression was, and I don't remember the words that you use anymore. It was two years ago. But I was like, you and your team seemed a little bit... upset or, you know, you're kind of disappointed with this. And so we're going to fork and do it a different way.

0
💬 0

957.17 - 964.933 Unknown

And it's hard to do that without ruffling feathers or without getting, even if you don't mean to, people think that you're angry or mad or whatever, you know?

0
💬 0

964.993 - 984.361 Glauber Costa

Exactly. And we spent a lot of, we did spend a lot of time, by the way, crafting that message. Yeah. Regardless of how much, there's always some room for misunderstanding, which is why I always appreciate. Now, I also don't want to run from it. I want to make something clear. We disagree technically with that decision, right? It doesn't mean we think it's wrong.

0
💬 0

984.961 - 1002.95 Glauber Costa

There's nothing like we don't think badly of them. They clearly created something very successful. But the way Pekka and I always run open source communities and the way we always participated in open source communities, for us, this... this component of like being able to tap into the creativity of the community was always something very important.

0
💬 0

1003.35 - 1019.921 Glauber Costa

And you have to put it in context as well, that Pekka and I grew up in the Linux kernel community, right? And a lot of people say this about SQLite, so SQLite has to, and with that, I disagree, SQLite being what it is, has to be kept as this very small thing, because that's the whole beauty of SQLite.

0
💬 0

1020.501 - 1037.283 Glauber Costa

And again, I disagree with that because if you look at Linux, Linux started as this project that would only run on x86 computers and et cetera. And today it is an operating system that runs everywhere. It runs very well on the server, you know,

0
💬 0

1037.523 - 1056.602 Glauber Costa

Pekka and I worked together in memory management for very big box machines, like how to make sure that when you have two terabytes of memory in 2010, which was a lot of memory back then, it's still a lot of memory today, but like those algorithms to manage memory don't take three minutes and hang up the system. It's a problem that doesn't happen in embedded devices.

0
💬 0

1057.362 - 1078.779 Glauber Costa

And Linux still runs on embedded devices. So through this big community, you actually can create something that runs well everywhere and don't necessarily have to make those compromises. So we don't agree that SQLite has to be protected and has to be maintained by just a single individual to be the success it is. But look, who knows? Maybe I'm wrong. Maybe I'm right.

0
💬 0

1079.139 - 1088.689 Glauber Costa

We act based on what we believe. But we don't believe there is anything wrong with what they're doing. We just think that it could benefit with a different direction.

0
💬 0

1089.149 - 1094.155 Adam Stachowiak

But you do believe that you can replace. You said, we believe we can replace SQLite.

0
💬 0

1094.175 - 1117.972 Glauber Costa

100%, and we will. And I want to clarify that now, because with LeapSQL, it became very clear to us in maybe a month that that was not going to happen. So with LibSQL, I joked about it today. We had this goal of replacing SQLite for 10 minutes because it's a dream, et cetera. It's obviously a big prize. It's obviously this big prize and et cetera.

0
💬 0

1118.573 - 1138.448 Glauber Costa

As I wrote in my blog post, were we bummed about it? No, because we built a pretty successful business on top of LibSQL, which is today our company, Turso. So for us, it was it was this exercise of like if you heard the expression shoot for the stars, because even if you miss, you're going to hit the moon.

0
💬 0

1139.229 - 1152.537 Glauber Costa

For us, it was like, yeah, you know, it turns out I don't think it's going to be possible for us to replace the light with the fork. But that's us hitting for the stars. We didn't hit the stars. We hit the moon. The moon's great. Again, we built a super great project. We're having fun.

0
💬 0

1153.077 - 1173.974 Glauber Costa

Leap SQL brought something to SQLite that wasn't there, which was like the serverless, running this on serverless environments on the cloud platform, syncing databases. One of the things that we do with Torso is that you can sync your databases between different devices and backups and all of that. And this is going great. I mean, this is going fantastically for us. And so we were never...

0
💬 0

1176.076 - 1201.592 Glauber Costa

sad or disappointed that our, you know, that this thing that crossed our mind, imagine how amazing it will be if, you know, people just start using our fork instead of SQLite. We're never disappointed at that. But when we put the rewrite out, it became very clear. And I've been using this analogy from chemistry that the reaction was right. Like LibSQL was something that the market was saying,

0
💬 0

1202.453 - 1226.14 Glauber Costa

clamoring for. I think what it became very clear seeing the reaction of Limbo now is that it's just that the activation energy is not enough to make the reaction happen. And after talking to a lot of people, and I want to tell, there's a whole story of what happened after we announced this. I talked to a lot of the people who came, started contributing to the project,

0
💬 0

1227.321 - 1250.942 Glauber Costa

And what it became clear is that they all wanted, like, they all agree with us that, like, we need a better version of SQLite. I mean, it would be fantastic if you have something like SQLite, but there's this, all our original vision. But the fork just wasn't differentiated enough. It just wasn't, you know, ambitious enough. It just wasn't bold enough to get people to come and contribute.

0
💬 0

1251.622 - 1265.949 Glauber Costa

Furthermore, it is actually very hard to do because one of the things you did mention, like the SQLite's test suit is fully proprietary. So once you start making changes that are very deep into the core, you start hitting a bunch of issues.

0
💬 0

1266.009 - 1280.832 Glauber Costa

And some of them is just like yourself saying, I know that the best way to solve this problem is this, but I'm going to solve it in this other way, which is almost as good, but not great. but it's less risky. So you start having this on your mind.

0
💬 0

1280.872 - 1293.383 Glauber Costa

So the fork never really unshackles you fully to go pursue this, which is why when we announced the rewrite, what we saw, like, again, was something I've never seen before. We got 8,000 GitHub stars in a week, essentially.

0
💬 0

1293.784 - 1316.96 Glauber Costa

We had for the first, what I consider to be for the first time in the history of computer science, a Hacker News comment section that was mostly positive with, you know, just a... I think maybe one or two comments only that were trying to denigrate us, but the rest very supportive. We had this fantastic, inspiring story of this individual who is in prison at the moment.

0
💬 0

1317.3 - 1331.608 Glauber Costa

And he was one of the first people in the United States to be granted access to the internet in prison. He can access Discord, but not X, by the way. Maybe say something about it. But he is now the fourth top contributor to the Limbo project.

0
💬 0

1331.628 - 1331.988 Unknown

Amazing.

0
💬 0

1332.188 - 1349.927 Glauber Costa

And many, many other things. I mean, the level of contributions that we started to get, we started getting people running this on the browser. A week after we announced, it was running on the browser already, just like this compiled to WebAssembly. all from contributions from third-party people.

0
💬 0

1350.347 - 1372.358 Glauber Costa

Again, Leap SQL, we did get a lot of contributors, but they were all contributing to the drivers, to the server, you know, things at the margin, which they're important as well for an open-source project. But with Limbo, we actually managed to see our dream, like the 10-minute dream come true. Like, we're going to build a community of people coming here and writing this database with us, right?

0
💬 0

1372.578 - 1382.581 Glauber Costa

So it... We revisit it. Now it's no longer a 10-minute dream that went away. We truly believe now that once we get the momentum going, we can replace SQLite. And that's the goal.

0
💬 0

1382.881 - 1397.324 Unknown

Well, I think you hit a great chord, as you've given evidence to. I know my guttural response to Limbo was quite a bit more positive than mine was to LibSQL, just by reading the announcement post. I'm like, okay, this makes a lot of sense. And then I realized later, oh, it's the same people that tried LibSQL.

0
💬 0

1397.344 - 1397.884 Unknown

It's the same people, yeah.

0
💬 0

1398.524 - 1415.694 Unknown

a couple of years back. And it seems like with a rewrite, there's just so much more to do. There's, there's, it has its own identity. There's new, there's fresh and new ideas. And whereas a fork almost comes from a different place. And so I think you're, you're absolutely describing it on the testing front though.

0
💬 0

1415.734 - 1426.74 Unknown

I would imagine it's gotta be just as hard, or maybe I would expect it to be harder with a rewrite. Why, what makes it easier with a rewrite than with a fork? If you still don't have a test suite.

0
💬 0

1427.345 - 1442.793 Glauber Costa

Your intuition is correct, by the way, which is one of the things that all things considered, we went with the fork. Because at least there is some level of guardrails that come with the fact that at least you know that this code that you're importing was tested by this proprietary tester.

0
💬 0

1442.873 - 1443.773 Unknown

It started off right.

0
💬 0

1446.015 - 1468.808 Glauber Costa

I don't think your intuition is incorrect, but something amazing happened. And by the way, the way I look into the situation, another thing that I want to make it clear, I don't actually think that we made a mistake with the fork because if we were rewriting back then, there were many decisions that we made today that we would not have made But now we did because we had the experience.

0
💬 0

1468.828 - 1488.53 Glauber Costa

We've been running the Thurso platform and we learn a lot of things. So it actually allows... I almost see it as a prototype and it actually allows us to make tremendously good decisions with the Limbo project. And one of them is exactly the testing. What happened is that... Becca and I got hooked into something called deterministic simulation testing.

0
💬 0

1488.75 - 1510.664 Glauber Costa

And deterministic simulation testing is a very niche thing that most people have never heard about. I had heard about it a couple of times, but I didn't truly quite understand what it was until I met Yoran. So Yoran is the founder of of, and I can say this, I mean, even though I am the founder of a database company, Yoran is the founder of the most amazing database company I've ever seen.

0
💬 0

1510.824 - 1530.43 Glauber Costa

Although now I think with Limbo, we have a chance to reclaim that title. Yoran and I are going to fight about it. But, you know, ask me a month ago, I would say, man, I have to give it to him. This is the most impressive database company of all, Tiger Beetle. Tiger Beetle is a database, just to keep it very brief, that is designed for financial transactions. So it's completely written in Zig.

0
💬 0

1531.331 - 1555.834 Glauber Costa

And their goal is to replace entire systems, the bespoke systems that most financial institutions have to process transactions. So if you're selling to banks and financial institutions, if your goal is to replace the backbone of world's commerce, It's pretty hard to do, right? And one of the challenges that you have is just this, like, how do we trust this thing?

0
💬 0

1556.775 - 1581.133 Glauber Costa

And you're unfound the solution for that. The solution that you're unfound was to write Tiger Beetle entirely with a technique called deterministic simulation testing. Deterministic simulation testing is essentially a fuzzer. So it will generate a bunch of inputs. It randomly generates as many inputs as it can on the space of possible inputs. And this is the disadvantage.

0
💬 0

1581.153 - 1604.741 Glauber Costa

You kind of have to write software in a special way to lend itself to deterministic simulation testing. It's very hard to boat it to an existing code base, right? But you write it in a way that every single operation that you do on IO, on thread scheduling, everything that happens goes through a special interface that abstracts all IO, and then you plug a simulator.

0
💬 0

1605.802 - 1627.999 Glauber Costa

When you're testing this, you plug this into a simulator, and the simulator is included in the code base. So what the simulator does is that it's going to explore the space of possibilities, create the most arcane, impossible situations ever, And then when something breaks, it gives you the exact steps deterministically that everything in the system had up until that point.

0
💬 0

1628.039 - 1648.951 Glauber Costa

So debugging those problems become very easy. So in record time, Tiger Beetle managed to create a system based on deterministic simulation testing, which is their database. And the stories that Yoram would tell is like, look, we found this bug that would only happen if you would call fsync on a disk, and then fsync would return an incorrect result.

0
💬 0

1649.331 - 1669.781 Glauber Costa

And at the same time, a packet would come from the network. I mean, he would describe the most complicated scenarios. Oh, and the end result of that, the simulator gives you a seed, and you type that seed into the simulator. Now you have every single step that happens to make that happen, right? So when we... This is one of the things that led us to believe, what if we try the rewrite?

0
💬 0

1669.901 - 1679.555 Glauber Costa

Because we knew that we could now try the rewrite using deterministic simulation testing. And then we also partnered with a company called Lantitis that offers like a...

0
💬 0

1680.817 - 1702.48 Glauber Costa

almost like the integration test analogy for deterministic simulation testing, which is a full system version of that, that will simulate things between machines, it will simulate the network, it will simulate hardware failures and stuff like that. So whatever bugs our simulator does not catch, Usually we just, you give it to antithesis the next day, antithesis catches the bug.

0
💬 0

1703.2 - 1721.69 Glauber Costa

So we knew that with that, that was the missing part of the puzzle. And then with that, we will be able to actually create something that probably even surpasses the level of testing the SQLite has. But here's the catch. It is easier to do this on the rewrite because you have to write the system with this in mind from the get-go. It's not something that is easy to bolt upon.

0
💬 0

1722.39 - 1731.998 Unknown

Is that popular enough that you can go out and there's, you know, rust crates that will give you this functionality? You have to write all this DST stuff yourself?

0
💬 0

1732.338 - 1752.613 Glauber Costa

All of it. Yeah. So for people who are interested in DST, but don't want to go through the pains of just rewriting all sorts of things like that, I truly recommend taking a look at antithesis. Because, I mean, antithesis is amazing. It's the second... is the second next best thing. We use it in conjunction, it's not an either or.

0
💬 0

1752.653 - 1776.141 Glauber Costa

The analogy that I have is that our own DST is like unit tests, like we can run centuries of possibilities in two days. And it's very fast, all things considered. And Antithesis is like integration tests. So you wanna have both and we do have both. And Antithesis has been a great partner for us. But the problem with deterministic simulation testing

0
💬 0

1776.781 - 1799.668 Glauber Costa

is that once you start importing other crates in Rust, for example, you have no idea what those crates are doing. Those crates are calling IO. So you kind of have to write everything. So it's not even that you have a crate for deterministic simulation testing. You might not even want to import something super simple 100%, but you don't want to import anything that does IO, that has a timer.

0
💬 0

1800.529 - 1819.04 Glauber Costa

You want to write those things. So it's something very hard to do for a general purpose testing for a database like Tiger Beetle, for a database like Limbo. You know, it's worth it. And the scope is quite limited. Right. And you don't want to be importing crates all the time anyway. You know, Tiger Beetle is way more insane than we are.

0
💬 0

1819.06 - 1834.77 Glauber Costa

I mean, they have a policy that they just don't have dependencies. That's it. They write every single piece of code that we try to be a little bit more flexible. But we will not import a crate anymore. And again, it also depends because SQLite comes with a CLI.

0
💬 0

1835.41 - 1835.63 Unknown

Right.

0
💬 0

1836.111 - 1850.72 Glauber Costa

If you want to import a crate that does whatever crazy stuff to implement callers in the CLI, that's fine. But for the core of the database, we try not to import anything that could potentially do I.O. because we want to make sure that everything goes through the simulator.

0
💬 0

1859.076 - 1876.105 Adam Stachowiak

Well, friends, you can now build invincible application thanks to Temporal, today's sponsor. You can manage failures, network outages, flaky endpoints, long-running processes, and so much more, ensuring your workflows and your applications never fail. Temporal allows you to build business logic, not plumbing.

0
💬 0

1876.345 - 1890.351 Adam Stachowiak

They deliver durable execution and abstracts away the complexity of building scalable distributed systems and lets you focus on what matters, delivering reliable systems that are faster. An example of this is Masari. They are the Bloomberg for crypto.

0
💬 0

1890.772 - 1906.558 Adam Stachowiak

They provide market intelligence products to help investors navigate digital assets, and they recently turned to Temporal to help them improve the reliability of their data ingestion pipeline. This pipeline collects massive amounts of data from various sources, and then they enrich it with AI.

0
💬 0

1907.438 - 1926.95 Adam Stachowiak

This process previously relied heavily on cron jobs and background jobs and cues and the design worked well. However, these jobs were difficult to debug at scale because they needed more controls and more observability. And as they look to rethink this ingestion flow, they wanted to avoid cron jobs, background jobs, cues.

0
💬 0

1927.43 - 1967.496 Adam Stachowiak

They didn't want to create a custom orchestration system to oversee and to ensure these jobs and work was being done reliably. Here's a quote. End quote. So if you're ready to build invincible applications and you're ready to learn why companies like Netflix, DoorDash and Stripe trust Temporal as their secure and scalable way to build and innovate, go to Temporal.io. Once again, Temporal.io.

0
💬 0

1967.776 - 1981.452 Adam Stachowiak

You can try their cloud for free or get started with open source. Once again, Temporal.io. This DST sounds like magic. Like actual magic.

0
💬 0

1982.053 - 1982.354 Glauber Costa

It does.

0
💬 0

1982.414 - 1999.158 Adam Stachowiak

Yes, man. If it's like writing unit tests, what exactly... And you said that the rewrite is better because you have to write, assuming this DST is in place, what actually is the writing and the creating of this deterministic simulation testing? What is that? How do you do it?

0
💬 0

1999.499 - 2020.576 Glauber Costa

Yeah, so again, the simulator itself, you just write a couple of scenarios. And again, it's not truly magic because you do have to write the simulator, right? So the simulator just writes, imagine, for example, a workload, and the workload is like generate a couple of queries and then write, etc. But the simulator will include a fuzzing element to that.

0
💬 0

2020.636 - 2039.268 Glauber Costa

So we will, instead of like generate this query that I wrote in the unit test, you give it almost like a query generator and then it starts generating random queries. Then you start in the simulator itself. What happens, imagine this, if you want to write to a file in any software, you would call a operating system API, write.

0
💬 0

2040.388 - 2062.09 Glauber Costa

If we're talking about Rust, it's like a FS write, and then you write to the operating system. When you're writing software for DST, you don't do this. You have your own IO interface abstraction, and then all of your IO goes through this. So when you're running this in production mode, your abstraction for write just called the operating system write.

0
💬 0

2062.651 - 2083.864 Glauber Costa

But when you're running in deterministic simulation testing mode, your abstraction for write runs the simulator code that will start injecting failures into this. And again, injecting failures in a deterministic way, because then if the query fails, because you injected a IO failure at that moment, you will be able to replay that session piece by piece.

0
💬 0

2084.918 - 2092.643 Adam Stachowiak

As my friend that runs, well, he used to be on Screen Rant. Wow, wow, wow. I'll just leave it there. I'm sure.

0
💬 0

2092.743 - 2101.808 Glauber Costa

Is that the guy that, is that the guy that. It's barely inconvenient. Very easy, barely inconvenient. That's right. Super easy. I love that guy, man. Wow, wow, wow. Super easy, barely inconvenient. Wow, wow, wow.

0
💬 0

2101.928 - 2104.31 Adam Stachowiak

That's what I'll say right now. That's how he respond.

0
💬 0

2104.83 - 2116.934 Glauber Costa

So how do you rewrite SQLite with the confidence that it will actually have the level of trust that SQLite acquired? It's super easy, barely an inconvenience, right? All you have to do is write the determinants.

0
💬 0

2117.374 - 2138.902 Unknown

I love it. Just throw a DST at it. So when you talk about SQLite compatible, that's Limbo's goal. There's a lot of different fronts that that has to be on. Are you talking file? structure are you talking syntax language are there performance compatibility there's all kinds of things that sqlite is what's limbo's goals with regard to these different areas of compatibility

0
💬 0

2139.295 - 2156.263 Glauber Costa

First of all, compatibility in a project like this, and this is the experience that we gained at Scylla when we were rewriting Apache Cassandra. Compatibility has to be like a one-way street, right? You don't want to shackle yourself and say, I will always... Because sometimes, for example, to implement a new feature, you have to do it in a different way.

0
💬 0

2157.304 - 2184.33 Glauber Costa

So it's usually something, hey, we're going to offer you the same feature set as SQLite. We're going to read your SQLite files. We're going to execute your SQLite code. And if you're not using any of the specific features that we have, we can generate SQLite files as well. But the moment you start using new features that are only present in your implementation, it's impossible to be compatible.

0
💬 0

2184.39 - 2206.22 Glauber Costa

It's just not possible. So that's it. It's limbo so far, and this is something that we want to keep. The language is the same. Again, we want to keep the language. We want to keep the ABI. We want to be able to load SQLite extensions. The file format, obviously, it's the thing that defines SQLite. So obviously, again, we might add to it in the future, but we'll be reading SQLite files.

0
💬 0

2207.751 - 2229.014 Glauber Costa

Normally, and we are bytecode compatible as well. So one of the other ways in which we test Limbo is by now out of the simulator, just generating random SQL statements, and then seeing that the bytecode that is generated by Limbo is the same bytecode set of instructions that is generated by SQLite.

0
💬 0

2229.734 - 2241.839 Glauber Costa

So that doesn't catch bugs in the implementation of the bytecode, but at least you know that the query plan is the same and et cetera. So this is yet another way in which we've been testing to make sure that it's up to standards.

0
💬 0

2242.319 - 2254.004 Unknown

So I mentioned at the top that it also has some ambitious goals, one of those fully asynchronous IO, which is quite a bit different than SQLite, right? So how do you accomplish that but maintain this compatibility?

0
💬 0

2254.568 - 2277.934 Glauber Costa

Yeah, so async is something that a lot of people misunderstand. Async doesn't mean, for the rust-minded people in the audience, to be clear, async does not mean that we're going to use an async runtime like Tokyo or like others. I actually personally wrote an async runtime when I was a data dog that's still around called Glomio. We don't use any of that.

0
💬 0

2278.334 - 2297.465 Glauber Costa

And again, part of the reason we don't use any of that is that we want all of that to go through our simulator. So if you look at the limbo code, it's just really sync rust. It's not async rust, right? So all the async means is that when you call an operation, if that operation is not ready, it will return to you instead of blocking.

0
💬 0

2298.105 - 2319.833 Glauber Costa

And the main, now I'm getting a little bit too technical, but let me just give you the full context. The SQLite C API has a bunch of functions, but the most important of them is called SQLite step. And SQLite step is essentially take another step, take another step in processing this query for me. And this is a blocking function.

0
💬 0

2320.233 - 2343.476 Glauber Costa

So what that means is that if you call SQLite step, it will block until it resolves. let's say in that step you want to execute 4, 5, 10, however many bytecode instructions, until it does that, it will block. In limbo, if you detect that you're not ready to execute those bytecode instructions at that time, you just return something saying no. Call me back later.

0
💬 0

2343.596 - 2364.792 Glauber Costa

So that is essentially what async means. And then with that, it becomes very easy to plug something like Tokyo on top in Rust or run it on the browser or whatever, because you can call those async functions. And if it's not ready, it will not block. That's all that means. But it doesn't mean that we have to use the async runtime internally. In fact, quite the opposite way.

0
💬 0

2365.27 - 2373.436 Unknown

So what does that unlock for you? Because SQLite historically is sync, but super fast because it's right there in process and stuff.

0
💬 0

2373.997 - 2398.175 Glauber Costa

Well, it isn't that fast. And I ran a query yesterday, for example, that took 10 minutes to run. There was a query with five joints looking at all the user data about users who are signed up to the Turso platform. it takes a long time just because the query is complicated. SQLite is very fast for CRUD style, just like, hey, here's the key, give me the value. Sure.

0
💬 0

2399.556 - 2423.353 Glauber Costa

But it's not super fast for analytical stuff. The first thing that it unlocks is just this, queries that are much more complex that you can run in dashboards and things like that. Also, for the serverless environments, Durso being one example, and for clarification, our product will be renamed to Durso Cloud. And we have all the intention in the world to rename Limbo to just Durso.

0
💬 0

2423.954 - 2442.966 Glauber Costa

Limbo is just not a great name, just a parenthesis here. And it just, you know, the story, it's all in the blog post, but we never expected this level of support. So we just came up with a made up name for it. That means nothing, right? But now we'll address that. But our product, the Durso Cloud, allows you to do like serverless SQLite on the cloud.

0
💬 0

2443.726 - 2465.29 Glauber Costa

You have a bunch of HTTP requests there in the middle. It allows you, for example, to host your data on S3 because if you have your data hosted on S3, your query now is not necessarily super fast because you may be hitting a page that is not local, right? So it allows you, for example, to run SQLite with partial storage, like with most of your data on S3 and some of the data locally.

0
💬 0

2465.35 - 2484.9 Glauber Costa

It allows us to run on the browser because the browser, as you know, is a very... sync unfriendly environment. Like if you block, you block everything. The page just doesn't load. So all you have in the browser, you have to be async. So there is a lot of things that it unlocks in terms. In fact, the one thing that Limbo doesn't even support transactions.

0
💬 0

2485.88 - 2501.928 Glauber Costa

There is, you know, it's very, very early, which for a lot of people who came to contribute, they actually saw as an advantage, you know, just because you come with this energy and there's so much to do. But we're not even yet at the point there is support transaction. And the one thing that is already different and is resonating a lot is the fact that it is async.

0
💬 0

2502.309 - 2517.222 Adam Stachowiak

Well, you put a lot of thought behind this, you know, not expecting it to be so well received. You put so much, it seems like just so much thought in the design of it from the DST, you know, to async to all the things, but... Why were you so surprised it was so well received?

0
💬 0

2518.262 - 2535.862 Glauber Costa

Technically, we put a lot of thought on the product. And the full story there is that, first of all, we consider, just revisiting, especially if people are joining now, we consider rewriting SQLite from the get-go anyway. So this was always on our mind. This was always on our mind. And... We added to Leap SQL.

0
💬 0

2535.902 - 2557.01 Glauber Costa

So one of the things, if you download Leap SQL today in the fork, Leap SQL comes with vector search out of the box. You don't have to install any extension. You can do your rag pipelines of SQLite out of the box with vector search. It was very, very hard to do. It was a ordeal to actually get vector search working. And there are lots of things in the syntax

0
💬 0

2557.99 - 2582.536 Glauber Costa

earlier when i was saying you might do it one way but then you end up doing this other way which is more conservative because you don't vector search was the thing that i had in mind like many things in the syntax they're okay but they're not great because we had to be a little bit more conservative and then pekka started thinking and this is all on pekka he started thinking okay how would that look like how would that look like if we were to rewrite it and

0
💬 0

2584.469 - 2612.956 Glauber Costa

Our goal was something like, hey, look, if this project keeps going and it goes well and it's an experimental thing, maybe in two years, in three years, we can make something out of it. So we were very thoughtful about the technical decisions that we were making. But everything around the presentation was just like, yeah, whatever, man, whatever. So, again, this was not on our company's GitHub.

0
💬 0

2612.996 - 2633.148 Glauber Costa

This was on Pekka's personal GitHub. He spent 10 seconds thinking about the name and then just wrote on and said, it's going to be Limbo because it's a state of confusion. Like, we don't know what we're going to make of this. Like, the whole story of the name Limbo was essentially Pekka. What? What are we going to make with this? I have no idea. I just want to experiment with this concept.

0
💬 0

2633.528 - 2655.405 Glauber Costa

The logo, he just asked Chat2PT to generate. We took another five seconds. So there was not a lot of thought on the presentation and publishing, but there was a lot of thought about the technical side because we thought that maybe in two years, maybe in three years… There is a play for us once we are a much more established company to tackle rewriting C. coli.

0
💬 0

2655.866 - 2682.726 Glauber Costa

What we do not expect is that what we decided to do is just make it an official torso experiment. But if you read the posts in which we were announcing this, we were very clear. We don't have a roadmap for this. Don't ask us what our intentions are. There is nothing that we want to do with this one on the short term. This was an experiment on Pekka's GitHub that actually did pretty well.

0
💬 0

2683.326 - 2705.247 Glauber Costa

It did pretty well based on two metrics. We got 1,000 GitHub stars and we never talked about it. We never talked about it. And we got two incredibly good engineers that started, one from Red Hat, who is now with us. I mean, so we hired those two engineers. We got two incredibly good engineers that started contributing to the project. Again, it was just a project on Pekka's GitHub. Mm-hmm.

0
💬 0

2705.387 - 2726.925 Glauber Costa

So we knew that we knew there was something there and we decided, let's just now publish it as a official torso experiment. We wrote in the blog post, maybe if this goes, if that goes well in six months, we can start pouring some of our resources and interest and then we're going to see what happened. So we had. We had some idea that this will be useful at some point.

0
💬 0

2726.965 - 2740.278 Glauber Costa

We just did not know it will be so well received to the point that it was to the point that we changed our entire company strategy now to be able to pour our whole weight of our resources behind the project. Right. And that's what we're going to do.

0
💬 0

2740.739 - 2741.46 Unknown

So you're all in now.

0
💬 0

2742.02 - 2760.617 Glauber Costa

We're all in. In fact, we wrote a blog post last week telling the story, a lot of what we're discussing here. Hey, I mean, this is – look, in my wildest dreams, in my wildest dreams, I would expect maybe this to gain like 2,000 more stars in a month or so than from 1,000 goes to 3,000.

0
💬 0

2761.378 - 2779.786 Glauber Costa

Maybe a couple of other engineers that would come and contribute as well and start slowly, but we would see some potential on it. That was my definition of success. And every single metric that we thought Limbo could be successful at, we saw three times more, four times more than what we anticipated. So we decided to go all in.

0
💬 0

2779.806 - 2798.411 Glauber Costa

And there is a blog post that we wrote recently with all the changes that we're going to make to the platform to allow us to do this. With all the changes that we're doing to the company, we had a lot of reorganizations internally. And this is really something that we decided in a couple of weeks in January. Because we just like, how can we ignore this?

0
💬 0

2798.451 - 2809.037 Glauber Costa

I mean, it sounds, it seems very clear to us now that the world at large really wants a evolution of SQLite. Like the signals are very strong. So I think we just need to get behind it, right?

0
💬 0

2810.018 - 2824.566 Adam Stachowiak

What do you think made those two contributors that were so pivotal contribute? If this was an experiment on Pekka's personal GitHub, it wasn't like from a technical level well thought out, but just an experiment. What was compelling to them?

0
💬 0

2825.146 - 2842.278 Glauber Costa

Yes. And again, those two individuals in particular, they are now working for us. And we hired them already last year because, you know, they're great. And it's back and I was like, man, if Limbo has no other value, it's a great hiring tool for a company, right? Because it's attracting this kind of people. But those are people, of course, that we have direct access to.

0
💬 0

2842.298 - 2870.253 Glauber Costa

So we talked to them extensively and all the others that came as well after. We had 32 contributors to Limbo the day we announced it as a torso experiment. As you know, Days later, we had already 60 and we're reaching 70 now. And again, a lot of that very core contributors doing a lot of great work. So Perry and Yusei, those two contributors are the first ones that we talked to.

0
💬 0

2870.773 - 2892.654 Glauber Costa

And the story is always the same. The story is always the same. And it's exactly the thing I said. They were very excited about the prospect of a better SQLite. They knew about LibSQL, but LibSQL never enticed them because it just wasn't... In UC's word, it just wasn't crazy enough, which I translate to bold. But it just wasn't...

0
💬 0

2893.955 - 2913.272 Glauber Costa

it kind of caps where you can go because of all, you know, you still shackle by the fork. So when they saw this thing that, and I think you add this sprinkle to that as well too, man, there's so, this is a new project. So there's so many things to do, which engineers get attracted to, to some extent as well. But if you talk to them again, it's always the same story.

0
💬 0

2913.292 - 2923.798 Glauber Costa

I love the vision and I already loved the vision when I heard of Leap Sequel, but this thing here, is just the right amount of ambitious that I want to start devoting my time to that and I want to be a part of that as well.

0
💬 0

2923.838 - 2937.522 Glauber Costa

So that's exactly, and we wanted to, one of the things that we said a lot in the early Leapsicle days is that we said that a lot because we heard from a couple of people, a couple of companies even, we love the idea of what you're doing because it gives us a seat at the table.

0
💬 0

2938.402 - 2958.448 Glauber Costa

And then my interpretation of that is that it turns out it's actually pretty hard to give people a seat at the table if you don't own the table. So Limbo gave us this. I mean, it's our table now. Now, it's a table that is modeled after another table. But we own the table and now we can really truly give those people a seat at the table.

0
💬 0

2958.468 - 2973.643 Glauber Costa

I mean, they're coming, they're writing code, they're contributing, they're reviewing code, they're helping with the direction, they come out with ideas, things that we never consider, things that we never consider, things that will not be a priority. Like the thing, SQL already limbo runs on the browser. Somebody showed up today, today.

0
💬 0

2974.124 - 2990.281 Glauber Costa

Oh, I got LibSQL to compile, run the limbo CLI in WebAssembly and now I'm distributing the CLI. It's not even something we would have done, but the person took an interest because it doesn't have a ceiling. You essentially can take this anywhere.

0
💬 0

2990.922 - 2993.785 Unknown

So does LibSQL just go away then or be replaced outright?

0
💬 0

2994.548 - 3018.869 Glauber Costa

Leap SQL came to be a little bit, and a part of that because, again, as I said, after we had this dream of replacing SQLite for 10 minutes, and then it became clear that it wouldn't happen. So we had this something that was very valuable. We knew it was valuable, but we had to struggle a little bit more with what do we do with that. So the Leap SQL project today is, again, it's two things.

0
💬 0

3019.429 - 3041.178 Glauber Costa

It's a fork of SQLite. with some of those changes like vector search, but it is also, and it's all in the same repository. It's all part of the same project. It is also a server implementation to do serverless SQLite over the wire. So that is what, that is what lib SQL is. This will, it's, it's an open source project. So we're still going to be maintaining that, but the, the,

0
💬 0

3042.219 - 3064.176 Glauber Costa

The client side part is going to be full Limbo. We intend to eventually get all of those things we've done, like vector search, all of that. Leap SQL is still maintained for the time being because, again, we have a business around it. We have features that we depend on, but our goal is to eventually port all of those features to Limbo, which, again, we plan to rename to Turso.

0
💬 0

3064.897 - 3078.664 Glauber Costa

And then the client is that, and Leap SQL can become just the server implementation as an open source alternative to do that.

0
💬 0

3080.525 - 3099.171 Adam Stachowiak

Okay, friends, I'm with a good friend of mine, Avtar Swithin from Timescale. They're positioning Postgres for everything from IoT, sensors, AI, dev tools, crypto, and finance apps. So Avtar, help me understand why Timescale feels Postgres is most well positioned to be the database for AI applications.

0
💬 0

3099.691 - 3115.094 Avtar Swithin

It's the most popular database, according to the Stack Overflow Developer Survey. And Postgres, one of the distinguishing characteristics is that it's extensible. And so you can extend it for use cases beyond just relational and transactional data for use cases like time series and analytics.

0
💬 0

3115.214 - 3128.096 Avtar Swithin

That's kind of where Timescale the company started, as well as now more recently, vector search and vector storage, which are super impactful for applications like RAG, recommendation systems, and even AI agents, which we're seeing, you know, more and more of those things today.

0
💬 0

3128.416 - 3141.285 Adam Stachowiak

Yeah, Postgres is super powerful. It's well-loved by developers. I feel like more devs, because they know it, it can enable more developers to become AI developers, AI engineers, and build AI apps.

0
💬 0

3141.685 - 3158.716 Avtar Swithin

From our side, we think Postgres is really the no-brainer choice. You don't have to manage a different database. You don't have to deal with data synchronization and data isolation because you have like three different systems and three different sources of truth. And one area where we've done work in is around the performance and scalability.

0
💬 0

3158.816 - 3170.101 Avtar Swithin

So we've built an extension called PG Vector Scale that enhances the performance and scalability of Postgres so that you can use it with confidence for large scale AI applications like RAG and agents and such.

0
💬 0

3170.341 - 3189.591 Avtar Swithin

And then also another area is coming back to something that you said, enabling more and more developers to make the jump into building AI applications and become AI engineers using the expertise that they already have. And so that's where we built the PGAI extension that brings LLMs to Postgres to enable things like LLM reasoning on your Postgres data, as well as embedding creation.

0
💬 0

3189.811 - 3195.575 Avtar Swithin

And for all those reasons, I think when you're building an AI application, you don't have to use something new. You can just use Postgres.

0
💬 0

3196.075 - 3216.299 Adam Stachowiak

Well, friends, learn how Timescale is making Postgres powerful. Over 3 million Timescale databases power IoT, sensors, AI, dev tools, crypto, and finance applications, and they do it all on Postgres. Timescale uses Postgres for everything, and now you can too. Learn more at timescale.com. Again, timescale.com.

0
💬 0

3220.62 - 3226.965 Unknown

Is there trailblazing left to do? Or have you done the trailblazing and now it's just a matter of work?

0
💬 0

3227.485 - 3248.553 Glauber Costa

There's so much. There's so much. Because one of the things that we heard as we launched Leap Sequel, for some reason I'll never forget that individual. So we create this Discord community. We announced Leap Sequel. People came and one person in particular, but others said the same thing as well. Give me better schema changes and I'm switching to this tomorrow.

0
💬 0

3249.642 - 3272.613 Glauber Costa

So one of the areas, for example, the SQLite is really bad at is schema changes. You can make schema changes, but you cannot alter the type of a column. There's all sorts of limitations like that, which is something, by the way, that we made better in LibSQL. So LibSQL does better schema changes than SQLite, but it's not better enough. Same story, because we were limited in what we can do.

0
💬 0

3273.073 - 3291.983 Glauber Costa

So we want to run... replication, like native replication to the browser, which is something like people have been asking us for a long time. Imagine you have this SQLite database running on your browser that can then sync with an external server or whatever and just get pages on demand. That's one of the things we want to do. We want to tackle the problem of schema changes.

0
💬 0

3292.003 - 3312.077 Glauber Costa

We want to tackle the problem of write throughput because SQLite is a very bad database for write operations. We want to make SQLite much better for analytical workloads. So there's just so much. that we believe we can do, right? But together with all of that, there is also the boring work of, you know, we don't even support transactions yet.

0
💬 0

3312.658 - 3318.984 Glauber Costa

And again, some of that you can do in parallel, like a sync, like browser, but some of them is just time in the saddle, right?

0
💬 0

3319.383 - 3322.126 Unknown

How much time do you think? Are we talking months, years?

0
💬 0

3322.987 - 3338.864 Glauber Costa

Nine months to a year. Nine months to a year. For two reasons. First of all, because of the deterministic simulation testing. Because it allows you to move with a lot more confidence. Imagine trying to get into the thought process of a database writer.

0
💬 0

3339.705 - 3356.655 Glauber Costa

It's always like I want to make this change, but I don't fully understand the impact that this change will have in all of those environments that I don't control, like running on an embedded device here, running this or so making changes to systems like that again. And those are the kind of systems that we dealt with at Linux.

0
💬 0

3357.535 - 3378.511 Glauber Costa

sometimes Linux has a lot of weird stuff in the process of development around the idea of like, we know that this thing can break on a processor that only three people have in the world and it's not supported anymore. And we don't want to break those things. So we always move very, very, very, very carefully. And the deterministic simulation testing just allows it to make a lot faster. Like,

0
💬 0

3378.891 - 3402.277 Glauber Costa

paired with antithesis, which is our integration testing in this analogy, it just allows you to move so much faster. The second reason for why we believe like a year is the reasonable timeframe here is a SQLite, as it turns out, that's why one of the reasons we believe And forking and rewriting is not the biggest code base in the world. SQLite is not that complex.

0
💬 0

3402.678 - 3422.458 Glauber Costa

If we were rewriting Postgres, it's a completely different story. But rewriting SQLite, it's actually doable in a year. And also, as I said, we're not going to do this in Q1 because we're treating our first quarter of the year as a transition quarter. to allow us to finish other work so that we can be free to do this.

0
💬 0

3423.018 - 3442.67 Glauber Costa

But we're going to have at least seven, eight engineers just working on it full time. So imagine that you have seven people working on a code base that is less than 200,000 lines big for nine months with a deterministic simulator that catches all possible bugs that you can muster. We believe it's a very reasonable timeline, like nine months to a year.

0
💬 0

3443.328 - 3449.012 Unknown

So next January, we'll be talking about, is it production grade? What, 1.0? What does it look like?

0
💬 0

3449.052 - 3462.122 Glauber Costa

Yeah. We want to, we want to release a 1.0 much earlier than that, much earlier than that, because we want to, you know, we'll see, like we still believing in, in like release, release often. Just, just put it out there.

0
💬 0

3462.142 - 3463.003 Unknown

Get it out there. Yeah.

0
💬 0

3463.443 - 3479.131 Glauber Costa

A lot of people are going to be early adopters. A lot of people, I think, like our thesis that, you know, it doesn't even support even without transactions. There's a lot of people that with a little bit more support on the read side, they can already use it because a lot of workloads for SQLite, like you get the file, like you don't write to the file.

0
💬 0

3479.151 - 3494.96 Glauber Costa

You get the SQLite file from somewhere and then you just run a bunch of stuff on it. So even without writes being fully supported, we believe there's a lot of use cases that this unlocks. So we want to be very aggressive with like making releases. But the moment we're going to say, hey, like this is stable.

0
💬 0

3495.981 - 3515.875 Glauber Costa

And look, we can actually even give us more time because it's a great commitment to say something is stable. I mean, you know, people trust you that this is going to work. So we can take even more time than that to say, hey, this now has our seal of approval. We're going to support all of that. But much earlier than that, it can already be production ready. Yeah.

0
💬 0

3516.789 - 3525.075 Unknown

How does this all affect Terso, the business, and how does it fit in? I assume it's going to be an MIT-licensed thing, open source.

0
💬 0

3525.436 - 3545.046 Glauber Costa

So our business is going to change. We announced recently a lot of what we need to do is that we need to simplify our product a lot. So we had a very... Very hard decisions to make. So again, it was a tough time for us to go through those decisions. But we knew we kept the mission in front of us.

0
💬 0

3545.386 - 3569.5 Glauber Costa

We kept this idea like we need to make those changes because I think the community now trusts us to rewrite SQLite, which is like the biggest prize we believe. So we have to make those changes. A couple of features, a couple of features that we have. Some of them that I love. A lot of users, unfortunately, got quite upset with that, obviously. I mean, the features that they use and came to trust.

0
💬 0

3570.301 - 3592.091 Glauber Costa

We will discontinue. So the way we're doing this is that if you're a paid user, after a certain cutout date, you're going to be allowed to keep using those features. But new users, new signups, anybody else who is not a paid user at that point will no longer have access to those features. With that, we believe our platform will become a lot simpler. We're still going to maintain our platform.

0
💬 0

3592.271 - 3611.062 Glauber Costa

But again, we think that having a single person just running the platform will be enough. And we're not going to be investing in having new features into the platform. So the platform will essentially go, which is the Turso Cloud, will essentially be a place where you can still run. Like if what we do today is good for you, great.

0
💬 0

3611.782 - 3628.487 Glauber Costa

And what we do today is two things very well, especially after those features will be discontinued. You can access SQLite over the wire from serverless environments. So you have a serverless managed SQLite database. And we have our features, point-in-time restore for backups, branching. We have all of that that a serverless database needs.

0
💬 0

3629.247 - 3651.798 Glauber Costa

And you have syncing of databases between devices, servers, et cetera. So you can start with your SQLite file and you can upload that file to Torso and then replicate that to other SQLite files that you own. That's the platform. We're not going to be investing too much in new features, in front-end features, in quality of life features. The platform will essentially go into a freeze for a year.

0
💬 0

3652.499 - 3670.591 Glauber Costa

And the platform is pretty good. It's at a position that we can afford to do this. And then for one year, we're going to go all in in getting Limbo to replace SQLite, right? And that's it. And then at that point, Limbo then becomes the thing that we run on the platform. And we have the money. We have the running to do this. I mean, we It's all accounted for.

0
💬 0

3671.032 - 3681.301 Glauber Costa

So if we managed to, which I believe it's very doable, to get limbo in a production-ready state in 15 months, our plan still works. But 9 to 12 is what we're thinking here.

0
💬 0

3682.008 - 3690.918 Adam Stachowiak

When you get to that place, whenever Limba becomes Terso and Terso Cloud goes away, how will you differentiate this new launch?

0
💬 0

3690.938 - 3716.445 Glauber Costa

Terso Cloud doesn't go away. What is Terso today becomes, it's called Terso Cloud. And then Terso, the embedded database just becomes Terso. And... Again, this is a lesson that we learn. Not that we haven't heard this before, but I think we're just stubborn. People kept telling us it's very hard to create two brands. It's very hard to create one brand. Two brands is even harder.

0
💬 0

3717.186 - 3739.001 Glauber Costa

But the reason we kept those things separate is And the reason LibSQL wasn't called Durso was exactly because we wanted to give people the sense of, you know, this is a welcoming community. We have our business, but your interests will be heard, right? Like Linux. We model a lot of that on the experiences we had at Linux. Very few people care about this.

0
💬 0

3739.341 - 3751.431 Glauber Costa

And that's the lesson that we learned, right? Just almost nobody. Of all the reasons that we heard for people not contributing to Leap Seek, this was never something that showed up. Never, ever, ever, ever.

0
💬 0

3752.392 - 3768.982 Glauber Costa

And then when we announced our changes to the Thurso platform, we had many questions from many people about, hey, this feature, they're going to discontinue this other thing they're not going to support. And what happens is... Nobody asked about the name. Nobody said, well, with the name Turso, nobody cares.

0
💬 0

3769.642 - 3789.36 Glauber Costa

So this idea that we had that we have to keep things very separate to create the welcoming community, we learned that nobody cares. So we're going to take the opportunity here just to fix it. Because again, it is true what we heard from many of our advisors. Creating a brand is already incredibly hard. Creating two brands is very...

0
💬 0

3789.5 - 3811.093 Glauber Costa

very hard and the people in the know, they know, but people who are just hearing about it, now you have to explain that this is this and this is that. We're going to just consolidate the names. Turso today, again, is the Cloud offering that is going to be renamed to Turso Cloud. Then what is Limbo, which is the client offering, will just be renamed to Turso and that's it.

0
💬 0

3811.113 - 3817.017 Glauber Costa

We're still welcoming, we still want people to come and build this with us. We just learned that the name doesn't matter.

0
💬 0

3817.717 - 3834.612 Unknown

How do you manage that relationship between the open source Terso, the interests of Terso Cloud, third-party contributors who may want Terso to go a different direction that maybe you don't want it to? Success brings all kinds of challenges.

0
💬 0

3835.252 - 3857.804 Glauber Costa

It does. And that's why what we decided to do is never port the server code to Limbo. Because then the server can be kept completely separate. And again, I think the reason we had the server and the client together was exactly because it was clear to us after a month or so that we would not, with LibSQL, with the fork,

0
💬 0

3858.544 - 3879.31 Glauber Costa

have achieved our goal of really just replacing SQLite, this goal that we had briefly. So the best strategy then became have the server code here as well, because now there are other things you can do with this open source project. But with the success of Limbo, again, it's back on the table and it's a very realistic goal that this will replace SQLite.

0
💬 0

3879.73 - 3899.069 Glauber Costa

So we just revisited the strategy and said, this has to be just the client-side library exactly because of this, because we don't want to be in a position. And the name was a part of that. We just learned that it doesn't matter, right? But we don't want to ever be in a position. We want to design things in a way that we are never, ever, ever in a position to even think about

0
💬 0

3899.75 - 3917.817 Glauber Costa

We would like to, you know, there's a contributor coming with this code and maybe we don't want to merge it because, you know, this might affect our business. You never want to be in this position. In fact, we want to make people in the community maintainers as well. We want to we want to have people hopefully like Preston, the person that I mentioned is in prison and others.

0
💬 0

3917.837 - 3938.314 Glauber Costa

They're doing fantastic work. I mean, if they keep working for another six months. or a year, and you see that they're committed to the project, they should be maintainers as well. And they should have the ability to just merge code without talking to us. That's what we want to see. So the way we're going to structure things to make that happen is that our business now has to be just the cloud.

0
💬 0

3938.354 - 3959.564 Glauber Costa

And then all the code for the cloud become, that is a separate project. That's going to be its own thing. And then the client, SQLite, for example, they don't have encryption at rest. They don't have a bunch of things because they do try to build a lifestyle business around it, as far as I understand, by selling those things. They're very specific. We want all of this in the open source for us.

0
💬 0

3959.624 - 3966.166 Glauber Costa

We want all of this. We want encryption at rest. We want all sorts of extensions. We want everything that runs on the client to be 100% open source.

0
💬 0

3968.263 - 3981.356 Unknown

Very cool. So how will you know when Limbo slash Turso has made it, has arrived, has replaced SQLite? Will it be on Mars? Will it be on trillions of devices? Like when will you think it's here?

0
💬 0

3981.396 - 3994.528 Glauber Costa

I think, yeah, I think if this was a consumer business is when your mom calls you to say that she's using it without knowing that this is the software they work with. I don't think my mom's going to use Limbo or Turso or SQLite or whatever. So I can't use that.

0
💬 0

3994.548 - 4016.181 Glauber Costa

So the criteria for us is really when we can see somehow through some fuzzy metric that we've got, like we have a billion databases out there, right? So, you know, SQLite has a trillion databases. And I think that we don't think we're going to replace SQLite in a year, right? That's not the goal, right? But we think that in a year, we can get to this point where, hey, we got our first billion.

0
💬 0

4016.201 - 4035.011 Glauber Costa

This is clearly working. You do this for another two years, you do this for another three years, you get there. So the first milestone that we have in mind is that when we can, through some proxy, because a lot of that you cannot measure, but we have a good level of confidence that we have a billion databases in the open. Then I think, okay, that's success, right?

0
💬 0

4035.453 - 4039.939 Unknown

If people wanted to play with it today, how much is there? What could you play with, if anything?

0
💬 0

4040.436 - 4061.574 Glauber Costa

A lot of the read stuff works. A lot of the read stuff works. And one of the things that Pekka, and this was entirely Pekka's idea, that he did very well, and he was very praised by a lot of people, including those contributors. Some of them said, this is one of the reasons I contributed, is that he wrote on day one, because technically, this was very well thought.

0
💬 0

4062.355 - 4079.266 Glauber Costa

We thought about everything very well on the technical level, just on the presentation that we didn't have a lot of time. But there is a compatibility matrix. So if you go to the repository, there is a file there, compat.md, that is linked in the readme. And there is a full compatibility matrix with everything.

0
💬 0

4079.806 - 4095.975 Glauber Costa

So what you're going to see there is like the read stuff from SQLite already works pretty well. So you can, in a lot of ways, if your queries are not the one I run today with 10 joins and very complicated use cases, the basics of like reading from a SQLite file, they work already.

0
💬 0

4096.767 - 4099.449 Unknown

Adam, any other questions for Globber before we let him go?

0
💬 0

4099.929 - 4121.025 Adam Stachowiak

I'm just trying to understand this future, I suppose. And I'm trying to piece it together, and I don't know how to ask this question necessarily, but I'm reading your most recent roadmap where you mentioned changes to our server implementation. And sorry for my stuttering because I'm still drastically behind on my breaths because of getting over the flow. But client-side, open source.

0
💬 0

4121.825 - 4127.169 Adam Stachowiak

Server-side, closed source. And so LibSQL... is Terso Cloud.

0
💬 0

4128.05 - 4149.149 Glauber Costa

And if you want to self-host... More or less. Just to add to that, it wasn't something very relevant, so I didn't bring it up, but we already had, for other reasons, we already had a proof of concept of a new server implementation that is not based on libSQL. And the reason we did this is that we also want our server to have deterministic simulation testing.

0
💬 0

4149.89 - 4150.15 Adam Stachowiak

Okay.

0
💬 0

4150.29 - 4168.637 Glauber Costa

So that new server will be closed source. LibSQL, so that's one of the changes in strategy that we're making. And again, we understand that in a perfect world, everything will be open source. I would love that. But the reason we're doing this is that we want to have this very clear separation without concerns of what goes where.

0
💬 0

4168.797 - 4184.185 Glauber Costa

And we think the best way to do this is if the server is fully proprietary. And the client is fully open source. So Leap SQL is open source. The protocols are all the same. So you would be able to run your Thurso Cloud databases on Leap SQL. That's the goal.

0
💬 0

4184.866 - 4205.96 Glauber Costa

But the new server that runs deterministic simulation testing, that has a lot of additional features, that is designed to be multi-tenant, the design goal that we had is that we will be able to run queries to a billion SQLite databases in a single box. So all of that, like that level of scalability, we're going to keep it closed source. So that's going to be the dividing line.

0
💬 0

4206 - 4226.852 Glauber Costa

And then LibSQL still exists as a reference implementation. If you truly want to run an open source version, I think it's more accurate to say if you want to run an open source implementation of our cloud platform, LibSQL will do this for you with limitations. But our new server,

0
💬 0

4227.352 - 4241.236 Glauber Costa

We decided not to, you know, seeing the success of Limbo, we decided not to release that as open source to keep things clear. And then Turso, the embedded database is 100%, 200% if we can, open source.

0
💬 0

4241.256 - 4251.739 Adam Stachowiak

200%, gotcha. So in the future, when somebody runs Limbo, which will be renamed Turso, they can run that on their own. But if they want this massive scale multi-tenant world, they've got to host with you.

0
💬 0

4252.557 - 4261.449 Glauber Costa

Yes, or write their own implementation. Or use Leap SQL again, which is still going to be there, but we believe our implementation is going to be better. That's why we didn't.

0
💬 0

4262.13 - 4274.383 Adam Stachowiak

So you're long betting on hosting a lot of databases in the future. Yes, yes. Okay. Yes. You said you don't need money, right? So you have the runway for the next 15 months. You're getting dangerous.

0
💬 0

4274.423 - 4277.345 Glauber Costa

No, we have runway for more than that. Everybody needs money.

0
💬 0

4278.086 - 4280.568 Adam Stachowiak

No need to raise more now, though, to get to this goal.

0
💬 0

4280.788 - 4296.2 Glauber Costa

We don't need to raise more money now. Yeah, that's right. For more than two years. I mean, I've been talking about 15 months, not as a cutoff point for that. It was more about like, when do we believe that this can become, but we have capital for a lot longer than that.

0
💬 0

4296.658 - 4306.805 Adam Stachowiak

Okay, good. Let's talk in a year, I guess. I mean, that's all the questions I have for now, for the most part. Everything else is speculation, but there you go.

0
💬 0

4307.305 - 4310.607 Glauber Costa

Send me an invitation today for a year in the future.

0
💬 0

4310.627 - 4311.868 Adam Stachowiak

January 22nd, 2026.

0
💬 0

4311.948 - 4316.571 Glauber Costa

Same time, same time, same date. How about that? Yeah, exactly. Let's do it.

0
💬 0

4317.351 - 4320.433 Unknown

Thanks, Labar. Hey, we're rooting for you. This is really exciting stuff. Thank you. Yeah.

0
💬 0

4324.025 - 4349.028 Adam Stachowiak

Well, the future of SQLite or SQLite is not necessarily in jeopardy, but it is being primed for disruption. If Globber and his co-founder has it their way, they will unseat and they will replace SQLite. And Limbo is full steam ahead with lots of inertia, lots of momentum, lots of contributions. And there's lots to do for you, for me, for others. So you can dig in.

0
💬 0

4349.428 - 4374.3 Adam Stachowiak

It is open source and it's also open to contributions and is being rewritten, which means there's new things to do, new innovation to be had. And that's exciting. Make sure you go to turso.tech, T-U-R-S-O.tech to learn more about what they're doing and to learn more about Limbo and the future of this database and this platform. Okay, big thank you to our friends and our partners over at Fly.

0
💬 0

4374.34 - 4403.093 Adam Stachowiak

We love them. Fly, you're awesome. Fly.io, launch your app in five minutes like we do. It is the cloud for developers who ship. That's us. That's you. Fly.io. And to our friends over at Retool, retool.com, and our friends over at Temporal, temporal.io, and our friends over at Timescale, timescale.com. And to the beat freak in residence, Breakmaster Cylinder. Those beats are banging, banging.

0
💬 0

4403.673 - 4409.296 Adam Stachowiak

Okay, friends, it's on Friday. Stay tuned. It's awesome. The show's done. We'll see you on Friday.

0
💬 0

4472.571 - 4480.778 Unknown

Can you tell us more about this? Is it Preston Thorpe? Do you have more information on this fella contributing from prison that you can tell a little bit more of the story?

0
💬 0

4481.059 - 4501.335 Glauber Costa

Yeah, he wrote a blog about his situation. I actually mentioned that on X. The Primogen, which is this big streamer that I'm sure a lot of your audience knows, read his article on stream based on my tweet. So there's a lot. If you want to see a long account and hear from him, I do recommend you look into his blog. I'm happy to give you the link here.

0
💬 0

4501.775 - 4526.785 Glauber Costa

But long story short, as far as I understand, I mean, it's this guy that was imprisoned for nonviolent drug offenses. So he's in prison somewhere in Maine as part of a pilot program. He was allowed to have a remote job. And for his remote job, he has access to the internet monitor and with restrictions, of course. But when Limbo was announced, he took a tremendous interest

0
💬 0

4527.345 - 4548.394 Glauber Costa

And I actually spoke to him. I called him the other day. And the story that he told me is that it just so happened. We announced Limbo in December, right, December 2025. So he was on a break. So he was on vacation from his work. And when you and I go on vacation, you know, maybe you go travel somewhere or I go play with my kids, et cetera.

0
💬 0

4548.474 - 4553.817 Glauber Costa

Preston's vacation is just like I sit here in prison for 12 hours. Change love plus plus.

0
💬 0
Comments

There are no comments yet.

Please log in to write the first comment.