DHH talks with us about his journey back to Linux, Apple's control over the App Store, setting up servers on the web, and tries out Terminal coffee.Want to carry on the conversation? Join us in Discord. Or send us an email at [email protected] — A delightfully fresh take on email + calendar, from 37signalsHEY — Apple vs. HEY, antitrust, and monopolyFix Consumer ElectronicsMake your dayONCE — CampfireInstall WSLIntroducing OmakubOmakase Developer SetupVisual Studio Code LinuxTokyo NightHomeLazygit Terminal UIZen MaintenanceUncomplicatedFirewallGimpTopics:(00:00) - Are we recording? (00:30) - How high is DHH really? (02:27) - Thoughts on AI + generative AI (05:31) - Linux on the desktop (18:07) - DHH's journey into Linux (27:51) - DHH's Omakub Linux set up (35:00) - The hardware journey (41:37) - Zen and the Art of Motorcycle Maintenance and connecting your computer to the internet (53:07) - Setting up servers like pets vs cattle (01:14:36) - Are we just doomed to repeat this every generation? (01:19:19) - What would make DHH want to stop programming? (01:26:51) - Plugging Terminal.shop
Hello.
Oh, there we go. It's DHH. Are we recording?
Oh, we are recording.
So are you in that amazing office that we see pictures of all the time?
This is the office. This is the office. This is the other picture out the back of it.
What's the elevation?
You're above the clouds? Good question. I think, what did I look it up? 2,000 feet or something? It's not that high. It just looks high because, I mean, I like to take a picture of it when the fog rolls in and the fog looks like it's clouds.
Oh, it's fog.
It looks like a climb of like 7,000 feet or something. Yeah, I thought it was clouds. I didn't realize it was clouds. I mean, it is clouds, but I think not the kind of clouds most people would call clouds, I suppose, because it is sort of fog clouds. But yeah, no, it looks great. And we've just had like a streak of it for a whole week. I would walk into the office and it would look like that.
And I was like, hot diggity damn, I've lived here for seven years. It still doesn't get old.
nature is amazing like that it's funny because i saw some random account post it the other day with like some like meme caption and i'm like oh yeah no no it goes meme almost every time i mean part of it is i don't even think they know where it came from i think they just got random pictures no no a bunch of people are just like um i got a bunch of people saying like this is ai i was like it could be but like this one isn't
Yeah, Adam gets comments about that with his face. Like we get comments on our videos all the time where they're like, that brown guy looks real, but that other guy looks AI.
Yeah, it's usually when I'm getting really monotone that we had a whole TikTok where like six people said I was clearly a skinwalker, which I learned what that is. I don't know. I got to get more intonation.
I'm always like, is that a compliment?
It should be because AI people look perfect, you know, so I think you should take it as a compliment.
Yeah, I was going to say, I mean, most AI people I see now, all the latest models, I mean, they all look great. There's not a lot of ugly people, AI models being pumped out.
Yeah. Yeah.
I've not heard you really talk about the AI stuff, like the generative AI stuff.
Yeah, I mean, part of it is because I don't have any special insights there. I'm just a happy consumer. And not only am I a happy consumer, I'm even perhaps like a slightly... conservative consumer when it comes to that.
I mean, we haven't shoved AI into every crevice of the applications we make because I have yet to see a lot of useful examples of that being done in, say, productivity or an email. Most of the cases like, hey, summarize these boring emails, I find actually vaguely dystopian that we're going to have a bunch of AIs like writing each other because we just can't tell it straight.
Like I'm going to tell the AI to write some bullshit for me. Here's the five bullet points. Could you just send me the five bullet points? Like that's what I'm going to have my AI turn it back into anyway. And now we're playing a game of telephone through AI. It's probably going to get it wrong about 15% of the time at least. And that just seems a little silly to me.
And also it doesn't seem that useful. So, I mean, for me, the thing that blows my mind is, is a, it's ability to be a pair programmer, even though I almost want to chug it in the bin. When I see for the 500th time, someone goes like programmers are now already obsolete. I just hired my co-founder cursor AI. It's like, Jesus, have you tried to use it for anything real again? Yeah.
I'm impressed by what it does, at the same time, utterly annoyed by the hype cycle that comes with it. But maybe this is just how it goes with everything. And then I think it's the gen AI stuff that blows my mind. As you said, how is this person not real? The ones that just came out where they had a string of images of... women giving presentations at like TED Talks or something.
Yeah, yeah, yeah.
I zoomed in. I was like, what's the tell? I was like, I can't find the tell. I would have to spend – I think it was something like arm flap folds or something like that was supposed to be the tell. Now I'm like, oh, the flaps aren't exactly – I was like, wow, it's gotten good when we're down to flaps. We're not even counting fingers anymore. We're down to – oh, the angle of the –
Flap is a little off. So that's why I don't talk about it that much because I don't feel like I have any unique input to offer beyond perhaps just the temperance that comes with the fact of having seen more than a handful of hype cycles. Yeah. Self-driving cars, I think, was a great example of that, right?
I got quite excited about that in 2017 when Elon was like, at the end of the year, your Model S is going to be a taxi earning you money. It's actually financially irresponsible not to buy a Tesla right now. driving other people around. It's like, that didn't quite happen, did it? While at the same time, then, whatever, the FSD 12.5 is actually pretty impressive.
It's not fucking a money printer driving around doing the things. And this is now we're, whatever, eight years later. Very hard to predict when these things tip, but...
But I think the reason we wanted to have you on here was you've been talking a lot more about Linux on the desktop, which I've been a lifelong Linux on desktop person. It's been like 12 years, I think, so far. But yeah, we just want to hear about... I guess, what got you into that? Is this a new thing for you? What's the story behind all this?
It is a new thing for me to run Linux full-time on the desktop in anger as my primary operating system. And it came about also out of some anger. The final straw, the motivating sort of light to the fire here was Apple. And I've been... I was going to say a happy Apple fan. I think that's fair to say, actually, for 20 years, about 20 years, a little more than 20 years.
Not only was I a happy Apple fan, I was an Apple evangelist. And that came about in part by very much a replay of what's just happened. In the mid to late 90s, I was a very reluctant Windows user. I was actually kind of a pissed off Windows user because I was using Windows because it felt like that was the only option, but I didn't like it.
I didn't like anything about what Microsoft stood for in those days. I didn't like the leadership of Microsoft. I didn't like the... choke off their air supply, business tactics that they were applying to Netscape and the other competitors they had at the time.
I didn't like the overly smug, I want to punch you in the face look of Bill Gates when he was sitting for the deposition at the antitrust cases. I just, I didn't like any of it. And I thought like, I'm begrudgingly using this operating system because I simply feel like there is not a realistic choice. And Maybe there will be a bunch of Linux people. Well, Linux was already great in 95.
Yeah, it wasn't for me. I had friends using Linux back then, and what I saw didn't win me over. Let's just put it like that. And then 2001, I believe it is, rolls around. Apple rolls out the new Mac OS 5. OS 10, I think it was, what was it, Puma 10.1 or 2 or 3, I forget. Whichever version was out in 2001 was the one I hopped on.
And I hopped on to that because for the first time it felt like there was a credible alternative to Windows that I could use as the web developer and as a general computer user without being like this super nerd that was going to figure out Linux on the desktop. And I thought, you know what, isn't this the best of both worlds? I get to use like a BSD-based operating system.
That's kind of like Linux. It's the same ethos underneath the technical underpinnings. This is why I don't have to install an antivirus system within the first five seconds of booting the computer, short of it getting taken over by some botnet. So there were all these advantages to it. There were all these advantages when working on open source software.
I was already working with Apache and MySQL and... picking up Ruby a few years later, PHP, all that stuff was just better on a Unix-based setup. Now, there were about a million trade-offs nonetheless. That original version of macOS I ran, ran on an 800 megahertz power PC with 256 megabytes of RAM, and it sucked in terms of performance.
It was so much slower than what Intel and Windows were able to put out. And I still didn't care. I was like, I'm just happy to have some alternative here that gets me out of Gates land. And I pushed on that for the next 20 years. And. didn't really realize that I was getting slowly boiled.
Apple went from being the ultimate underdog in 2001, who saw an interest for them to be a good partner to developers, to be emphasizing and assisting and doing all these things for people who wanted to build software, especially for the web. And then slowly, but well, actually not that slowly, suddenly, they had a much bigger platform on their hands after the iPhone got released.
And that platform introduced the concept of the App Store. And that, to me, if I was going to pinpoint a moment when we look back upon this history, everything started to go to shit, was when the App Store and native apps started taking off. Now, that's a...
Perhaps a weird thing to say because in many other ways, this idea of having mobile supercomputers in our pocket that can run this incredible software, video games. I used to be a big gamer. They can run video games at 120 FPS with graphics that would shame even the wildest, craziest dedicated PCs of the 2000s. That's amazing, right?
And that's true, but it really put the squeeze on developers' freedoms and independence. And kind of that kept building to a crescendo. First, when we launched Hey.com, our email service back in 2020. When we put that out there and got that dreaded no.
I'm just remembering this. I had forgotten all about the Hey and Apple stuff.
Yeah, I haven't. I'll tell you that. Because we had literally spent two years almost at that point developing this system. Millions of dollars poured into it. I had slaved away for many hours developing this system. We were incredibly proud that we were going to take on Gmail with a fresh new system based on thinking from 2020, not 2004. And we thought that was going to be the big boss, right?
We're going to take on Google with everything. an actually quite good email system. Gmail is a good system. It's dated, it has issues, whatever, but it's a good system and it's free. And we're going to come in and say, do you know what? You should pay for email. What?
So already that had long odds, but we didn't even get to begin that fight before a bigger boss showed up and just like Apple sat down on our chest and said- You're going to give me your lunch money and 30% of everything you own in perpetuity going forward. And we're like, what? No, no.
So that led to a big fight over multiple weeks where the future, the life of that new service hung in the balance because – The reason Apple has all this power is that they have all the economic activity, at least in the US. What we eventually found was for our email system, hey.com, 85% of the paying users we have, they use Apple products.
Wow.
So if we were shut out of Apple's Garden of Eden, it was going to be dead. There wasn't going to be a business. There was going to be 15% left, and you just can't build a competitive business in there. So that's where the whole antitrust thing comes in. But it was also, for me, just a wake-up call. Okay, the pot has been boiling for several years.
I had already, well before we even introduced Hey.com, I testified in front of a congressional field hearing in Colorado where they were looking into all these antitrust matters and going like, hey, here's my perspective on this. Things are not going in a good direction. Developers are getting squeezed in all sorts of different directions.
The guidelines of an app store is dictating the kind of businesses that can exist in the world because of this economic activity issue. That's not a good place to be. In fact, we should look at the internet as a far better alternative model for both application distribution but for business formation, for independence, for freedom, all these things. And then I got my own kind of fight with Apple.
And you know what? Even that wasn't enough. I fucking still bought three iPhones after that. And like three or four more MacBooks. So, I mean, I think that goes to tell you, A, how sticky platform choice really is. And B, that Apple does make good stuff. Oh, their hardware is so incredible. Their hardware is really good. I have found since that...
Even though that point is true, I was a little myopic. There are actually other people making really good, interesting hardware. They don't make the same. If you want the sort of millimeter, unibody, aluminum case stuff that Apple does so well, Apple does that better than anyone else. But there are different things you can choose to focus on where you can actually end up with something better.
repairability and and expandability and there are other parts of the hardware experience that can be better what are the laptops i'm blanking on the name i've seen you talk about these they've got like modular are you using a framework currently i am framework is my uh is my laptop it has been my laptop that was actually i mean i know it's a little long-winded here but this is how we get to the story of how i ended up on linux because it was framework's fault
And I use that currently. I also have a desktop PC. I mean, the other thing I realized was Apple does their stuff really well. And they also charge proportionally, maybe they would say. But, I mean, I had a specced out MacBook Pro. The last one I bought was an M3 Max something. I think it was $3,500 or $3,700, whatever it was. It was more expensive.
than the cost of my Dell Precision with the latest, fastest Intel chip, which by the way is quicker than that fucking M3 Max chip kind of work that I do, which is web work and running Ruby and whatever. It's actually quicker than that. And then I also could buy a framework machine and add that on top. Now, I fully understand there are a lot of people who are like, I don't want two computers.
I just want a laptop and I want to run everything on it. So there's still advantages there. But it was eye-opening to see what was actually going on because I really have for quite a long time just thought like, do you know what? I'm not in the market for anything not Apple because I'm in the Apple world. I'm running Mac OS. I'm deep in the ecosystem. That's the other way they get you, right?
Like the little bangs, little hooks. I was running the photo setup they have. I was going to call it iPhoto. I don't think it's called that anymore. I think it's just called Apple Photos. And the syncing and all the things.
And all the historical data you have. It's like impossible to switch. All the data was embedded with it.
Before we introduced Hey Calendar, I had my calendar in Apple Calendar. Before we introduced Hey Email, I was using Apple's own email client, so there was just a lot of hooks in it. But one by one, those hooks were already coming out. I mean, Hey Email and Hey Calendar took out two of perhaps the biggest ones I had been used to, that syncing of things back and forth.
Now we had Hey Clients on both email and – or on mobile and desktop. There were still some, and I had to sort of pry them out like thongs, right? Like you've got to stick them down your throat. Get it out there. And it took a little while. And for a while, that's the amazing thing when I look back upon this. This just happened, by the way, right?
I just switched to Linux full-time in, what, February or March? And it already feels like it's been five years. But I still do remember those dark spring days trying to get out of the ecosystem and thinking, I don't even know if I can do it. I didn't even know if I want to do it. Oh, it just seems so impossible. There's so many things. Blah, blah, blah. And then I spent...
Well, first I got hooked. And the way I got hooked was I was like, all right, I don't want to use Apple stuff anymore. The final straw was when Apple, in their sort of malicious compliance way, said we're going to pull PWAs, the standard for making web applications, out of Europe.
And we're going to rip it out of Safari because that's the way we choose to interpret what the European Union is doing with their antitrust drive. And I thought, I just built another app that sort of depended on PWAs. We had built a app called Campfire under this once.com umbrella where we will sell you a web chat system, kind of like a slack in a box for a single fee.
And part of that setup was we were not going to make native apps. We were going to lean on the web and we were going to lean on PWAs and we were going to lean on all this stuff. And then Apple like literally two weeks later went like, yoink. We're going to take the PWA support that we barely just introduced out of the whole thing and we're going to screw over the entire ecosystem.
And I just went like, I can't buy another iPhone after this. I can't buy another MacBook because if I, in this position of almost infinite privilege in terms of resources and whatever, if I can't even make it to Switch, then I should just roll over and accept that Apple owns all, and I don't want to do that.
Then, all right, let me just go back to my native Danish roots and grow potatoes or something. Eggs and bacon or whatever else. I don't even freaking know. I just don't want to do computers anymore. That's really actually the point it got to.
That was the final motivating point, which is insane that that is the level of pressure it needs to bring that like, I don't know if I want to do computers anymore. As a profession, as a business, because I felt like the reason I got so, or have been so excited about computers and loved working with computers was the internet. And why was it the internet?
Because it was a free platform of distribution. I didn't have it to ask anyone for permission to launch Basecamp back in 2004. We bought a domain name. We pointed that at an IP address, running a server, and voila. Yeah. We're in business and anyone who shows up to Basecamp.com can buy our project management system. And if no one shows up, that's no one's fault but my own, right?
Like my faith is in my own hands. That's not true with native applications anymore. Your fate of your entire business is in the hands of Apple bureaucrats who every single time you submit an update go –
Me, I don't like it. That color blue is a little too blue. Rejected. And then you see me again. Sir, I brought you the right color blue. Me, I still don't like it. The sign-up screen, the font is too small.
And you're just like, it's like begging some petty, inconsistent tyrant for the right to grow wheat. And I'm like, are we fucking back in 1506? What the hell is going on here? I don't want to be that kind of peasant. I want to own my own landstead here, right? And the internet allowed that. And I just thought, like, how are we going backwards?
How do we go from this glorious age where developers and software businesses who have direct access to their customers, and we got to enjoy that for, what, seven fucking years from the start? Com bust until the iPhone was introduced. And then another Dark Ages was introduced where we basically went back to, oh, no, you can't just distribute software. What are you talking about?
You need a publisher and that publisher needs to make a deal with the distributor who needs to have a deal with blah, blah, blah, blah, blah. And it's like, again, no, I don't want to do that. So anyway, very long story too. Now I have all this pressure going on me. It's either not using computers anymore and fucking retiring with the potatoes and the pigs, or it's finding another platform.
And the first step I went was like, oh, let me see what my old arch enemy is up to these days. So I went back to Microsoft with an open eyes and I thought like, do you know what? People change. I mean, It's been 20 years, 25 years actually at that point. Well, 23 years. Let's call it that.
Since I last ran Windows sort of proper and there's a new boss in town and I hear good things about like what he's doing and I like the investments they've done into VS Code. I like their stewardship of GitHub. I like a lot of things. Like maybe this Microsoft is not the same thing anymore. Yeah. And I showed up and I ran Windows for a couple of weeks and even said, you know what, this is it.
I'm switching to Windows. And then I was running Windows on this Framework laptop. And I was like, I guess I could make this work, but it was kind of begrudgingly, right? Like I was like, ah, shit, I still don't really love Windows. I found a new respect for it, I'd say. First, I used to think, you know what, there's no way I can enjoy doing what I do on a Windows machine.
And that was in the days prior to WSL. Before the Windows subsystem for Linux, the Windows subsystem for Linux, especially WSL2, is really effing good. Like the amount of steps it takes from you to have, I have a Windows machine to I'm running Ubuntu inside of it. And it actually mostly kind of works.
There's a little bit of the file system barrier you have to finagle around, but it's actually quite good. That was really a revelation. And I thought, oh, this is why I can make it last. But then I installed Linux on this framework laptop. And the first thing I noticed was, holy shit, it runs faster. Like I just ran my Chrome speedometer 2.1 test and I was like, Jesus, why is it 15% quicker?
Why am I getting 15% more juice out of this CPU that's inside of it? Huh. And then I thought, like, that really is nagging at me. And I thought, like, you know what? Maybe I'll give Linux a try. First, I ran into a bunch of issues. Like, the original Framework 13 screen was of a dimension set where if you ran it at 100%, the font was too small and it didn't look great.
And if you ran it at 200%, everything was a little big. And I was like, that's not quite right either. Yeah. And I just went through all of these steps of finding, oh, I like something about Linux. Oh, I don't like that thing about Linux. And I spent maybe five weeks just going like, all right, you know what? I'm going to learn this. I'm going to figure it out. I'm going to get a desktop.
I'm going to get a framework. I'm going to set it all up. And I spent like two of those weeks battling with NVIDIA drivers, trying to get them to work with my Apple 6K XDR display, which, by the way, spoiler alert, it never will. It won't. I went through... About 400 layers and wild goose chases trying to decompile things and kernel flags and whatever. Don't do that.
Don't put yourself through that. Just buy an AMD graphics card. If you're not doing literally top tier gaming that requires a 490, then the AMD stuff is awesome. It works out of the box with a 6K XDR display. I could have saved myself two weeks. But I still like the journey. And I like the journey because Linux was a challenge. And I think Linux still is a bit of a challenge. I mean, it's funny.
I just posted something yesterday about this where I said, like, hey, do you know what? I think more developers should run Linux because it's a challenge.
You learn a lot about how stuff works.
Because it's not as straightforward and paved as Windows is or Mac OS is. And then I get the normal stance Linux turns. What are you talking about? Ubuntu is just as easy as Mac OS. Yeah. I mean, come on. Let's get real.
It's not.
It just isn't. It just isn't. I'm glad you got your grandma set up with Ubuntu and she's running Firefox and she doesn't need anything else. That is awesome and I'm not disputing that. But for a lot of normal software – like if I was going to convince my wife to run Linux, yeah, I was not going to do that. She just uses too many apps and they're too specific and I'm like –
I'm not like, hey, babe, do you want an adventure in setting up like your webcam? She'd be like, no, I don't want that adventure. What are you talking about? Versus if you are a developer, I think there are actually legitimate benefits. And maybe some of it is just you kind of just acclimate to your environment.
But I thought in just those six months that I've been using Linux full time, I've gotten far more comfortable with Linux in ways that are pushing me on like, what are we doing on the server? And this is what I see as one of those main benefits. Anyone who works in web development today, they may work on a Mac, but they deploy on Linux. No one is fucking deploying on a Mac box.
They're not deploying on a Windows box either. I know there are some people actually literally doing that, but like 95% of the internet runs on Linux. And if you are a developer working with the web, I think you owe it to yourself to understand what's going on. And I don't think that the cloud is able to pave over that experience to a degree that is, helpful for you.
And I also think it locks you into a bunch of overpriced premium AWS wrapper services that you'd be better off without. That's a mission I'm pursuing. But regardless, Linux is a little bit more of an adventure, but holy shit, what an awesome adventure it is. And what an incredible oasis you're able to arrive at. That was what actually blew my mind.
I didn't know legitimately if you spent the time to set it up, Linux could be Not just good, but here I'm really going to go out on a limb, better. And again, I am biased at this point. I accept it. But I like the Linux setup I have today better. I'm not like making do with scraps here. I like it better than what I had on Mac OS. That's pretty freaking incredible.
I'm not saying, again, it's better for everyone. I'm not going to force my wife to convert to Linux tomorrow for the reasons we talked about. For me as a developer, Linux is better. And that was quite the revelation. I mean, I literally legitimately did not know. I still had an idea of Linux that was based in, what, 2009 or something? Yeah. That wasn't accurate.
My mental model was out of date, and I got a nudge, thank you, to explore what modern-day Linux looks like. And not only was it better in terms of ergonomics, not only was it better in terms of sort of the infrastructure plumbing, not only does Docker run about 400 million times better on Linux... But it could also look amazing. I mean, discovering Linux porn was actually kind of cool.
I don't run a Linux porn setup, but I took a lot of inspiration from that. When I finally compressed all of this journey, exploring all the Linux tools and whatever, into sort of my... take on what a great out-of-the-box learning experience should look like for developers, which is what ended up becoming Amacube. Amacube.
Which is what I run today, which is Ubuntu 24.04 underneath, and then a layer on top that is aesthetic personalization, or not even personalization, customization,
And a bunch of things installed just out of the box make the Linux experience feel like it's incredible without having to sign up for 140 hours of YouTube Linux nerdery videos and tutorials and like, oh, you have to compile this thing with 400 different flags. Then it works with the other thing. Someone just do that. And I ended up doing it. And so I'm like, let's share.
And that's where I am today. I am running Linux full time. I freaking love it. I'm doing it on a framework laptop, on a Dell Precision desktop. And I even got into freaking mechanical keyboards because of it. And then I put all of it into Amakoop and released it. And now I'm on the evangelical trail, trying to at least give other Mac users an invitation that...
There is life outside of Apple's garden and it's quite pleasant in a lot of ways. It's not the same. And this was one of the actually hurdles I had to get over. When I first switched to Linux, I was like, okay, the text editor I have on my Mac is called TextMate. It has a certain theme. It looks a certain way. I run all Hallows Eve and like all the keywords are exactly like this.
And it's been drilled into my brain that this is what code looks like. This is what code feels like. So the first thing I tried to do is replicate TextMate in VS Code on Linux. And you can get about like 96% there. And I was ready to blow my brains out over the last 4% because it was total uncanny valley. It was just like, yeah, everything is the same. It's not.
And then we just get frustrated and then we're just like, I should just go back to the Mac. Why am I making do with something that's only 96% as good? I could just swallow my pride and come crawling back to St. Cook. But I was not going to do that. I was going to stick with it, and I realized the way to stick with it was to make a clean cut, do something else.
So the first thing was like, all right, I'm not going to run TextMan. I'm not going to try to replicate TextMan. I'm going to give VS Code a fair shake, and I'm going to run it with a different theme. And this was how I discovered Tokyo Nights, which is the theme of all time.
Yeah, it's what I used to.
And then I set that up and I worked with it for a bit. I was like, actually, VS Code is not bad. Not text-made. I miss a few things, but now it looks different enough. It feels different enough. That is something else. But the final kicker that really got me, like, this is actually better. I was like, VS Code, I was like, this is tolerable. I can see why people like it. It's not text-made.
I miss a few things. But then I went to NeoVim. NeoVim, by the way. There we go. There we go, right? NeoVim, by the way, I think you were supposed to say. And with NeoVim, particularly through the distribution of the legacy Vim setup, where everything comes pre-configured for you, I went like, oh, what? This is, yeah, nice. I remember some of the Vim shortcuts from literally 20 years ago.
I think back in 2000, I was doing free BSD server setup for some stuff and I had to learn some Vim motions to do that on the server. And I still fucking remember how to program. And I also remember a couple of other things. And NeoVim was basically like, reactivating that ancient knowledge from the back of my brain. It's like the emotions never leave.
They just take up space and maybe it gets moved to the attic at the back of your head, but they're still there and you can pull them out of the attic and you go like, oh yeah, actually I can see why this editor that is literally 50 years old still has purchase on people. And it was through that that led me to discover all these amazing TUIs.
I mean, TUI was even a word that, that's actually the nickname we use for one of my boys, like a TUI bird. I was like, oh, TUI is also terminal user interface. And I discovered NeoVim, then I discovered LazyKit. And I was like, holy fuck, what even is this? It feels like I'm in a movie.
Yeah.
Like the alien mainframe or something. And there's just something incredibly appealing about that neo-vintage, neo-nostalgia kind of look that isn't just about aesthetics, although it is also about that. And this is, let me just make a quick aside here. There's a huge contingency of Linux super neckbeards who are... Dax. What? No. Or with aesthetics, right?
Like it's as though they take pride in making things as ugly as humanly possible with the worst color coordination you can imagine and about five pencils in their pocket protector. And I love that those people exist because that is why we have the lens that we have, right? Like that's the sort of the cement that we can then build upon, and that's great. I really care about aesthetics, though.
I want a beautiful workstation. I want a beautiful work environment. And NeoVim, together with all these TUIs like LazyGid and whatever, offered me a more compelling aesthetic than the one I had on the Mac, while also leveling me up in terms of capabilities.
That to me is sort of that match made in heaven, where it both looks incredible and it feels incredible, and you're able to be productive with it. And now that I've been using NeoVim for quite a while now, I mean, you feel like a god.
It's amazing.
I sometimes do a set of motions where I'm like... Making or selecting text inside of like a quote with like two commands. And then I do whatever the thing where it's just like, I just got to stop. I was like, did I just fucking do that? I just fucking do that. Where you're just like, this is incredible. Like. I just benched a new record.
Yeah, it like created a mini game to text editing. It's amazing. I've loved it late in my career.
And it's the best kind of game. It's one where like the payoff is actually literal productivity that you can deposit towards goals that you have in life like building businesses and so forth, right? It's not just sort of – wankery for its own sake. So anyway, all that led to my first being a reluctant Linux user, almost like a Mac OS refugee.
I just need to find a tent somewhere where the wind is not too harsh and whatever. I'll make do on this rock. Until realizing, do you know what? I could just walk five feet further. There's a fucking oasis down here. And there's goddamn coconuts and pineapple and just everything is flowing. How did I not know this? This is actually the reaction I had. I knew about Neovim.
I had never heard of Lazy Kid. And I was like, how the fuck did I go 20 goddamn years not knowing that like this entire universe existed just around the corner? Just around the corner. I could even have been running it on Mac OS. I mean, a lot of these TUI tools, they run just fine on the Mac too. I just even didn't know. So that's where I just got more and more confused.
grateful for the entire expedition because it was taking me to foreign lands and foreign experiences. And I liked a lot of it. And now I've ended up in a place where I love my computer more than I have loved a computer in at least 15 years. The last time, and this is both on the software and hardware side. On the hardware side, the last computer I really loved, I mean, I don't know why.
Why does love work the way it does? It was the MacBook Air 11 inch that ran the shittiest Intel chip I think you could buy at the time. So it was woefully underpowered, but it just, it felt right. Like the size of it was right. The screen of it was right. I loved that computer for many years and I carried it for many years. It was my primary development tool for many years.
Then the MacBook era came in, and first we went through the awful five years in the desert with the butterfly keyboard, where at our company, 30% of all laptops we bought ended up broken. That's an incredible failure rate.
Within two years, we did the math, over 30% of them had to go back for repairs, some of them multiple times, it never really got to work proper, and Apple was just like, you're typing it wrong. I don't think that's the problem when 30% of us are ending up with that. So you went through that, right? And then we got to actually the best place Apple has ever been in. They went back on that.
They went back on ports. They introduced legitimate steps forward with the M chips and efficiency. And you ended up with a product that was really good and also really boring. And boring in the ways that something that is too perfect can be. I think this is one of the reasons why there's such a surge or interest in imperfection. Why do people like a manual stick sports car?
Why do they like vinyl records? Why do we like mechanical watches? Why do we like these things that are objectively worse than their counterparts? Oh, you can get an automatic transmission. It'll shift smoother than you. You'll get better gas mileage. You can get a quick... quartz watch for $20 that'll beat the accuracy of any Rolex you can buy? Yeah, but also no.
And that's what I found when I found with that Framework 13 laptop. I was like, do you know what? This has quirks. Weird, interesting quirks. It's not perfect. But the things it's good at, it's really good at. First, A reminder that keyboards really fucking matter. Holy shit. Again, I'd only been in Mac land. I was like, actually, this Mac magic keyboard is fine. It's great. Great.
What?
I didn't know that that, Matt, why does half a millimeter or 0.7 millimeters of travel matter? Oh, yeah, oh, yeah, it really does. Matte displays. I was like, oh, shit, I remember the PowerBook that had the matte display. Matte displays are really nice for programmers. It just fits text rendering much better. Maybe it's not as good to watch a blockbuster movie on, but it's great for text rendering.
Three by two aspect ratio. Holy shit, what have I been doing in this 1610, 1609 land? That is a weird, elongated, optimized for just consumption kind of display. That's not what I use my laptop for. I use it to make stuff. And when I make code, 3.2 or 3 by 2 aspect ratio is much nicer. So there were all these advantages on top of the upgradability.
Framework came out with a new freaking screen like whatever, a month ago.
Yeah, the 16-inch, right?
You could buy the screen by itself and replace it on your freaking laptop. I was like, what? That's crazy. You mean I don't have to buy a whole new computer just because I want the newer display?
As an Apple user, that's just insane. That sounds impossible.
I mean, I have literally, and I'm not proud of this. I'm actually ashamed. I have four MacBooks sitting in my closet. And the reason why I have four of them was... I wanted the fastest chip. Every time you wanted a newer chip, you had to buy a whole new goddamn computer. I wanted whatever, there was some upgrade to the screen that came out at one point, buy a damn new computer. It was like...
this doesn't feel quite right.
Yeah, it starts to feel bad when you see all that stuff piling up in your closet.
Yes. And then now I have my framework. I've upgraded the RAM. I think it came, I think I bought it with like 32 gigabytes. It was like 200 and some dollars to buy 96 gigabytes of RAM like off Amazon. And I could just plop it in. I bought obviously the new display for it. I upgraded the battery capacity in one of them, too.
It came with a 51-watt-hour battery, and they had a newer 61 that gave you another 10% battery. It's like, holy shit, I still have the original one that I mutate. That's really cool. So I fell in love with that Framework laptop. And to me, then, the pair of the two, Framework plus Amacube, that is the Linux version that I had.
Mm-hmm.
tingled and optimized, really felt like this is incredible. So that's where I am right now. I mean, really happy with the Linux setup and just, I don't know if I'm angry or disappointed. I'm just a little like, I kind of wish I knew this like four years ago. Like I should have done this in 2020 when Apple was just like, we're going to break your kneecaps here unless you pay us 30%.
That should have been the signal where it's like, oh shit, why am I still here? Just like it was when the original iBook came out in 2001. I was like, why am I still on Windows? If there is an alternative and it's good enough, I should switch. So, all right. That was, that was, that was. That was a nice one.
No, I mean, it's good to hear it all.
Yeah, that transition is painful, but like you said, you can get to a better place on the other side. And I think there's a lot of technical details, but I think what it comes down to me is, you know, I sit at my desk and there's this thing sitting here and my whole life is coming through it. Like it's what helps me provide for my family. It connects me to the whole world, everything.
And knowing how that thing works, like having deep customization of it and like feeling that harmony with it, like it's just good for your soul, like beyond any of the technical details. And kind of when you get there, it's hard to explain to other people, but you really feel a lot more connected with this really important thing that's, you know, sitting on your desk every day.
It's funny. I'm reading. I don't know if I started it right at the same time. I don't even know if it was an accident or just serendipitous. I'm reading the book, The Zen and the Art of Motorcycle Maintenance. I knew that's what you were going to say. And it speaks exactly to that point. It speaks exactly to the point of, do you just want to be a consumer?
I mean, this is a story about riding motorcycles in the 70s and they needed, a fair amount of tender love and care to keep them running. And you needed to know about timing chains and how to oil things and tighten things and whatever.
And in the book, he makes a very compelling argument for exactly what you say, that there is just a satisfaction in the soul of knowing how the things you interact with on a daily basis even depend on work at least at a basic level. And I find that Linux... just forces you down that path because there is a few more timing belts you have to tighten yourself.
And it is good to know where they are and what the commands are and so forth. But what you come out on the other side as is a more knowledgeable, experienced, comfortable, and confident technologist. And I think that is in great lack right now. And where I see it hugely is in this discussion about the cloud.
So cloud technology and services and platforms as a service have convinced an entire generation of developers that connecting your own server to the Internet is way too fucking scary. It's like running that motorcycle backwards, blindfolded on one hand. Like, that's just fucking crazy. That's evil, kenevil shit to connect your own computer to the Internet. I was like, wait, what?
How did that become to be the case? Why are we so afraid of connecting computers to the Internet? And I think a lot of it is based on, first of all, outdated mental models.
I mean, anyone who's been around technology, as I said, there was a time where if you bought a new Windows machine and you connected to the internet and you hadn't already set up the antivirus stuff, you could give it five minutes and it would be infected. That's kind of a long time ago. And Linux was never like that. And I think those mental models haven't fully updated.
And then the cloud providers have literally put in billions and billions of dollars to scare the living bejesus out of anyone from connecting computers to the internet. Because if someone realizes that they can do that,
they will realize that a lot of the cloud machinery is insanely overpriced, that it's an economic model of renting versus owning that they actually don't appreciate, that it's a surrender of independence and whatever else that isn't very flattering. And Linux, I think, is the antidote to some extent.
If you run Linux on the desktop yourself, you will be familiar enough with the basics of, oh, UFW as a firewall, for example. It's actually super easy to use. It has one of the best user interfaces for any firewall I've ever seen. It's super quick just to say like, hey, lock this shit down so I can only access this on 22. Oh, this is a web server? Open up 80 and 443. That's it. Done.
Oh, yeah, okay. Turn off password authentication. Only SSH keys. And now you have fucking Fort Knox. Good luck getting into that. If you could get into a box that is set up like that, it has UFW running. It locks down to just those three ports and you switch from password authentication to SSH key authentication.
If you get into one of those, if you can get into a box like that, I mean, you just won the fucking lottery. You have a zero day. I mean you can sell that to North Korea or China or make them bid on it and boom, $5 million clean in the bank. So that's the sort of basics, right? Like it's actually not that difficult to secure a box.
It's not that complicated to run pseudo-app upgrade if you need to update something. You don't need to do it that often. And also, by the way, didn't you just set up a fucking JavaScript – interpolation deployment compile pipeline that involved 700 megabytes of node modules across 5,000 dependencies and you were able to figure that shit out?
You're telling me you can't add one line of configuration to the SSD daemon and you can't run UFW allow 22? Oh no, that's too hard, man. Shit, just way beyond my capacity. What? Man the fuck up or woman the fuck up. That's pathetic. That is pathetic. And we can beat that shit out of you. We can do some server phobia.
reverse cognitive behavior with you you're gonna okay listen listen i know you're afraid of the server it's not gonna bite you it is not gonna hackers are not gonna get in there but first let's expose you to linux see here's a computer it runs the linux you can touch it it's okay just put one hand on the space bar and push it push it one time see nothing bad happened
And I think that level of cognitive behavioral therapy, that's part of the reason why I'm excited about things like Omicube. Can we introduce that level to this generation of developers such that they will realize that running computers on the internet is not some exotic, dangerous thing that's only for Linux wizards and neckbeards who've been doing it for 40 fucking years. It's for you too.
It's never been easier. It's never been cheaper. And... The stand you're able to take for independence and pricing and all the other advantages that you get out of that is totally worth it. Totally freaking worth it. It's nowhere near as complicated as figuring out how to make a 1972 Yamaha motorcycle work or the timing belts. So...
We need to disabuse people of this nonsense, which includes sort of addressing head on the fact that the AWS reseller model is incredibly profitable. This is why all the VCs want to do it. There was just an announcement, I think, yesterday about another web framework who also wants to be an AWS reseller. And I totally get it. I totally get why VCs look at that.
Wait, so you're telling me I don't even have to own computers? I can just rent those computers and then rent them again to someone else? Oh, that sounds like a great business model. I have no inventory. I have no staff. I have no nothing. I just mainly have to convince developers that they're stupid and fragile, so that they need my shit. And then I can just... run the money printer.
Yeah, no, that's not the model of the internet that I'm interested in. And again, I'm putting things to a point. I appreciate the advantages in developer economics. Heroku, for example, really did amazing pioneering work in the AWS reseller space, and they taught The rest of us, do you know what? Deploying to a server could be a whole lot easier, and we should make it easier.
But to me, that is essentially shining a light on a part of the infrastructure level of how we work with the internet that open source should then address. And it's not like, oh, you've got to come up with Photoshop, right? That's the whole Photoshop versus GIMP debate that happens. Yeah, do you know what?
Even as a now converted Linux fan, I can look at that and go like, yeah, GIMP is not as good. I'm not saying GIMP is a bad piece of software. I'm just saying end-user software like Photoshop is difficult to make on this model. What's not difficult to make on this model is goddamn infrastructure. All the best infrastructure in the world is open source, from Linux to MySQL to Nginx to whatever.
The entire stack, even the entire stack that's being used in the cloud, it's all built on open source stuff. So this fact that we're going to give up the last mile to a bunch of VCs and a bunch of closed source commercial passes that. No, no, no. This is where we take up the mantle and go like, we can make this easy. We can make this better.
I've been trying to push that envelope with Kamal, a deployed tool that we used to build on top of Docker to get out of the cloud. We used to be in the cloud. We used to be on AWS. We used to be on all this stuff for a bunch of hours, things with Basecamp. And hey, and we yanked all of it out because cost was just getting ridiculous. And we built a bit of tooling.
And now I'm on a goddamn mission to make open source as capable, as easy to use as all these AWS resellers we have against any box running basic Linux with an IP address you can connect to.
Yeah, I think beyond the fear that you described that people have, I actually think there's another dimension too. There's almost like a pride. I think there's like this narrow definition of productivity where they're like, if I have to learn something, That's not productive. And they kind of have this very narrow path of like, I'm someone that loves being productive. I'm all about business.
That means I value these things. But a lot of this stuff, like you said, when you go and understand these underlying things, that does make you more productive in ways that are really hard to explain. You get advantages in other places. So yeah, I think for me, it's more like there's a weird pride with not knowing this stuff. I think that's like... hard to shake people up.
That's straight out of Zen and the Art of Motorcycle Maintenance. The guy he's riding with is riding a BMW motorcycle, and he explicitly takes pride in not knowing anything. It's part of a core identity.
There's a whole philosophy in that book about why that is, why does that develop sort of our struggle with modernity and the hardness of knowing everything and then essentially becoming eluded, not wanting to know anything, and then developing an entire identity around that. And... I think that's really interesting.
I also think it is sad when it comes to developers because it is not internally consistent. It is not an internally consistent model of how do you become a capable developer? Because first of all, as we've discussed, you're figuring shit out all the time. To make a JavaScript Project from four years ago compiled requires a fucking PhD at this point, right? Like archaeology and whatever.
You've got to dig out all these old – oh, we're using Webpack. Politics, yeah. Oh, that was in the ancient times of 2018 that we used tools like that to compile our stuff. So if you're able to figure that shit out, Don't tell me you can't set up a Linux machine. They come pre-baked out of the box.
For most cases, there's very little seasoning you have to apply to that dish for it to be delicious and good and nutritious. So I think it is weird, and it's weird in fascinating ways of like why are humans weird? And I also think it's fixable. I think it's fat-driven, and I think we can absolutely reverse course. The cloud, when that was first introduced, sounded nuts.
Wait, I'm going to rent computers the entire time, even if I need them all the time from some other person who runs a everything store? What? That sounds nuts. Why would I do that? And then a few years happened. And again, we realized that there were some real advantages to that. But a lot of it was also just like a mental shift, right? A mental shift. And do you know what?
Those things can shift back. And we can knock things straight. And some of it is pendulum swing, right? When it comes to setting up servers, I think the main shift for me is the difference between pets and cattle, right? Is this like your lovingly nurtured machine where you've tweaked out your little configs in a bespoke way and it lives just in this box?
And if you need another box, you're like, holy shit, it's going to take me 20 hours to set that up. Or do you live in a cattle world where you're deploying containers? And that's where all the interesting configuration happens in this world. It's repeatable and it's scalable and it's all these other things.
Now, that pet versus cattle idea, which really got pushed forward with the cloud, can totally work on your own stuff too or work on dedicated machines or whatever. So some of it is – relaying the fact that you don't have to give up on modernity to understand how things work. You don't have to give up on containerization to understand how a Linux box can be connected to the internet.
Like these things are actually not in opposition. And as we found when we moved out of the cloud, We had a team of about 10 people on our ops team, and a bunch of them were cloud native specialists. That was their main thing. Then we moved out of cloud onto our own stuff, and they could retain like 90% of their skills.
90% of the system administration work was the same whether we owned the boxes or we rented them from Amazon, right? And I think that's the part a lot of folks aren't appreciating. They're thinking, wait, computer? Does that mean like... It's running in your basement. How do you secure that? Do you have a baseball bat down there in case someone tries to break in and get it? What even is this?
He's like, no, that's not how it works. There are professional data centers that you just rent a space for, and then you put in your own computer. And there's just a basic education here where I kind of feel like it's like we're discovering an ancient civilization. And people go like, how the fuck did they build those pyramids, man?
I don't even understand how the stone could get up there and weigh like 400 tons.
Kind of crazy technology. And then the people who built the pyramids were just like, hey, man, I'm like 45. We did it like nine years ago. This isn't ancient technology that we need archaeologists to figure out. It probably wasn't fucking aliens either. Like we knew – and we know how to do – the people are still alive.
The people who set up the internet as you know it today, which was set up not on hyperscalers but by individual companies owning computers and connecting them to the internet, they're still fucking alive. There's kind of a lot of them too. And – We can rediscover that. We can rediscover that and we'll be better off for it if we do.
And a bunch of the advances we've had in technology since then have actually made it easier to do that. When I got started with the internet, well, not even the internet, just to say, when I got started with Basecamp, when we launched Basecamp, we launched Basecamp on an Intel Celeron 1 core, I think, CPU with 2.8 gigahertz. Is that right? Something like that, right? One fucking core.
One 2004 core, by the way, that shit was slow. I mean, by modern standards, two orders of magnitude faster. I just bought a hobby box from Hetzner just to rent and play with an experiment, right? 48 cores, hyper-threading, 96 vCPUs for $220 a month. What? The amount of computing power that's in that machine could operate probably 95% of all SaaS businesses that exist in the world.
And a napkin math that I did yesterday on Twitter said like, do you know what? We probably only need like five of those boxes to run all of Basecamp, which is a SaaS company doing tens of millions in ARR. Computers have gotten insanely fast and capable and productive. And you wouldn't know...
half the time if you're dealing with a reseller of a reseller of perhaps a reseller of AWS because that shit is still expensive. I mean, Heroku, again, I love Heroku and I'm just going to bang them a little bit. Like, we'll literally charge you like 200 bucks for like half of a vCPU and two megabytes of RAM. That's a little over the top, but
Hardware is really good, really fast, and it's getting faster and cheaper like all the fucking time. Moore's Law, once you look at multi-core setups, is very much alive. Shit should be getting twice as fast at half the cost in no time at all. And it is, except if you buy it from a reseller of a reseller.
Yeah, the point about the tools working, like the cloud tools, like also all those things being retained, that's a good one. So I work on infrastructure as code stuff and all of our tooling was developed for deploying cloud services, right? So there's good ideas there, like declaratively defining config, like applying that. And we realized the other day,
we we can just point all the same stuff to like a docker daemon running on a machine and it all works exactly the same so all the great benefits that we kind of invented in the cloud uh you can just use that same model uh anywhere so yeah it's quite a different experience and i also posted the other day i was remembering
the first my first company i actually launched it off my laptop like it was i even kind of it was like a it was wi-fi it was like a wi-fi laptop right and this was like in 2010 uh eventually we moved that to servers in my parents basement like literally in my parents basement and we sold the company at that point and i remember like the first thing they did they were like yeah we need you guys to move that off of there it's probably not secure but i actually did build and sell a company that
We had a dog, you know.
We had a dog, yeah.
Love it. Yeah. It was dusty down there. Those servers are still there, too. I could probably still use them. But yeah, like, you know, I did build and sell a company that way. It's a lot better these days to do something like that, though.
It is a lot better. And I think that that's also where... These things, I'm not in opposition of it. I'm glad the cloud thing happened. I really am. I think it moved the ball forward in a lot of ways of how to understand how to run things. Some of the conceptual models, the pets versus cattle, the declarative setups, all that stuff worked. We owe to the cloud.
We owe to the billions and billions and billions of dollars that got pushed into it. But we don't have to stay there, right? I like to think of it a little bit sometimes big pharma. So someone puts in a billion dollars to develop a new drug. We should be thankful as humanity for that to happen. It is really expensive to do the R&D. Lots of the R&D doesn't work.
Even when the R&D does work, it goes through all these clinical trials. They cost absolutes. Absolutely astounding sums of money to do requires investors to underwrite all of that. And those investors will only do it if they occasionally get a slam dunk to pay it all off, right? Incredible. Equally incredible, the fact that patents expire after 19 years, 18 years, and then shit goes generic.
And after it's gone generic, the price goes to essentially what it costs to produce the compounds, which is usually nothing, right? Like, I mean, I guess I can get as riled up as anyone about the state of the American healthcare system, but I can't get that riled up about the idea that like a pill costs more than the chemical compounds it takes to produce it because that's not the expense, right?
Like the expense is developing the drug that works and is safe and could be rolled out. And America, just for whatever reason, Peculiar reasons. It's a bankrolling the whole world on this, right? Like Americans pay through the nose for their medication and that's why we have all this medication.
Anyway, so when all that happens, it's also amazing that we have the forcing function of the generic threat, right? Like first of all, it means that these pharmaceutical companies can't just develop a drug and then the next 200 years milk it like Mickey Mouse. Things will actually expire and they have to come up with new shit or they'll go out of business.
And just as importantly, the advances for humanity accrue to all at zero cost once shit goes generic. That's the model we should think about with commercial software development and cloud development. The cloud was on a patent for quite a while and it developed a bunch of cool new shit and that's great and we should be grateful for it and I'm grateful for it, but also now it's time to go generic.
Now it's time for these advantages in developer ergonomics and deployment and containerization and of the wazoo to go generic so we don't have to pay rental fees until the end of fucking time. The internet today exists the way it does because of open source software.
There wouldn't be – and I owed as many businesses if every goddamn vendor wanted a per CPU license charge as though you were buying Oracle in 1998, right? Like that was just going to seriously curb the dynamism of the entire economy. So we need to –
Respect the fact that the capitalist forces are pushing these things forward at the frontier and that's amazing and I'm a capitalist in that regard very much so. And I'm also a goddamn communist when it comes to open source software and like that it all should be free and we should be fighting to move more things generic and so forth.
Those ideas, even if they sound contradictory, they fit inside someone's head. They can fit inside someone's head. You can wrestle with that paradox and be better off for it by embracing both at the same time. But it is well overdue that we go generic with the cloud nonsense.
I mean, that resonates a lot with me. The work that we do is... Some of these resellers you're talking about, sometimes they do come up with cool ways to deploy stuff or they come with stuff that we wouldn't have thought of. And what we do is we take it and we just make it open source so that people can use it wherever they need to. So...
There's like a it's a little antagonistic, but there's also like a like a harmony to it as well. Yeah, I think all capitalism, the idea of capitalism is there's no permanence. If you have permanence, it doesn't work. You need things to get destroyed. Yes. Occasionally. Yes. And you can't be a capitalist if you don't believe in that.
And I think this is why I've been so vocal about the cloud issue in particular, because I've been seeing what looks like the formation of new monopolies. And that is partly coming because these hyperscalers are not operating in a traditional free marketplace because it is very difficult for companies to actually switch. And again, it's been the promise of the cloud almost since day one.
Oh, we're going to use all these declarative configs and so forth. And you're going to be able to just like, you don't like pricing in AWS? No problem. You just switch to GCP. You don't like that? You switch to Azure. Yeah, no one's doing that. No one's doing that.
And in fact, much of the evangelism around the cloud is like, you're not doing cloud right unless you're using all these proprietary APIs. Unless you're fully taking advantage of the managed services and the serverless and the this and the that and the other thing that is usually just peculiar and particular enough to a single cloud vendor, you're not doing it right.
And that's, of course, the maximum amount of lock-in you will ever get on a business if once you go down the path of using bespoke, slightly tailored, slightly different version of these open source tools that most of this stuff is built on at AWS or other places. We went through that. It took a fair amount of time to get out of the cloud.
It was not about just changing the IP addresses of sort of the Kubernetes deployment target and then pushing the button and it all working. Maybe one day we will get there. I don't know. I think there's a very strong incentive from the hyperscalers for us not to get there. They don't want a competitive marketplace.
They don't want ease of migration or ease of change because that's going to bring pricing pressure. In fact, I just saw – I think it was yesterday that prices for the same amount of compute in total refutation of Moore's Law is going up. Like, cloud is getting more expensive for the same amount of compute. That's fucked up.
Like, that to me is an example of a captured market that's not responsive to the normal competitive pressures of a functional market, right? And I think we need... Open source it for nothing else for that. Pricing power becomes extreme when you have full control of the market. Just ask Apple. We started out that discussion.
How the fuck are they able to take 30% of everyone's business on the App Store? Well, because they can and there's no other alternative for you if you want to ship software to iPhones, right? So if you can control a toll booth like that, the cloud providers are not quite like that. They're a little more like a roach motel. You can check in, you just can't check out.
And we need to make it like a regular hotel. You can check in and then you can stay for a while and then you can leave without it costing an arm and a leg in the first born.
Yeah. So the one thing I will say I kind of disagree a little bit is I think if you are going to use a cloud, if you're going to use one of the public clouds,
if you try to use it in a way where you're just trying to rent servers you're gonna have a horrible experience like it's way too expensive to do that if your approach is uh i'm renting servers i'm gonna run my own software just don't use the cloud at all just go to like you said that's stuff that you guys are using um go directly to data centers that offer renting the hardware directly you can have a much better experience
If you are going to use a cloud, you might as well, like, just commit to something, like, commit to using it properly. And then, yeah, you can use the services, high-level services. They're more expensive, but at least you get the benefit. I think it's this middle ground where you, like, try to use a cloud in this agnostic way. Like, that's just never going to work out at all.
So whatever model clicks for you, like... If you like console servers, just go buy them directly. It's probably at least 10x cheaper, if not more.
That was my mental model, and we used a lot of the managed services. We used RDS and that whole setup. And Unfortunately, I would say is that that's where you get reamed the worst. Like the managed services on RDS and with search was the other one we used. What was it called? Open search or something else. AWS. Elastic.
I actually wouldn't count both of those because I guess what I'm talking. So those are like just shitty repackaging of RDS. Stuff that you can just run directly, right?
I actually thought they were decent in terms of economics. They did save us some degree of some kinds of backups and whatever. It was a little bit there. But the upcharge was so freaking insane. We were spending... On RDS alone, we were spending more than half a million dollars a year in rental fees.
It cost us, I think, to replace that, like the hardware and setup, we bought $125,000 worth of machine and then we owned those machines. Now we still have to pay for power and we still have to pay for some other. But holy shit, like literally the payback on the machine purchase was like five minutes, right? Like five minutes to rent and we could buy all these huge, hunky beasts of machines with,
amazing amounts of cores and RAMs and all the other stuff. And we just have to connect them to the internet again. There's something there. So yeah, I'm not disputing the fact that there are cases where the cloud makes sense.
And where I've usually said is, especially if you have a huge dynamic range on like peak to trough in terms of your usage, like one day you're doing whatever, 100,000 requests a second, and the next day you're doing 10,000. That is a huge range where you're going to have a bunch of idle servers sitting around if you're going to build out your setup so that they can take it.
So there is a tip-over point where that dynamic range I think makes sense. And then there's also a tip-over point where you're not even using a whole machine, right? Like, hey, I just need to rent like 18% of a computer because that's all I need right now.
Yeah, the granularity.
So there's some granularity there. And then I'm also sure that there are other cases like access to specialized hardware is a good one. Good luck trying to buy a bunch of H1s or H100s, whatever. The hyperscalers did have access to them and they would rent it to you. So I think there are plenty of cases where it does make sense. I think my argument has been that the default case is not – Right.
The default case being like, hey, you should run cloud on everything and running your own shit is like trying to produce your own power. That was one of the analogies that were used for a while. Well, are you in the power production business? No. Then you should just buy power from the socket. Are you in the cloud data center business? No. Therefore, you should just – yeah, that one doesn't hold.
It doesn't actually work. You are much closer to the same dynamics whether you rent it or you own it. A lot of the difference is in the finance model. And CapEx versus OpEx and whatever. And if you actually ran the numbers, you would be surprised. Now, part of the irony here is it's actually very, very difficult to run the numbers in the cloud. We had like FinOps. That's a cottage industry.
You're a consultancy specializing in understanding your bill. If that is not a red flag of red flags, I don't know what is. Yeah, this is so complicated. I need specialists with PhD in statistical analysis to just understand what I'm paying for. What? I mean, and we saw this ourselves. We tried to, well, not just tried.
We worked very hard to understand our bill, like how to proportion it and what is that application spending. That was tens of hours a month. trying to understand the bill.
It comes up all the time. That's crazy. And after we bought our own servers, yeah, that
went away. Like there was just a whole category of work that just disappeared. We did one big analysis when we bought all the servers we needed to get out of the cloud for whatever. And it was like, I don't know, five, $600,000, not an insignificant sum, but like our cloud budget was 3.2 million. So like proportionally not that big of a deal. Anyway, we do that analysis once.
And then we're like, all right, see you in five years. That's the next time I'm going to worry about whether I need an AMD EPYC 9454 or a 9754. Like, yeah, I don't need to do this every day, right? With the cloud, you kind of sort of do. Because if you don't, it is so easy for these things to run away from you.
The $3.2 million we had on our cloud budget really should have been twice that at least. It would have been if we hadn't been Nazis about chasing down the last penny and stopping all the right services and doing all the optimizations and entering long-term contracts. It's the other thing that blows my mind. People have this conception, oh, the cloud is like just metered as you go.
Yeah, no, it isn't. No one actually using the cloud at scale is paying like per the minute. They are entering into at least one-year contracts on committed contracts instances, because that's the only way to get any pricing that makes any kind of sense. On AWS's S3 service, we were in a four-year fucking contract. Four years we committed to get any pricing that kind of sort of made sense.
There's a lot of contradictions in all this stuff. Again, different businesses are different in different ways. And I'm not saying there's one size that fits all here. I think there is an opening, though, for us to reconsider what the default should be when you should start doing the calculations. And also, if you do know Linux a little bit, the alternatives won't seem as...
foreign or as inconceivable as I think they do to some companies, right? Who have grown up in a cloud-first world and have a team that's familiar with instantiating new AWS services. And then they go like, what do you mean? Ethernet?
where like that that's fixable yeah yeah so i i think where we can agree on is uh like i'm still gonna like i use the cloud heavily like clicks for me um but what i don't like is the people that are like quote unquote on that side don't know how to do the other thing so it's just like i can't really trust your opinion even though you're like on this side because
you're doing it out of not knowing the other side. Like I've always had a home lab that I managed, uh, would run whatever, like I would use to expand my old, like Kubernetes with nomad, like all the different orchestration stuff. Um, so I've seen how that stuff has progressed and I, I enjoy it.
Like I said, just like with my, with my desktop, it's, it's really good when you like curate, that's a garden almost, um, running all like all the services that power my family stuff. Um, So I have a sense of that. So when I choose stuff, when I'm choosing the cloud, I'm choosing it with like awareness of what I'm trading off and what I'm balancing.
And I don't think that's what most people are doing.
Yeah, I agree with that.
I don't think that's what most people are doing. Most businesses are doing. They're not doing it from like, hey, I know how this works if we're going to run it ourselves. I know how it works in the cloud and I can make the math work or I can make the economics work. That is not what I'm seeing broadly, and I think that's a shame, and I think, again, we can fix it.
And I think part of the fix literally is exposure therapy, getting more developers exposed directly to Linux so that they realize that the operating system they're going to run their servers on is not this scary alternate universe where everything is upside down. It's actually, first of all, quite fascinating. familiar in many ways.
Like if you're a developer working on the Mac, the hop to Linux isn't like, if you were a Windows PowerShell user and you didn't want to run WSL, okay, that's going to be a culture shock of some dimension when you hop into Linux, right? If you've been running on the Mac and You run Homebrew and you're familiar with some of these tools, whatever. It's not like going to a different universe.
It is like going to perhaps a different continent, but they kind of still speak a version of English, just kind of broken. I guess like Americans going to Australia and just going like, hey, mate. That's like the Linux version for macOS folks switching over. It's a dialect, right? Yeah. The difference is in the dialect.
And the difference is realizing that this is literally the same stuff as you run on the server. That should make you far more comfortable in thinking you can run it too without needing a PhD in Linux hardening.
So I have a quick question for you. So this is a thought I had the other day because I feel a lot of the same things you're talking about where I'm like, people these days don't even know how to X. Yes, I am. And you remember when we were young, people would say that to us. So... Like, are we just doomed to repeat this? Is this actually going to change? Is it actually different this time?
Right?
I'm in perfect speed. How amazing. It worked out that way. I do think about that all the time. And exactly for the reasons that you say, I remember when I started working with the web and I had a bunch of programmer friends. And those programmer friends were used to writing
applications in assembler in c and a lot of them had a very difficult time with something like php where they just go like that is literally 10 000 times as inefficient that for loop as if i wrote it in assembler and did a move and uh whatever x or i don't even fucking know i don't know They had a very difficult time adjusting to that, right?
I do try to think about that often when I go like, well, back in our days when we had to set up servers and connect them to the internet, we walked barefoot in both directions and it snowed all the time. So you've got to be careful not to... Turn into the literal Simpsons episode of like the old man shaking at the sky.
Or the cloud.
Or the cloud, yeah. Right? While at the same time also realizing that not everything new is better. And that we've seen that over and over again in all sorts of domains where I think it is a very healthy time or healthy instinct that we should constantly be pushing forward. How can we make things simpler? How can we make them faster? And we should be using those –
additional CPU cycles, for example, to make ourselves more productive. And this was actually why I am so passionately angry at the cloud is because I felt like they didn't hold up their end of the bargain. I did not feel like things got sufficiently easier, that I was paid back in additional productivity. We were never able to materially change the configuration of our operations team.
When they worked in the cloud, there were some things that were slightly different. They weren't worrying perhaps about, hey, there's a warning here about the second PSU in this sled that needs to be changed. That was not a warning they would get. That would just go to the – I was about to call it cloud monkeys. We're all fucking cloud monkeys. Yeah.
And someone in that data center would run around and you would think, oh, man, the computer is so sophisticated. No, that's fucking just George. And George is sprinting with your damn CPU over to that box and he's pulling it out. You just don't know George. George lives behind an API.
Sometimes these big data centers, there's only like there's only like two people that are there like on call. Yeah, it's not even like a whole team or anything.
There's some of that. And that's why I think I would have, I mean, that's why I was a cloud believer actually for many years. I was very passionate. Like this is the way the future is going to work. We just need to get over this hump. Then it's going to get easier and cheaper. And it never fucking did. Never fucking did. That's why I'm still mad.
I tried for more than half a decade to find cheaper and to find easier. The only thing I ever consistently found was faster. And I mean faster in terms of procurement. It is still amazing. I will grant that with one API call, you can spin up like a thousand servers. It's going to cost you probably a house and a condo, but you can do it. That's pretty fucking cool.
You can't do that by going to Dell.com and ordering a bunch of PowerEdges, right? Like that's going to take like a week and a half, at least maybe two before they arrive and before they're racked. So that was always there. But to me, it was always about the three things. It was supposed to be cheaper. It was supposed to be easier. It was supposed to be faster.
And it's only really the speed of procurement that I saw at our business. Again, as we've talked about, there's different considerations and different analysis you've got to make on your own business. But I was just so disappointed. That the promised land never seemed to arrive.
That when I look at setting up a new AWS installation, when I read through the, like, hey, get started with AWS 101, I went, like, this shit is off the scale complicated. Just even remembering, like, how I do the IAM user management configuration in the cloud, I'm like, I feel like I have to go back to school to study again. Yeah.
Some of it's just different things and you learn different things and you could say Linux is also complicated. Okay, yes, but also no. Like there are different scales of complexity and I felt or I feel like the cloud in many ways did not held up its end of the bargain. That it promised us simpler and we didn't get it.
All right, I got a question and this is kind of moving off of all the cloud stuff. You said earlier that the Apple stuff, made you mad enough, you considered hanging it up, retiring. I think in my generation, we're kind of the generation right below yours that looked up to you. Are you calling me old? No, no, no. You might not even be that much older than us.
You guys have equal amounts of gray in your beard. I'm the one that's still all black.
I guess I've always wondered, like someone like you, you get to that level of success or whatever. You could not do this anymore, but you choose to. And I wonder, like, do you have this idea in your head of what it would take for you to kind of stop programming? Yes.
If Matt ever decides to introduce static typing into Ruby, I'm out. So that's one. The other one would be if somehow browsers would only accept TypeScript and not JavaScript anymore, I'm out. Interesting. No, seriously. As long as I'm still having fun and I feel like I have my independence, that's really the two factors I gauge things on. I'm having a tremendous amount of fun.
This is one of the reasons why I'm so passionate about the Linux thing. It's really been... I was about to say a journey of a lifetime. That's probably actually fair. I haven't had that many platform changing. I'm just going to mix it all up. I'm going to take 20 years of ingrained muscle memory about where the Apple key goes with copy and rewriting that to the control key.
That has been a ton of fun. And building all this tooling we've built to get out of the cloud has been an amazing amount of fun. So I actually don't think I'm going to stop thinking computers are fun. Like I just fundamentally really like computers. Like they are just on my days off, whatever, on a weekend occasionally.
I was like, I just want to go like type on my little keyboard like for a while. I just want to, I'm going to fix a bug. I'm not even hit by the bug. I don't even fucking care about the bug. I want to hear clickety clack from a mechanical keyboard. I want to commit something. I want to push it. I want to do a release because it's just freaking fun and I enjoy it.
So I don't think that part is going to go away. Again, Matt, if you're listening, short of just fucking putting static typing into Ruby, not going to happen, man. I'm going to stay on Ruby 3 version or whatever before that until the end of time. The other one that's more likely is that it stops becoming possible to run things independently.
That you have to run on, say, a hyperscaler and release through an app store. Then I'm out. Like I don't want to do that. I don't want to go through multiple layers of permission. I don't want to have to beg fucking Cook about whether I can release a new idea into the wild, whether I can release software. So as long as we can keep the internet strong, and I think that's not as obvious of a –
premise as i would have assumed and advocated for it being in like 2008 like the internet is under pressure from a lot of different forces and this idea at the very least of a global internet seems more or less over like we are fracturing the internet at an alarming rate and okay maybe that can happen and maybe we just have to accept that like
At some point, like there's going to be a European internet, just like there's a Chinese internet, just like there's halfway Brazilian internet. And they're not kind of connected, or at least they have very severe firewalls that prevent things from going back and forth. I hope that America is not getting there. I don't take it for granted, though, and I don't think anyone should.
If you care about the internet in its sort of original ethos and form, I think you're going to have to get ready to fight for it to some extent. What does that mean? I don't fucking know. I don't want to get into it. political about that in any way. Just this idea of appreciating what we have and realizing how unique it is. The internet as a software distribution platform is truly one of a kind.
If you had explained it to any VC today, they would have gone like, yeah, but where's the toll booth? Can we just install one around the whole thing or do you have five at different corners? How do you do it? How do you capture all the economic activities that's coming out of this thing?
And the fact that we ended up with such a powerful platform that wasn't owned by any individual company and backed by any individual VC is just mind-blowing. And I think it's You only realize how unique and rare it is when you look at the regression we've gone through with the app stores. That to get software onto mobile phones, it's not like that at all.
It's an absolutely dominated platform with a single gatekeeper who tells you whether you have a right to exist or not. So if that's what the internet either becomes or the internet disappears or gets so kneecapped by all these fencing in that it starts resembling that, okay, I'm also out.
But it sounds like the computer's thing.
That's for life. I've always wondered like... I hope so. I mean, this is actually one of the reasons I look at AI, right? Like we talked a little bit about that at the beginning. And people are like, they're so excited that like AI is going to take all programming jobs.
And to some degree, I can be excited for that in the same way that like, did we need, I don't know, 2 million people tending to horses for our economy to function? Maybe not, but I don't really like horses, but I know people who do and they still ride them. I hope to be that. Yeah. I am still writing the manual code. I'm typing in the commands.
I'm not just narrating to Cursor 5X AI to build the shit for me, right?
Like in a museum.
Exactly. And I do actually think that that is – there's still going to be space for that. But I think it's very – plausible, and you should certainly consider that we've reached the high watermark, like that the number of people employed doing programming for a living has peaked. It's entirely possible.
If you look at almost any other phase of the economy that has happened, the number of people who work in agriculture, the number of people who worked on the assembly line, there are plenty of precedents for this idea that there was some glorious times for a certain industry and for the workers in that industry, and that ended. For one reason or another, usually economic or progress or otherwise.
Right. And I don't know what that's going to look like. And I think I don't want to be a fear monger here, but I do accept the fact that there are people who make their living doing this stuff who are right to be a little bit. nervous about it.
Yeah.
Right? I mean, if you worked in Detroit in like, whatever, 1972, and you were putting together Buicks, and life was pretty fucking good, and you had a, whatever, good setup, and two cars in the garage, and all this stuff, like, Yet that doesn't exist in the same way anymore, right? Is that programmers' future? I don't fucking know. I hope not. I hope not and I hope also, right? Isn't that weird?
Like we think back on like do we want to go back to subsistence farming? Do we want to go back to like 98% of the population like literally swinging a hoe? Probably not, right? We also don't want to go back to the assembly line that that occupies everyone. But it's very hard in the moment to like, what comes after? What is the next? Where is it all going to cycle into? I don't know.
And I think sometimes the cheerleading is like, yeah, we're all just going to become prompt engineers. Yeah. No, I don't think so.
Yeah, I mean, it's just like your transition to Linux. It was painful. You didn't know that it could be better, but eventually you kind of got there.
That's what I choose to hope. I choose to hope that like our AI overlords are going to be benevolent and we can still prompt them and they're not prompting us and all this good stuff.
So one last thing I have to plug this because you mentioned you like to ease. You're like blown away by like how crazy to get. Have you seen what we've done with our coffee shop? I have not. So we built a coffee shop that's served entirely over SSH. So if you do SSH terminal dot shop, that's awesome. You get like a full coffee buying. You can literally buy coffee and it shows up at your house.
But I'm going to try that. I don't drink coffee, but I will try.
I don't drink coffee either, but I sell it.
uh but it was fun because so i love it adam worked on a lot of the like the front end for it but it was cool because it's a very constrained ui you can't have like different font sizes you can only do like colors and stuff but the constraints kind of breed like a lot of cool stuff and build yeah constraints are the best there's like amazing tui tools that i've also been going deeper and i've built a
I love it. Yeah, it's wild. Like it's better UX than a lot of the more powerful, you know, environments that we have UI in.
I dream of doing a TUI for Hay all the time and even a TUI for Basecamp. It's funny because I remember my first sort of exposure to TUIs was travel agents and flight attendants. Like they work in, I don't know if they still do, but they used to, right? And they go fast.
They go so fast.
Yeah. They were on a mechanical keyboard. They were hipster nerds before that became a thing, right? They're like, I know all the fucking things. F12, F13. Yeah, you can get to Bangkok through Shanghai if you stop over in And you're like, Jesus smokes. Have you tried to use a travel website these days?
I mean, this is half the banner on my wife and I's direct chat is just like, how the fuck did airlines get so shitty? How did shitification just devour the travel industry to the point now where I'm like, I pray for travel agents to make a comeback. Yeah. It's so goddamn painful to book any kind of travel through a web-based interface. And I'm a big fan of the internet. I'm a big fan of the web.
I've built like 20 years there. But I also have to accept defeat that like a lot of shit on the internet is just perpetually getting worse. And if they had the 2E constraints, you couldn't get that shitty. So maybe we should get back to both travel agents and 2Es.
There we go. Everyone's going to be running their own servers and we're only going to be serving up 2Es over SSH. That's the future. Yeah. All right.
Thanks so much for coming, David. This has been so good. Yeah, that was really fun.
It was funny. Appreciate it. All right.
Thank you. All right. We're really bad at saying goodbye and like in your podcast. So just a heads up.
Bye. See ya.