How About Tomorrow?
Rita Kozlov on Building In Public, Cloudflare's Birthday Week, and How Long Software Actually Takes
Mon, 07 Oct 2024
Rtiz talks with Dax about building towards an event like Cloudflare's birthday week, launching and getting feedback from the community, Cloudflare's platform journey, how long it actually takes to develop software, Dax's experience with containers, and weather in various parts of the country.Want to carry on the conversation? Join us in Discord. Or send us an email at [email protected] kozlov 🐀 (@ritakozlov_) / on Twitterrita kozlov | blogRita Kozlov articles on Cloudflare's blogRita Kozlov on LinkedInBirthday Week 2024 at CloudflareTopics:(00:00) - Adam doesn't like Rita (00:57) - Introducing Rita Kozlov (01:16) - Keeping secrets like Cloudflare Birthday week (06:01) - Excitement over launching and getting feedback (09:13) - Cloudflare's platform journey (19:54) - How long software actually takes to develop (27:52) - What's your experience with Containers on Cloudflare? (49:31) - Fall in various parts of the country (01:00:55) - Planning for our NYC November event
Adam couldn't make it today. He did this really fun thing where he was like, oh, I can't make it on Thursdays anymore. Can we reschedule to Wednesday? And then I rescheduled to Wednesday and he goes, okay, cool. Can't make it then either.
He doesn't like me. That's okay.
That's what it seems like. And we're just going to go with it. He really, we talk about this all the time. He hates conflict and he hates people, this type of situation. So it would bug him a lot if... that's the conclusion you came to. So that would actually be great.
Great. I'm going to assume that then.
Yeah. So today we have Rita joining us. Rita works at Cloudflare and Cloudflare just had their birthday week last week where they announced a bunch of really cool stuff. And we've been friendly for like a while now. And this is kind of overdue to hop on here and have a little chat. So how's it going? How are you doing? Are you tired from all the birthday week stuff?
Launching is exhausting, but it is also really fun. So a little bit of both. I had the weekend to recover, and so now I'm mostly pumped.
Nice. Yeah. So when you all plan something like this, how long of an effort is it? Is it like...
a six-month effort like how long are you are you in like okay we're prepping for birthday week mode it really depends i mean everything that we announce uh and especially this past birthday week uh came from user feedback and things that were already on our roadmap because users have been asking for logs users have been asking about you know
workers and pages convergence, users are always asking for better performance. So there are things that in some ways, you know, we start planning for it quarters ahead of time because that's a part of our roadmap. But then, you know, we start kind of planning the week generally about,
a couple months out, about a quarter out, and we start to look at, okay, well, these are all of the things that are happening. These are the things that are launching. What's kind of interesting out of that and start to piece together out of that, okay,
these are the stories that we want to tell during the week this is what we want people to come away with and uh you know refining that and sometimes that leads to new announcements that emerge that were like you know we weren't planning on doing that but maybe that makes sense for this week and yeah yeah it's always funny for me to see all of that because
I am so bad at keeping anything quiet. Like when I'm excited about something, I just, I mean, you guys see what I'm like on Twitter. I just start posting about it right away. So like, I'll make like 3% progress on something and I'll just want to announce it already. So yeah, it's like hard just keeping things under wraps because I'm sure it's really exciting.
I mean, to be honest, we're the same way. And so, first of all, we build in public so much. Our docs are open source. We have our Discord. So if you're paying close attention, nothing that we announced last week would have come as like a huge surprise, right? And yeah, you'll see in the weeks leading up to it, myself and Matt and Dane and a bunch of engineers on the team kind of, you know,
leaking sounds so dramatic but really we're excited about what's coming and we can't shut up about it so some companies are stealthier we're we're really bad at stealth we're like everything's out in the open but uh if you're not paying that much attention then you know this is your time to kind of tune in and we really try to then spell out okay here's the stuff that launched
Yeah, it's funny because I think, I guess now that you say it like that, this maybe is more common than I imagined because AWS has their re-invent thing once a year, but it's become such a common thing for everything to get announced beforehand. There's a good term for it. It's called pre-invent. Everyone's like, oh, pre-invent season is here.
And that's when you hear about everything before the actual launch. So I guess, yeah, I guess everyone is just the same. Yeah.
Yeah.
The closer it gets, the harder it is to keep shut.
You know, I really admire companies like Apple where everything is, you know, super on lockdown. And, you know, part of me wonders, is that something that you mature into? But realistically, I think it's either something that is or isn't in your DNA. And that's just really not who we are.
Yeah. It's funny. It's funny you bring up Apple because I was thinking about them, too. I have this weird I had this weird thought the other day where I'm like, Man, I don't know anyone that works for Apple. And that's not exactly true. Like, I have here and there run into people that work there. But I feel like people join there and they just disappear into a void.
And, like, you just don't know anyone that works there. You never hear about a single thing. So, yeah, how do they pull that off when they're, like, also one of the biggest companies in the world?
Number one rule of Fight Club.
Yeah, it's I don't know. I definitely couldn't work there because I'd be like, guys, look, look at this thing. Yeah, I think both strategies work because obviously you get the big single day impact when you're quiet. But then a lot of these things, especially developer tools like People might get... I think it's different than you see an iPhone and you're like, I'm going to buy it.
You see it, you're going to buy it. Whereas developer tools, you see it and you might think it's cool, but you're already working on something that you're not going to drop everything and switch to it. So I think the transaction times are longer. So just... like having stuff trickled out and staying on top of people's mind, I think, uh, does make a lot more sense for, for dev stuff.
I think it's okay.
Yeah. I mean, the other thing too, is we want feedback on stuff that we're building. So to your point, right. And iPhone, you ship it out and it's out there and that's it. And you can ship software updates, which also, if you're like me, you're maybe always a little bit behind because, uh, Who has time for that? But yeah, you ship the iPhones and they're in people's hands and that's it.
Versus for us, a big part of the reason that we let people play with stuff early or we pre-announce it, we do a lot of betas is because we want feedback. And by the time that we're launching something, and this is something that also is actually us developing a bit of a new muscle and something that's changed a bit over the years.
is, yeah, we take a bit more time to bake things in, to receive feedback. So the thing that launches on the day of the innovation week is much readier, has gone through a few iterations. Previously, we would first announce something and then put it into people's hands and then iterate from there. And there's pros and cons to both approaches.
And still, depending on the announcement, we like to have both in our arsenal, but a lot of the feedback. And I think especially for developer tools, it's just so much more fun if something is announced and instantly you can NPM install or whatever. The first thing you want to do is like, let me play with this. Yeah.
I think that's much more interesting and a lot more fun for us too, because then on those announcement days, we get to kick back and look at the Discord and people are asking questions or they're like, look, Ma, I built this thing. You're like, I just announced it. How did you already do that?
Yeah, it's funny. I think no matter how much progress we make, whenever we launch something and someone actually uses it, I'm always like, Really? Like, you actually used it? Like, it's always surprising, you know, that you get, like, people building real stuff. And, yeah, it's always surprising how far people go with it in such a short time.
Especially with stuff that, you know, like I said, we, like, we're, like, very public with everything we build. So even when stuff isn't ready, we'll see people grab it and, like, take it kind of, like, way too far. Like, maybe more far than I would say, like, you should even be doing right now. But, yeah, there's always people that'll do that. It's super motivating to see.
But I think that that's kind of the fun part, too, of working on developer tools. Yeah. Sometimes you don't even announce something at all. It's just an API and someone will find it and someone will build on top of it. And I forget what the there's a name for this law.
Yeah. It's like if you put something into API, someone will use it and become a permanent thing, like no matter what, even if you don't attend to. Yeah, I don't. There's a name for it. I'm blanking on it, too.
Yeah. Matt, the second he listens to this, will be like, Rita, it's obviously this.
Yeah, I know all the laws. I just never remember the names. They're all familiar, but yeah.
You don't get to look it out and sound extra smart.
Yeah. Okay, so you've been... So how long have you been at Cloudflare again?
I've been at Cloudflare for over eight years now.
That is so crazy. Okay, so I guess eight years ago... There's obviously been this... effort from Cloudflare to build, you know, like a Cloudflare platform for people to build applications on. That probably started around eight years ago? Like, when did that really start?
It started seven years ago, or I guess a little bit more than that. But... the, the first product that we announced in the space was workers. And, you know, the, the original, original intent of workers was, okay, we have, um, we have this network. We have, some people use it for CDN, other people use it for WAF and security. We're trying to expose all these services through it.
And there were kind of two parts to it. One, which is, uh, You know, we looked internally at our own pace of innovation. We're like, wow, we need to be able to go faster. And in parallel, our customers were getting more sophisticated. So they wanted to do, you know, crazier and crazier stuff that page rules just wouldn't cut it for.
Page rules were like, oh, you give me the URL and you can, you know. cache everything on that URL, but people started to want to do things by headers, by country, by cookie, and take all of these complicated actions. So we're like, okay, we need something that will allow people to just write code and deploy it. And we need a platform for ourselves. And so we released workers.
And if you go back to the announcement blog post for workers, you'll see a bunch of examples that are kind of around, you know, adding headers, caching things, but you quickly realize, oh, wait, I have an origin actually that's running, like if I can write code, In this very serverless way, like, what else can I do? It's crazy. Innovation is crazy in that way.
You come up with something for one thing, and then you realize how many other things are possible. And that was the beginning of the developer platform. And then we're like, okay, well, now if I want to be able to build something a bit more meaningful, I need storage. So then we introduced KV, and that was, you know... pretty limited in what it could do, but it exposed a ton of use cases.
And then we're like, okay, what's the next thing that people need to be able to do? Oh, they need all these deployment tools. So, you know, we released Wrangler and worker sites, which was the first iteration of Pages and then became Pages. And then we've continued to add different primitives.
Yep, to the point where today, you know, people are building full stack applications that I think were well beyond our imagination when we first started working on this.
yeah the being able to like program your cdn thing it's it's interesting the way you described it because we basically saw the exact same thing on the aws side we had you know we're helping people deploy all these things behind a cdn and they kept being like well i want to do this if the user it speaks this language or i want to do this if they're from this country and i want to like redirect them in this place and it just becomes
it's funny how like a whole computing platform came out of that need until it makes total sense. Cause there's just infinite custom logic people want to apply. And yeah, it was the same thing. Like, you know, initially it was just, you had patterns and you can like send them different places and like, maybe like rewrite the header somewhat.
Uh, they eventually did add like a computing thing, but it's, it's so hilariously limited. It, um, You can run logic when a request comes in, but it can only run for one millisecond. It has one millisecond runtime limit. And it's like a, you already know this, but I'm forgetting the engine. It's not like, it's like a very slim down. It's not like Node.js or like worker D or anything.
It's like pure JavaScript, like no other APIs, nothing. And even that like unlocked a lot, but yeah, it's just not enough. You need like a full thing eventually.
Yeah, it's interesting. As you were saying this, I was like, wow, actually there are so many different approaches to this. Before we actually embarked on workers, one of the things that we're looking at because that's what so many Fastly users had was VCL. That's a really, really complex, at the end of the day, it stands for varnish configuration language.
It's a config, it's not Turing complete, but it is very, very expressive. And we saw a few challenges there.
One, which was if you compare the number of people in the world that understand JavaScript versus people that understand VCL, like every time I would talk to a customer that was, you know, looking at moving to us, they would just be a little terrified because it's like, oh, well, Josh knows our VCL configuration and God forbid if Josh leaves or maybe Josh did leave.
And then you're talking to this team that's like, Yeah, I don't know what this does and I'm terrified to touch it because everything's gonna go haywire. So that was one approach. And we knew from the beginning that we wanted something more programmatic than that because you can't do particularly complicated if statements or for loops or really express logic through that. And at the time,
AWS had Lambdas and they had Lambda at Edge. And both of these are slightly different approaches too, but Lambda at Edge, it's interesting too, because they have a couple of different definitions of Edge too. So it wasn't running in all of their CloudFront locations and was still container-based. So Again, but more limited.
But again, for us, we're like, okay, well, we're not going to deploy containers because that's going to introduce a cold start. And for our use case, that undoes everything that we're trying to do. And so, yeah, you have that approach. And then, yeah, they kind of have this new approach now. That that's a bit more in the middle.
But to your point, yeah, I think when I played around with it last time, I was trying to do something even like return a custom response. And you really are limited to there's an assumption of you're making a request to origin. You're going to get a response from the origin and you can modify kind of the metadata on either side of that. But you can't go too far beyond that.
yeah and you can't change this is a huge issue we can't change the origin so if we want complex you can change the path you can't change the origin so if you want like to build like a complex routing logic you just can't do that yeah and for the longest time they kept directing people to lambda edge but that runs like, after the CDN. Like, you can't change a lot of things.
So a lot of this stuff feels like they stumble into a problem and they, like, do a quick reactionary thing. The thing you brought up with VCL is funny because I think this pattern shows up in a lot of places. There's something called VTL, which is a...
same concept in aw i mean it's i think the velocity template language or something um but aw supports in a bunch of places where instead of like writing code you can write these config things and like you said it's very i think it's very expressive but not like really that expressive and almost nobody knows how to use it and the upside is it's like it's quote unquote just config the downside is like nobody knows how to use it it like looks different than the rest of your code it takes you entirely out of like the mindset you have when you're working on other stuff and our son that's seaside
there's like great infrastructure as code tools out there like Terraform. But again, the same trade-off, it's more configgy. So if you have like conditionals or loops, it looks a little weird and it looks nothing like the rest of your code. So bringing a lot of stuff into JavaScript is really nice because you immediately get all the return complete stuff.
Like everyone needs to do conditionals and loops, no matter what type of thing you're building, like you're eventually gonna need that. And two, everybody knows it. I think the part that we've seen is tricky is even though we've given you the ability to do like a normal programming language, you should still like kind of chill out with it.
Like you shouldn't like, you should still kind of treat it like it's config. You shouldn't like, Like, we see people in their infrastructure stuff with SST. They build, like, these crazy abstractions and, like, they build all these packages that are reusable and all this stuff. And it gets really far away from something that really should kind of be like config.
So, yeah, it's just that, like, you give people more power and it's good. But, you know, you got to make sure that they don't create the same mess that often happens everywhere else.
Yeah, I think that's a really interesting question of what layer is the right layer for configuration and what layer is just your application logic. And an interesting thing for us, too, was if you actually look at that original use case of I want to modify my CDN, actually a lot of
teams that manage Cloudflare that are an infrastructure team that's used to CDN management, they're a bit like, don't give me the Turing complete stuff. I don't manage a code base in that same way. Give me a way to just manage rules.
And so over time, we've actually still continued to build out this really complex rules engine that is much more of a configuration language because that's what those people want. And yeah, it turned out that this layer was much more suitable for okay, well, actually, I just want to write my application in this way.
And yeah, if you're a team that's, you know, an engineering team that's building features, then that's exactly how you want to write it. And you want to, you know, you have all of the development practices that you need in order to maintain this thing from, you know, source control to testing to, you know, the rest of it.
Oh yeah, that's a huge thing. That's kind of what I mean by it takes you out of it. With normal code, there's just such a history with it. We all know how to write it. There's all this stuff we collectively learn over time on how to collaborate on it, how to test it, how to make it reproducible. There's so much knowledge there, and I think a lot of...
tools try to like remove code uh and replace it with something else and there's some benefits of that but like you're throwing away like such a deep understanding and also tooling like editors are so tuned to like some of these popular languages um and just having all that stuff work out of the box for everyone is uh is quite nice the other thing i wanted to talk about was um you said it's been seven years as working on this uh
Yeah, I just want to highlight how long something like this takes. I think people conflate this idea of you need to build something, get it out there, and get feedback. That's good, but they kind of conflate that with... And then it all works out really fast also, but like this stuff takes a long time and it takes like commitment to doing some of this stuff. So yeah.
How, how has it been just, you know, it's been, it's been quite a while. You've been seeing this and working on it.
You know, if you told me seven years ago that I'd be working on this for seven years, I think I would be a little surprised, but you know, it's in some senses really rewarding in that. you know, not to get into career advice and things like that. But I think an interesting thing about working on something like this for such a long time is you get to see your decisions play out.
And you're like, oh, you know, in retrospect, that was a mistake. Or I'm so glad we did that versus, you know, because it really does take a long time for these things to play out. Like in one or two years, you only see the beginnings, if any, of the ramifications of the different choices. And yeah, I think it's always there's a quote for this or something like that.
But anyway, yeah, you get less done, I think, in a year than you set out to. But at the same time, you get more done, I think, in five years than you would have thought you would. So when I look at how much is available on the platform now in terms of, you know, like primitives, that blows my mind. I don't think I would have predicted that we would at this point have
object storage, queues, workflows, logs, like there are so many AI, there are so many products. But then there are other things that, you know, one of the first PRDs that I worked on at some point was gradual rollouts. I think that was one of the first things that Kenton and I talked about.
I would talk to a customer and they'd be like, you know, it's super cool that I can push out a change and it's live to your network of 200 data centers in less than 30 seconds. But that also terrifies me. Can I please have a lover by which it happens over time? And there's this weird thing that happens where you're like, yes, that makes a ton of sense. Let's start building it.
And then a ton of stuff gets in the way where you're like, OK, but we really need to do this first. priority and this is another priority and all of these things kind of come up and there's unexpected complexity because this is another thing about, you know, building something for a long time is, you know, as you grow, you get more teams, there's more interdependencies.
So all of a sudden you realize like, okay, we built, you know, the like that part, but we need this other part and we need to connect these dots. And so, yeah, certain things definitely take longer than you expect them to. And then other things, A year ago, you were asking about how these innovation weeks come together. And, you know, probably like a month and a half, two months before that.
Last year's birthday week, we're like, we have to get AI running on Cloudflare. And we'd announced Constellation previously, which was the original iteration of it. And that was where we got a lot of feedback that was like, this is cool, but I need GPUs. And we went from idea to...
ai running on gpus all over our edge in less than two months so that's yeah kind of thing where you would expect it to take longer and you know that's the magic sometimes of having a deadline or forcing function is we're just like we just don't have the time for it to take a long time so it's just going to happen real fast
Yeah, I mean, I think that's one of the best parts about working on something that takes a long time. Like you said, you get less than a year than you thought, but then in five years, you're just like, what the hell? And I think it's because...
stuff compounds and it's like so hard to really imagine the power of that it's i'm sure you think about you know where you guys were like seven years ago and like how difficult it was to do anything and then now you're just like what were we doing back then like why are we going so slowly you can't even like understand how much things have compounded how much easier it's gotten uh like for us when we went from the v2 to v3
that was after several years of like, kind of like rotating around the space and like understanding it, like not really getting the primitives exactly right. Then I think we got the primitives right. And just in the past six months, like we've done so much more than we've basically done in the entire history of the company. Cause everything now is set up for us to go do. And this stuff is,
just takes a long time um yeah i was surprised at the gpu stuff too i'm like uh like i was kind of surprised how fast all the ai stuff came together um given that like crazy i mean all the is has been really crazy so um but yeah i mean that must be the result of like all the work before then
Yeah, I think to your point about having primitives, right, that's where, I mean, what allows us to sometimes get the stuff running so quickly is we build on workers. A lot of our new announcements are built on workers. So at the very least, getting to a POC can happen really quickly. Workers, AI...
is actually largely runs on something that we talked about during birthday week, runs on containers. This is a big conversation that we've been having internally about, is that something that we expose to customers or not? The fact that we've had this platform for some time and we originally built it for Cloudflare runs a browser isolation product as a part of our zero trust suite.
And you need a container for that. So we've had that platform for a while. And the fact that, you know, that was something that we had and we had a team that was ready to go and build on it made it that much, you know, the cold start was much faster on that.
Yeah, I mean, I mean, yeah, that's something we should talk about. I think that was maybe one of the more I mean, you guys said it for Friday, so, you know, is what the containers platform stuff was one of the bigger surprises, I guess. Because it now puts Cloudflare. In a completely different zone, I think for a while.
everyone was kind of able to understand it as this is like a new way of building things. And if you like agree with this new way or that you want to commit to this new way, then you can go build with Cloudflare stuff. But then introduction of containers means almost any kind of way of building can work. So we did a similar shift to where we started really serverless focused.
And there is like a large growing number of very big, serious companies that build that way.
but it is an uphill battle and that you're trying to like change the habits or like here's like a new way of doing things here you go learn all this stuff and you agree with it and you go do it uh and we shifted to now like we've done that for a while so like there wasn't much to do there anymore and now we started to support container stuff more and it's a totally different experience it's like
There's no one to like explain anything. There's nothing to explain. There's nothing to teach. It's just this is the way most the world operates and they just like use it and they get it. And it's much, much less of a battle. So I think it's going to be interesting to see when Cloudflare containers comes out. I think something similar will happen, obviously, on a much larger scale.
I'm curious to actually hear more from you about what your experience has been And, you know, to your point of on one hand, it unlocks so many things and I think really changes people's perception of what's possible. On the other hand, yeah, it was really interesting. You know, I think we were all wondering how that announcement was going to go and what people's takeaways were.
And the takeaway surprised me. A little bit, or maybe surprise is the wrong word. But yeah, I'm very curious to see what happens when we actually put this in people's hands. And so we'd love to hear about your experience of making that shift from serverless to containers. And did you see things that surprised you? Are people using it in the way that you expected?
Are people doing things that you're like, you know, this is... Not how I would advise you to do it, but knock yourself out.
I think the main it's more like if it just feels different. So what feels different about it is I suddenly feel like I'm talking to a thousand X more people. And I think that's just what happens when you break into a bigger market. I realized before I felt like I was talking to a lot of people. There's a lot of people that build stuff in a serverless way.
I didn't really feel like, you know, it was necessarily small. But the reality is the people that want to deploy containers, it's just so much bigger and it's hard to even fathom. And we're seeing people from other communities that have nothing to do with JavaScript. They're putting stuff together. Someone's working on a Laravel setup with SST and our container stuff.
And that's just so outside the range of... anything that we generally think about so yeah the surprise has just been like wow there's like a lot of people out there i feel like now we're finally seeing like everyone and our whole thing has always been progressive disclosure which is we don't want you to have to like
learn so much up front or like change it we want to give you an incremental path to using our stuff uh and over time we need to design a progressive disclosure thing where like you start to learn the things that are useful to know and you start to get more advanced and you start to use other things that you necessarily didn't know coming in uh and i kind of see containers in the same way i think based on probably some of your team and based on that blog post i think probably similar where
Doing a container option means it's a no-brainer for a lot of people. Existing workload, existing way of doing things, they can just go use it.
But now they also have the ability to pull in these other serverless things here and there, which even for me, I'm working on something right now that is going to be container-based, but I need to be able to spin up a lot of headless browsers, and that makes way more sense to do in a serverless way.
So like being able to mix the two and giving people the ability to do that, I think, you know, over time we've just realized like the more options we give people, like it's just, but we'll start to do funny things with it that you don't, that you don't imagine. I think you'll, you'll see that too.
I think you'll see people mixing containers and durable objects and all that stuff together in really unexpected ways.
I mean, people mixing all that stuff together, I think, is the really interesting part. And that's what we hope people will take away from this is that, yeah, you now have another tool in your arsenal that allows you to complete the picture or, you know, yeah, you don't have to... force everything into the same model.
But at the same time, you know, the thing that I thought was kind of interesting is I really think people see the word containers and stop reading. Which is not too surprising, I suppose, in some ways. There are literally some things like... I saw questions when we announced this that are like, does this run on Firecracker? And it, there's literally a subset that's like that.
It's a very long detailed article, yeah.
Having been doing this for seven years, one thing that you realize is distributed computing is really hard. I mean, computing actually period is really hard, right? That's the challenge that most people have with applications. And if containers were perfect, then what would we even be doing in the first place?
Because, you know, people have Kubernetes, like, why wouldn't they just happily be running that? And I think the reality is, There are a lot of challenges that come with that, both from if you have a distributed audience standpoint, which most people do, from a scaling standpoint and how many resources it takes.
And, you know, I think that that's kind of been the longtime argument about serverless versus not is... Yes, you can run your $5 VPS, but most things scale beyond that. And then do you want to spend your team's time on that or do you want to offload that to someone else? And a lot of what I think historically has been done with serverless and where we saw the opportunity with workers was that
If you try to build serverless on top of containers, you end up with something that's not really all that serverless because it's an abstraction and it gets leaked really quickly. You're still dealing with regions. You're still dealing with concurrency. You're still paying for idle time. And workers flipped all of that on its head. And so...
if you're looking for something really serverless, workers is just, it's going to be better. And yeah, we've had so many conversations with people over the years where they would be like, why doesn't Cloudflare just let me run a container? And you would start asking them questions of, you know, okay, well, what would you do if we let you do that?
And they'd be like, okay, well, I have a Go application and I just want to run it on Cloudflare's Edge. It's like, okay, do you want to run it in all 200 data centers? And sometimes they would be like, yeah. And you're like, okay, do you want to pay for it to be running 200 times? Yeah.
And they're like, no, you know, or like the other trade off that people didn't realize is like, okay, well, now our request can hit any one of these 250 data centers and. it's going to cold start. So you got closer to the user, but your net performance is negative, not positive, right? Is that what you want? And people are like, oh, haven't thought about that. No, that's not what I want.
And so that's where I really think the devil is going to be in the details and in the use cases and Yeah, I think a lot of the interesting possibilities are going to be where we can abstract away a lot of that by combining it with things like durable objects, where in most cases, I guarantee you actually don't need a container running in 250 data centers.
But we have the ability to be really smart about where we place your container so that it is always warm, so that it is closer to user's You can do a lot of things that, you know, where serverless was traditionally kind of associated with stateless. All of a sudden you can maintain state and do things like, you know, build games and all that kind of stuff.
So, and, you know, again, if I think about some of our use cases for it. Um, something like CICD, I think is a great example where it's something that's slightly longer running. It's not on the critical path of, it's not something that you expect to be instantaneous, right?
So we can kind of schedule it wherever we can actually in this instance, make it really cheap because, uh, you know, during certain times of day, we have data centers that don't receive as much traffic. We can schedule it over there and pass the cost savings onto you. So I think in some ways, uh,
it's a lot more actually interesting and opens up more stuff than people think yeah it's funny when i told frank uh about the cloudflare container stuff the first thing he asked was uh oh can we run cicd there because we we have a cicd product and it's like it's weird like ci is such a commodity but it's annoyingly difficult in a lot of ways because uh there's just these funny traffic patterns with it too like sometimes because of uh the panda bot or whatever like the github thing
It has a schedule and it'll like end up triggering a bunch of our customers CI stuff. So we'll just get like a burst of a bunch of jobs randomly and being able to like one CI needs to run anything because people can do literally anything in their CI. You have no idea what they're going to put in there. So you can only use compute. That's like very flexible. But like.
making sure you have capacity to handle like peak versus you know what your average load is uh especially when it's not predictable yeah it sucks we actually ended up deciding for like the next version the more newer version of our product we run ci inside the end users accounts at least for aws because just trying to manage like a centralized ci service it's just so much so much work um but yeah something like this would be super helpful because there's not a lot of options for
on-demand containers that you can just that can kind of like run anything yeah there's yeah so i think there's gonna be a lot of use cases that it opens up and again i agree with you that doing all these cool like next-gen stuff does require rethinking how you build things and adopting a new architecture and a lot of that is service related i think a lot of discussion never even gets that far it gets stuck on like really basic things like pricing or like do you need scale it's kind of the tired things we see all the time but
That's not really the reason I even shifted there. It's funny. I think the thing you said about Kubernetes reminded me of this. So for a long time, anytime I built something, I would just get generic compute and I would throw Kubernetes on it and then schedule everything through there. That's how I built everything. And what was nice about it was I never have to learn anything new.
Like I knew that I can throw anywhere, whether it's AWS, GCP, whatever. Then I joined this team. And for the first time in my career, because I'd always worked either on my own companies or early stage,
had a dedicated devops person so it wasn't my job to figure that out cool totally fine with that uh and i was looking at what he was doing and he was using uh aws ecs fargate which is like a managed container offering so you're not doing any of the work just you just have a container and you pay for it and i asked him like this is like three times as expensive why are we like why are we doing this versus the thing that i've always done
And he said something to me that like totally changed the way that I looked at everything. And I felt so stupid for not thinking about it this way. He was like, so at the time we were actually building out a payment processing system. So we had to go through all of the compliance stuff for it. He was like,
Yeah, you're talking, he was like, you're basically describing like a $15 a month per container versus like a $45 a month container situation. In the $45 a month container situation, we get to go tell all the compliance people that we don't have access to the servers. And that is literally worth like tens of thousands of dollars.
So like not have to be like, here's how we're hardening, like that whole thing. Right. Um, so that, like you said, like comparing the five hour VPS thing, it's not really about the little cost of the compute. It's about.
security questionnaires you get when you go what talk to like a big enterprise company it's about like the compliance stuff being able to say like i don't have access to any of that stuff like eliminates so many of the questions so much of the burden and that it felt silly for me to like bring up like a 30 difference you know in the face of that um yeah ever since then that's when i start to open my mind to okay like if i do more serverless stuff
All of these problems I've been dealing with my whole career suddenly got. And there's a lot of things like that that go away that I don't think get talked about a lot.
Yeah. You touched on two things that I think are really interesting, maybe three. But the first is, you know, that setting up Kubernetes and scheduling everything was kind of how you'd always done stuff. And I think that in that way, You know, people always ask, like, what's the best DX? And I think the best DX is the thing that you're used to.
That's the thing that's always going to – it's not an objective thing, right? It's actually – it's going to be the thing that feels the fastest to you. And that's something that we think about a lot is, like, how do we – we have this actually pretty different programming paradigm.
And so how do we expose it in a way that does feel familiar so that you don't feel like you're relearning everything because – When you do have to do that, it does feel like this really big hurdle of like, Okay, I know that my way of doing this is inefficient and requires like five for loops and that doesn't make sense, but at least I'll get it done and I can call it a day.
So, yeah, you know, that's why we've been working on the NodeCompat stuff and I think that containers is going to go a really long way there of just giving people something to hold on to that feels familiar and kind of get started from that. But, yeah, I think that the second thing that's really interesting here is, to your point, it did...
All of that compliance stuff didn't matter to you when you were building as an individual. And at the same time, you know, that difference between like $15 and $45 felt really significant to you. And I feel like that's where things sometimes get lost in the Twitter conversation too. You know, I think it's great that there's such a big community of people. You know, I think we're both that way.
We just like to build stuff in our spare time. And it's a great way to learn new technologies. But what you need for, you know, your business, blog or your side project or whatever is so different from the requirements of a company. And that's where, yeah, knock yourself out setting up Docker on a $5 VPS. It just doesn't matter.
But when you're running in production and you're doing this as a part of a multi-million, multi-billion dollar business where literally every second of latency correlates with revenue, where, yeah, you have compliance requirements to meet, where, you know, people's time really adds up.
If you think about, you know, the cost of an engineer, $100,000 even starts to look pretty small compared to, you know, all of the alternatives. Yeah.
Yeah, and the other thing is, and you talked about this a little bit earlier, when your team size grows, it's just always a negative thing. There's benefits to it, but there's always a permanent negative thing when there's just more people involved.
So sometimes for me, I'm like, even if it's cheaper to hire people to do it ourselves, the fact that we can keep our team smaller, that's worth some kind of crazy intangible amount. Yeah, I saw this at my last company where... we did something where it made us a lot more money. We like in-house something that made this again, this payment thing, it made us like a crazy amount of money.
And like on paper, it like made total sense. And nobody would really think like, of course, like everyone would just, of course we should do that. Like we're making all this money. but it like grew the organization. Like we needed to bring in people on the engineering side, which meant we needed more people in the recruiting side, which meant we need people, more people on the HR side.
And then, then we needed like more data engineers to handle this stuff. And again, it was all extremely profitable to do, but it kind of permanently changed the culture of the company. And I always wonder, it's not like we would have been sitting there doing nothing. We would have been working on something else instead.
And I kind of wonder if we could have worked on something that was a lot more differentiated. Um, But yeah, it's hard to see past the direct, this is technically more expensive than if we in-house it. There's a lot of these intangible things that come up.
Yeah, I think that's going to become even more interesting in some ways in the AI age where... the productivity of a single person can go so much further in some ways, right?
And so, yeah, I do wonder if especially, yeah, like as a small startup, as a team of three, you can go so far so fast just bootstrapping things and building in a serverless way that, yeah, I wonder if there's going to be like even more and more scrutiny over, you know, build versus buy and what's the, you know, liability of growing a team.
Yeah, and this stuff is so complicated because there are ways to... If you have a very traditional application and then you move it to the cloud, you're going to look at what you're getting and you're going to be like, this sucks, this is a scam. It kind of is. You're just paying for regular servers, but like... 10x more.
And your applications are very traditional applications that can literally run anywhere. So why would you pick to run in the cloud? So I totally get that perspective. I also get the perspective that not all serverless is the same. There's a lot of stuff that has... been serverless and is like really out in the world as like, this is a service way to do it.
That doesn't have a lot of benefits of being serverless in terms of like the cost being basically super cheap or the capabilities being really powerful. And so people have an impression of what serverless is that isn't necessarily like
accurate representation of it so um yeah i still really even though with our shift to like working on stuff outside of serverless and and i think same for you for all of you uh i still like believe in that model and there's just so many problems that now that i'm doing it i'm building something that's in a container way again i'm like i just saw this problem an hour it's like okay this thing can get like a burst of requests
And then that's just going to overwhelm the whole system and nothing's going to work. And like I'm remembering that was like 80 percent of my work. People always talk about scale where they're like, oh, yeah, you got to keep scaling things up and up and up. A lot of times my work was like, how do I slow shit down so it doesn't overwhelm the system?
And shifting the serverless like made that possible.
like kind of go away and now i'm like back dealing with it again so yeah it's what you said like people are used to the reality is you can get good at anything like whatever technology you're using there's stuff that is particularly good at there's stuff that's bad at you spend enough time with it you're gonna get good at mitigating bad parts and taking advantage of the good parts so
That's true of everything. So I get why if you're used to something, it doesn't feel that bad because you've gotten good at using it. But you can get good at using a new thing too.
I think also to your point about if you're managing a bunch of servers and then you look at something like the cloud and you're like, oh, that's so much more expensive and I'm still managing Servers, if you've been doing that and you have a team that has that in-house expertise, then it makes sense to keep doing that in some way.
But if I think back to your comment about the product is growing, we need to hire more people in order to keep up with customer demand and all of that. The availability of talent, I think, is something that everyone in this industry struggles with. And so then I think the question also becomes, you know, I think your odds of being able to hire someone that's really capable and
I think that there's this aspect of, sorry, I'm struggling to think of the word, but barrier to entry effectively. You can get a really smart engineer that if you give them the tools that they need vis-a-vis serverless can get running just slinging code versus I think finding people
that if we take the example of literally managing physical servers, there are just far, far fewer people in the world that are capable to do that. So again, like calibrating that, who is your next hire going to be? And, you know, do you want to spend a bunch of money trying to get talent that's very uniquely specialized and knows how to do all this stuff? Or, you know, like,
Today, you can get a really great college grad, especially, you know, with the help of AI that is going to be able to, you know, get running so much faster if, you know, all they have to do is figure out how to run JavaScript.
Yeah, and I think the other interesting thing, I mean, you have this perspective, I have this perspective too. We just see a lot of companies building in all sorts of ways. So in a lot of discussions, people describe like these horrible scenarios of like, it just makes it seem like this stuff can never work out. But like every day we see, you know, thousands of companies that,
are on their own making their own like rational decisions for whatever makes sense for them choosing to build in a certain way and it just works uh and it works particularly well again building it in all sorts of forms so you just kind of get less i guess for me it's just like the way people this stuff is described you feel so exaggerated because like
they're just people doing this every single day and they like never, they're never like introspecting. Like, is this right? Like, is this great? Like there's not thinking about that stuff. They're like, it's just never comes up. Yeah. It's hard to like get a sense for that when you just are seeing the discussions around these things. How's New York?
I know it's getting fall time, which was my wife's favorite time of year, but I think it's a lot of people's favorite time of year.
It's definitely my favorite. Every day, I'm just savoring. There are, I think, like 16 weeks in the year, maybe a little less than that, where New York is just perfect. And they're spread across fall and spring. And every day of those, I don't want to leave.
Yeah. It's funny because I'm such a little like... I'm like a little baby with this stuff where I'm like the moment it gets a little cold, all I can think about is that means winter is coming. So I could just never enjoy fall because to me it was like a impending doom. It was like a sign impending. Exactly. That's all I could think about. I'm like, it's coming. It's coming again.
It's going to be horrible. But most people, I think, know how to enjoy it. People love fall in New York.
I think the way that you feel about winter is how I feel about summer. Where winter, I'm like, I can handle, I can be cozy versus I don't deal with the heat very well. So that's how I feel in the spring where I'm like, oh God, the temperatures are rising. It's coming.
Yeah. My favorite part of New York was I had this like crystal clear image in my head because I feel like it literally happens this way every single year. You'll be the winter and all of a sudden one day you leave. It's like nice. You leave work and it feels like some AI generated scene because it's like everyone's walking around happy, eating ice cream.
There's like people skateboarding and everyone's like it's like 4 p.m. and everyone's kind of already out of work. And so the most idyllic scene you could possibly imagine has happened every single year. And I like clearly can remember that.
I know exactly what you're talking about. And yeah, it's like, it's always somehow a Friday. That's the first magical day. And yeah, everyone is just having the best time of their lives. It's so cute.
Yeah, I know. It's a, yeah, I think that's a nice thing about having seasons. There's like these rhythms to it. Now that I've been in Miami for two years, I guess maybe two and a half years or so, It's funny because it feels very different. It just feels like one continuous sense of existence. There are seasons, but I don't really perceive them that much. And it feels totally different.
It almost feels like time is frozen. And there's things I like about it, but I can see how it might drive people kind of crazy.
No, I know exactly what you mean. I used to live in San Francisco. And it's funny, I was just there last week for for birthday week. And every time I visit now, it is actually it's New York Falls weather, but year round, which for me is actually I'm like, that's my perfect climate. I know some people have Miami or San Diego where it's sunny.
Yeah.
For me, I'm like, this is ideal. And sometimes I'm like, I can't believe I left this weather behind. But there is a nice thing about the rhythm where you're always looking forward to something. And yeah, there's these changes and you get different produce and everything.
yeah oh yeah the produce stuff is i mean yeah i think we still have like seasonal eating here too but uh is it weird to eat i i don't know pumpkin soup when it's like 85 90 degrees okay this is what i was gonna bring up this is the this is the weirdest thing for me uh because i grew up in the northeast from the ages of 5 to 30 basically within the northeast
And there's this permanent thing in my head of it's Christmas time. It's snowy. People put up Christmas trees. You put up wreaths and it feels a certain way in Miami. Guess what? They do all the same stuff. They put up wreaths, but it's like 90 degrees and it just looks like I think aesthetically it just looks it just feels wrong.
I think innately you look at like all those green like Christmassy plants and stuff and it's
just does not make sense and i wish people here would stop like my wife says uh the northeast has like a monopoly on like holidays basically like the whole country like orients towards like the northeast version of of the holiday where i'm like this makes no sense like this this looks off like even seems to christmas like like the red banners and all that, it just feels totally wrong.
And I feel like we should just come up with our own thing that's totally different. It just feels off.
I mean, you think of October and Halloween and you think of Christmas, you imagine Meg Ryan in New York in the appropriate weather. If you transplant Meg Ryan into Miami, it makes no sense.
I know, it's horrible. And then like in fall, people were like, where Flano? And I'm like, you are melting in that. Like, that's just not the right thing. But yeah, it just somehow the Northeast has that influence over.
I think over more than just the rest of the US. So my partner is from Australia. And so I never really thought about it, but their Christmas is in the summer. And so, yeah, you know, like Christmas barbecue. Yeah.
Yeah, exactly. I was going to bring up the same thing. I met someone from South Africa who worked with me and he was like, oh yeah, Christmas, like watermelon and barbecue. And I'm like, what are you talking about? That just sounds insane. But yeah, it's all flipped.
Santa and watermelon and barbecue? Get out of here.
just like I to be honest I still like refuse like my brain like understands it's real like the same way I understand that dinosaurs are real but like I'm just like no they can't have really like it just I don't it doesn't really make sense in my head I feel like I need to go there and be there to understand it and that means like our summer holiday I guess our summer holidays are I guess like specific to the U.S.
like uh 4th of July but yeah then their summer holidays are like cold
really weird i feel like this is not talked about that much like half of the world is on the opposite schedule and we never talk about it yeah we're very northern hemisphere centric yeah we're like it's getting so cold and meanwhile the other side of the world it's like oh yeah we're looking forward to summer well it's wild you're gonna be out here in a Next month. So November 18th.
That's right.
Yeah. I haven't been cold in a very long time. Like physically, I haven't felt the feeling of cold.
Do you even own a jacket?
So my experience in New York was so I lived there for 10 years. The first five or six years, I was just like, I'm just going to layer up. I'm just going to layer. And that's how I'm going to deal with the cold. I'm going to keep layering. And it just never layered well. And I was basically like freezing the whole time. And then finally, I was like, you know what? I'm just going to give in.
I'm buying one of these crazy expensive, like meant for traversing. I don't know, like the North Pole, South Pole, whatever, jackets.
Like Antarctica or something.
Yeah, exactly. So I went to that company, Arcteryx. I don't even know how you pronounce it. And what's cool about their stuff is that it's pretty lightweight. I bought it and I was like, oh, this is why I was so miserable the whole time. I can literally wear a t-shirt and put that thing on and I'm good. Like I'm good for the whole winter. It's such a good jacket. So I have that somewhere.
I don't think November is cold enough to justify that, but I'm probably going to wear it just because I don't want to feel cold at all.
You might as well. I don't know. It does get pretty chilly, especially in the mornings. What you just made me think of with the layering is the funny thing about
san francisco and the thing that i do cherish about new york summers is that it's warm in the evenings versus in san francisco you can have the most beautiful sunny day and then the second the sun sets if you didn't think ahead and bring layers you just go to instant freezing oh yeah i hate that i'm so bad with that stuff i'm having to be in new york too in certain times of the year when it was like that uh yeah
it's uh it's just like i just didn't know basically most of the time i lived there maybe like seven percent of time i lived there i just didn't know how to dress properly for the weather so i got i got that jacket and i also got one of those uh those crazy hats that made with the raccoon fur look like that and they're called heaters you you have to bring that out here for sure
I'm going to bring everything. That's like that was like my setup. And once I had that, I was like, I am never going to be cold again. It's a solved problem. Like you have the technology to be warm. You've got to commit to it. Also, it's like so worth it, too, that the jacket was like,
thousand dollars which seems like a lot but i use it for five years it still looks brand new and it basically you know like saves my ass for half the year so yeah i i was thinking there must be some comparison there to be made with like servers and serverless and like amortizing it over the cost part yeah yeah a lot of life is just biting the bullet and just doing it my yeah like
Liz refused to make this shift because the reality is, is like the thing I hated about the winter is it's just so bulky. Like everyone's just so bulky. Like everyone's like wearing all this stuff and like you go inside and everyone's like taking off layers and like piling it on something.
Just instant overstimulation.
Yeah, exactly. And so for her, she's like, none of those jackets are like cute. So for her, she was just like very much resisting that. And she was just always cold. So that's a challenge.
I think that's why fall is the supreme season. So today the weather is like you can wear a sweater outside, but you don't need a jacket, which I feel like there's maybe actually like two weeks in the year that are like that.
everyone gets to see your outfit you look cute because it's not hidden under you know your tundra jacket so yeah yeah and then in the winter i would just i would literally i've gone like 10 days without leaving my apartment and stuff i would just hide inside so it's not the i think just was not the environment for me but you know i do miss it i think uh i'm excited to be back there and November.
I was back. Oh, yeah. When I met you for the first time, that was my first time back. And you might have experiences like this, too. I was walking around and I just like these flood of memories come back. Like I walk down streets and I like remember all these like random things from years ago. And it was a really crazy experience. That was my first time going somewhere where I used to live.
And it sounds like you've lived in a bunch of places. So you might have. I mean, when you were when you were in San Francisco, you probably had a similar experience. That was my first time. And I was like, felt really crazy. Like I never had something like that.
It's such a weird feeling. I do experience it every time I'm back in SF, especially because the last couple of years of me being in San Francisco were during the pandemic. And so there are the years that I spent going to the office. And so, and then there was all this time that I didn't.
And now every time I'm in San Francisco and I go to the San Francisco office, especially in Soma, in that area, it feels like I have so many memories, but also I was such a different person
person then and yeah you just kind of become that person again in a way or yeah yep exactly it's a very out of body feeling yeah it was also kind of overwhelming i was like after two days i was just like i kind of want to leave like this is uh this is too much because that's the only place i'd ever lived you know so it was like a lot of very intense uh but i am excited to to be back in November.
I think you'll probably, you're coming to our, our event we're doing, right?
Yeah. Are you, are you going to drop a sneak peek of what, what you guys have planned or are you going to, are you going to manage to keep a secret?
I think what we're going to do is we're not going to say anything, uh, but we're going to be building stuff for it live two days before. So whoever joined is what you were saying before. Like if you're paying attention, you're going to know what we're doing. Um, but yeah,
I don't want to officially announce it because during the event, I want to start off with a series of misdirections where it seems like I'm just going to tell you one of them. I'm sure a bunch of people aren't listening to this. I want to start off the event and I want to thank everyone for coming.
And I want to be like, okay, today we're here to discuss the caching differences between Next.js 14 and Next.js 15. I'm going to pull up a slide deck that has like 200 slides. I'm going to start like going into it.
so we want to like you know just throw people off but uh yeah i'm looking forward to it me too it's gonna be awesome yeah all right well thank you for joining today this is fun yeah we're all super excited about everything you're working i haven't dug into any of it yet but i'm gonna do that tomorrow today or tomorrow most likely um have a whole list of things to go through and i'll probably do it on stream again nice yeah so much of our team tuned into your last stream and we're all like
Oh boy, rough edges. Let's go fix them. So yeah, if you want to cover more for us, actually, we're doing an internal thing called Fixed Oberfest. So this month, we are very focused on our own observability and sanding down all of those rough edges and finding paper cuts and fixing them. So this is a perfect time to erase all those things.
Cool. All right. Well, thanks for joining.
Thanks for having me.
Yep.