Menu
Sign In Pricing Add Podcast
Podcast Image

The Changelog: Software Development, Open Source

Kaizen! Pipely goes BAM (Friends)

Fri, 28 Feb 2025

Description

It's Kaizen 18! Can you believe it? We discuss the recent Fly.io outage, some little features we've added since our last Kaizen, our new video-first production, and of course, catch up on all things Pipely! Oh, and Gerhard surprises us (once again). BAM!

Audio
Transcription

Chapter 1: What is Kaizen and how does it relate to Pipely?

15.24 - 33.78 Jared

Welcome to ChangeLog and friends, a weekly talk show about birthday presents. Thanks as always to our partners at fly.io, the public cloud built for developers who ship. Learn all about it at fly.io. Okay, let's Kaizen. Kaizen.

0

42.639 - 57.945 Adam

Well, friends, before the show, I'm here with my good friend David Shue over at Retool. Now, David, I've known about Retool for a very long time. You've been working with us for many, many years. And speaking of many, many years, Brex is one of your oldest customers. You've been in business almost seven years.

0

58.105 - 68.871 Adam

I think they've been a customer of yours for almost all those seven years to my knowledge. But share the story. What do you do for Brex? How does Brex leverage Retool? And why have they stayed with you all these years?

0

Chapter 2: How does Brex utilize Retool for its operations?

69.312 - 88.327 David Shu

So what's really interesting about Brex is that they are an extremely operational heavy company. And so for them, the quality of the internal tools is so important because you can imagine they have to deal with fraud. They have to deal with underwriting. They have to deal with so many problems, basically. They have a giant team internally, basically just using internal tools day in and day out.

0

88.347 - 102.491 David Shu

And so they have a very high bar for internal tools. And when they first started, we were in the same YC batch, actually. We were both at Winter 17. And they were, yeah, I think maybe customer number five or something like that for us. I think DoorDash was a little bit before them, but they were pretty early.

0

102.972 - 121.127 David Shu

And the problem they had was they had so many internal tools they needed to go and build, but not enough time or engineers to go build all of them. And even if they did have the timer engineers, they wanted their engineers focused on building external facing software, because that is what would drive the business forward. Breck's mobile app, for example, is awesome.

0

121.347 - 138.298 David Shu

The Breck's website, for example, is awesome. The expense flow, all really, you know, really great external facing software. So they wanted their engineers focused on that as opposed to building internal CRUD UIs. And so that's why they came to us. And it was awesome. Honestly, a wonderful partnership. It has been for seven, eight years now.

0

138.558 - 156.833 David Shu

Today, I think Brex has probably around a thousand Retool apps they use in production. I want to say every week, which is awesome. And their whole business effectively runs now on Retool. And we are so, so privileged to be a part of their journey. And to me, I think what's really cool about all this is that we've managed to allow them to move so fast.

157.313 - 173.989 David Shu

So whether it's launching new product lines, whether it's responding to customers faster, whatever it is, if they need an app for that, they can get an app for it in a day, which is a lot better than, you know, in six months or a year, for example, having to schlep through spreadsheets, etc. So I'm really, really proud of our partnership with Brex.

174.85 - 198.65 Adam

Okay, Retool is the best way to build, maintain, and deploy internal software, seamlessly connect to databases, build with elegant components, and customize with code, accelerate mundane tasks, and free up time for the work that really matters for you and your team. Learn more at retool.com. Start for free. Book a demo. Again, retool.com.

203.502 - 212.148 Jared

We are here to Kaizen, which means Gerhard Lazu is also here. What's up, man? In the house. Gerhard Lazu in the house. Yes. Welcome.

212.208 - 214.65 Host

Everything's up. Everything's up. That's right.

Chapter 3: What was the recent Fly.io outage about?

758.929 - 772.016 Jared

I noticed because I went to go share something and wanted to look at something and I don't know, it was down. Although I think I already knew that because maybe you posted it. I don't know. But I definitely just went to the website and 503 or whatever.

0

772.536 - 794.484 Host

So for anyone, if you're wondering if changelog is down, go to status.changelog.com and you will see what is down, when it is down. So in this particular case, we had a previous incident. There's a bit of a red right there, and this is the origin. So the origin was down. And if you click on that, it takes you to the status and you can see like the whole history.

0

794.564 - 817.331 Host

So this is something that I do update whenever there's an issue like this, especially when it's a big one. We had a few small ones, like just like a few minutes, but those don't show up. But this one was significant. And February 16th, 10 a.m. Actually, it was before 10 a.m. So that was a Saturday. Saturday or Sunday? No, I think it was a Sunday. February 16th.

0

817.351 - 818.351 Jared

Yeah, I noticed on Sunday.

0

818.371 - 833.155 Host

Yeah, it was a Sunday. So that was half the Sunday looking at this. So what happened? Well, if you go to the discussion, 538, that's where all the links are. But basically, as it happened, it was a fly issue.

834.623 - 852.955 Host

and the fly.io issue it wasn't fly.io itself fly.io and i'm going to scroll down to that particular message has providers so in this case at one of the upstream networks so let's see let's see where is it i'm looking for there

854.056 - 873.907 Host

It was a far upstream issue, and I'm looking at a post from Kurt, the CEO of Fly, and he was saying that the failure was far upstream from us and a single point of network failure. So one of their vendors let them down, basically, and there's not much that they could do about it. So this is what happens when, you know,

874.807 - 897.634 Host

because we all depend on other systems and other systems there's always upstream systems you have internet the internet provider i'm sure you know has um transit links and peering links and all of that some of those can be down if you don't don't run to everything everything in this case they didn't have to have two of everything The switch went down and it took four hours for someone to fix it.

898.434 - 904.918 Host

And it was, I think, Sunday, very early morning on the East Coast, which just made anything.

Comments

There are no comments yet.

Please log in to write the first comment.