Charles and Valentino are joined by special guest Brian Vallelunga, CEO and co-founder of Doppler, a leading secrets management platform often described as "GitHub for secrets."Dive into an engaging conversation about best practices for managing sensitive information, such as API keys and encryption keys, and treating all environment-configured settings as secrets. Brian shares insights on using tools like AWS Secrets Manager, Docker, and Doppler’s seamless integration with popular development workflows, ensuring robust access control and audit logging.They discuss about the severe consequences of data breaches, compelling real-world scams, and the human cost of leaked data. Learn how Doppler helps developers avoid these risks while enhancing productivity. They also explore the intersection between developer efficiency and security, and emerging trends in secrets management like passwordless authentication.Tune in for valuable tips, personal stories, and a peek at the future of secrets management and developer security. Let’s get started!SocialsLinkedIn: Brian VallelungaBecome a supporter of this podcast: https://www.spreaker.com/podcast/ruby-rogues--6102073/support.
Hey folks, welcome back to another episode of the Ruby Rogues podcast. This week on our panel, we have Valentino Stoll. Hey now. I'm Charles Maxwood from Top End Devs. And this week we're here with Brian Vallelunga. Now, you are the CEO and co-founder of Doppler. And that's an app that helps manage secrets. You probably have a better pitch than that. So I'll let you explain what you do.
Hey, all. It's great to be here. Yeah, Doppler is a secrets manager designed to be your single source of truth for secrets across all your projects, environments, team members, and infrastructure. So you can kind of think about it like it's get up for secrets. Right.
And I think, I don't know, in the Ruby community, at least where I deal with people, and especially in Rails, right, we kind of understand what the secrets are at your passwords, maybe your, you know, you have your rails master key, which gives you access to the secrets in rails, but you also have like your, what is that? It's a token for your sessions. I can't remember what that's called anyway.
So you need that secret key and yeah, people can compromise your security if you don't have that secured. Are there things besides like passwords and API keys that you consider secrets?
Yeah. Outside of API keys, like a Stripe token, database URLs, encryption keys are typically the ones that we see the most. I also just have a general thought process around this of anything that's configured by the environment should be treated as a secret, so it should be treated like the most sensitive thing. So we recommend to our audience that
You should kind of treat your environment variables and your secrets all the same, right? Your port and feature flag should be treated as securely as your secrets. That way, developers don't need to make the choice of is this secure or is this not? They're always just doing the secure thing by default.
And then the trick is just make that that whole workflow and tooling as developer friendly as possible. So they wanted to go down that path.
Right. So I'm just going to kind of cover the.
basics of how i do this and then maybe you can tell us if there are other ways that you're seeing people do it but uh typically these kinds of things where i see them and i'm sure valentino's in kind of the same boat is they either show up in environment variables and or they show up in um in the rail secrets or and effectively what it is is it's a uh it's an encrypted file
And so then the encryption key is usually put into an environment variable or, you know, passed into the system that you're running it on on the other end in some way. And lately I've been doing my deployments with Kamal, which uses Docker. And so it kind of sets that all up when it sets up the container.
But I've always kind of wondered because you can ask the system if you can get into it, what the environment variables are. So I don't even know if that's the best practice anymore.
Yeah, it's a that's pretty much the two paths that we see, either some encrypted file somewhere or it's just directly available in the environment. I'm a pretty big fan of. if you're going to do the environment path or inject it through the environment, immediately do a cleanup of the environment before all your other dependencies could get loaded in.
So like basically load those secrets out of the environment, then clean them up from the environment, then load everything else. That way you can't have like a rogue dependency, go and capture all your secrets and then send it somewhere. The other path of like using a file that's encrypted is also a really good path too. So I think both are pretty good.
What are you doing, Valentino? Is it kind of the same?
We use a lot of AWS stuff to handle a lot of the... They have a secrets management deal in there. Yeah, and I think it may even be isolated from the main environments too and handled that way in an isolated environment. We use Docker as well, so It could be like an isolated container that then feeds back into the other container.
So are you I'm guessing you have your secrets stored on like AWS secrets manager and then those secrets are then pushed into the container.
Yeah, I think so. Something like that. It may even be one step further where it's in AWS and then at like run run time. It's like the secrets are loaded into the app's memory for the particular use case. I don't really know all of the details, to be honest. I'm not on the infrastructure team. Everyone's like, well, we have to rotate keys. And that's definitely an easy process for us.
But yeah, we have some command line that does it. It's not just like changing the environment variable.
Sounds like you guys are pretty well set up. I mean, that's the dream is to make Secret management is super easy and simple, but also very secure at the same time.
Yeah, that makes me curious because I think at one time we were using... I forget what HashRocket's product is. We ended up moving away from that. Yeah, I don't know. We were using some vendor for doing the secrets management. But yeah, having a command line or some kind of central place, a command line doesn't even need to...
be your UX of choice, but just having the centralized place where all of that access points happen, it definitely helps isolate and track, to be honest, who's touching what and changing it. I don't know if Doppler has those kind of features as far as tracking changes and getting the full logs and things like that, but that's definitely been super valuable.
Yeah, so I think Doppler kind of comes from a first principle standpoint about what does a good secrets manager look like? And for us, we have a couple of things that we deeply care about. First is that all your secrets are centralized in one place across your projects and environments.
And once they're all in one place, we can set up pretty robust access controls, either by users or by groups that are policy driven. And then from there, you get a complete audit story.
So, you know, every time someone has read a secret, written a secret, changed some settings on every device with their IP address, you get a whole picture about everything that's going around or going on around your secrets. And then the other key part is that it's deeply connected to their workflows and their infrastructure.
So on the workflow side and like local development, we want you to be able to edit your secrets side by side with your code. So you have a VS code extension. And so those secrets actually kind of look like a YAML file, but they're actually not. It's like something called a virtual file system. So it behaves like a file, but it's actually not as connected with Doppler in the back end.
So it's super secure. And then on the orchestration side for production is we will write into AWS Secrets Manager and Kubernetes or wherever it may be so that your infrastructure immediately gets those secrets when they change and restart your deployment so you don't have any downtime.
Yeah, I remember using a system like this way back in the day. I think you had to log into their interface or use their command line interface in order to edit your secrets. And I can't for the life of me remember what it was called. But I played with it and it was cool. It was just there were steps to setting it up and I'm lazy. And so I just went back to what I was doing.
I am too, for that matter. You got to make it like we have this theme at Doppler and it's quite simple. It's make vegetables taste like candy. Vegetables being the security, candy being the developer productivity. If you do not make it, if the tool does not make the developer's lives better, like you can't just say, hey, use this tool because it's more secure.
Like that doesn't fly because developers care about moving fast and shipping product. And so you have to aid them in that. Like in our case, we try to get them two hours a day, or sorry, two hours a week back in productivity. And so if you do that, if you give them good candy, then you'll get the vegetables too. Yeah, that makes sense.
I kind of want to dive in a little bit more on just why we protect the secrets. Because I think we all kind of intellectually know, but... I don't know when I get into stuff and I start, you know, dealing with the secrets, it's like, Oh, somebody got access to my, like my Stripe tokens. I mean, that, that could be really, really bad, really bad.
So I like to kind of bucket the risk in, in, or sorry. Yeah. Bucket the risk into two buckets. The first is like the risk to the company. And the second is the risk to people. And I think the company side is like pretty well understood, right? You're going to have a pretty, if you have a data breach, you're going to have a pretty big brand hits and trust hits.
You're going to probably have some unexpected spend because now you're investing in legal, PR, insurance. You may get sued as well because if you're not managing your secrets and the secrets are things that get leaked that leads to a data breach, that's just straight up negligence and the customers are in their rights to sue.
But I think the thing that's most of the time not talked about is like the cost to real people. At the end day, all these companies that we use, we're trusting them to store our data and keep our private data private. Imagine if like every YouTube video that you ever saw got out or every ride you ever took on an Uber, every transaction you ever made on a credit card all got out.
And I actually have a real personal story of how this has impacted me. Sadly, I've been part of a couple of data breaches now from some of the services I've used. And that came back and really bit me in an interesting way. So I just moved to Austin, Texas a couple of months ago from California.
And I convinced my mom to fly out with me and see Austin for herself when we were at this barbecue joint. And while we're eating food, I get a call from, uh, the Texas customs and borders. And they say, Hey, we found a package in your name that has, uh, illegal money and drugs in it. And we're, uh, federally investigating you now.
And this is like the phone call for me where I was like, Oh shit, my life has just ended. Like from the highest, really, really quickly. Um, and I was really scared and, but like also kind of like, this is odd, like never bought anything from Mexico before. So like, okay. Um, And the back of my head, I was like, this is a scam probably.
But then they started rattling off all this information about me that they knew. All the places I had lived in the past, locations I had been to, like, just so, like, a wealth of information about me that I was like, no one could possibly know this but the government. And so, like, I trusted them. I was like, okay, I'm actually being investigated here.
And obviously we get lawyers on the call as well. Like, this becomes a really scary ordeal for all of us. And it wasn't until about like an hour in that our lawyers picked up that it wasn't that it was a scam. And but during that hour, we also gave them a whole lot more information about me. Right. Because we thought we were being investigated and they were asking questions.
And so like this is and like, again, I'm a CEO of a cybersecurity company. I'm trained on this. Our employees are trained on this. Our legal team is trained on this. And they still got us for an hour. Right. I can imagine everyday Joe who's not trained could really get attacked here.
And like, I was lucky that all they did was get a little bit more information about us, but they were able to get that information because of other information that got breached. Right. That bought them the credibility to get new information.
But imagine my mom, my sister, or anyone else like that who's not gone through any of that training, does not have legal counsel immediately readily available to them that can jump on the call for this. Oh my gosh, I cannot imagine all the information that would be given up and all of a sudden their bank accounts are drained or their credit score is completely ruined or whatever it may be.
um it can get really terrifying really fast and that's why i come back to like whenever i see like a data breach and it's like x millions of people's data was just got out it's like wow like scammers are now like parading in the streets going like oh my gosh i just got a ton of data that's going to get me so much more data that can eventually get me to like the jackpot of all this money or or whatever they're trying to go after um and so there's real people's lives including mine that have been impacted by this and that is something that's like often not talked about at
And so when you're a developer and you're like, oh, should I put this in a new file or not? Or should I secure my secret? It's just one token. It's just a string. It's like, well, that string may impact a million people's lives someday. So to treat it right.
Right. Well, it reminds me your story a little bit of if you read Ghost in the Wires. Is it Kevin Mitnick? Yeah, Kevin Mitnick. Yeah, I've watched the movie about it, but I haven't read the book. And see, I didn't know there was a movie, but I read the book. But yeah, he talks about how effectively, yeah, he gets just a little bit of information about the next person in the chain, right?
And so he doesn't start with the analyst or whoever, right? He starts with the low-down person on the totem pole and then gets enough information to know who the next person is and then works his way up. it's yeah, it was, it was a fascinating read just from the, the sense of, yeah, you know, just, just anyway. Um, so yeah, so I can definitely see that.
Um, I think the other thing that I worry about is like, if I have a key to, I don't know, like chat GPT or something like that, somebody could run up my bill or, you know, somebody gets into my hosting account and, you know, all of a sudden I've got, you know, they, they breach my other stuff that way. And anyway,
No, I think you're totally right. I mean, there's two common things that we see all the time. One is like some source code goes public or some email gets leaked and that has a secret in it to like an AWS account. And you'll see one or two things happen immediately.
The first is their bill just like skyrockets in seconds because now they're using AWS for some other bot attack or for mining crypto or whatever it may be. That happens very, very commonly. And it's surprisingly how fast those secrets get found. I think there was an average stat that it's within three to five seconds of a repo going public.
If it has a secret in it, even in the history, it will get found and start getting exploited. So it's very quick. It's insanely quick. And then... The other side of it is now that they have access to your AWS, they can start scanning all your resources. Okay, what execution environments are set up? What databases are set up? Can I get access to those databases? How privileged is this credential?
And you start having this ripple effect where it's like, oh my gosh. Now, especially if it has like strong super admin permissions, it can start creating other credentials. And now you have this like cat and mouse game of like, how do I get out of the system? Could it lock me out of the system? Are there back doors it's placing in? So even if we remove it, it's still actually there, right?
it gets scary really quickly. Once they get into the system, it's very, very hard to get them out.
Yeah. That's always makes me wonder, like, cause there's two parts of this, right? Like we have the secrets and then there's like the whole encryption aspect of it, which is his own separate beast of the discussion.
Well, wait, when you say the encryption aspect of it, is that like how you're securing your secrets or is that like encryption keys?
Well, what you're, I mean, ultimately what you're using a lot of the secrets for, I would imagine is to encrypt something or, or even to, yeah, I mean, to encrypt stuff.
Yeah, you're either hiding stuff or accessing stuff is usually what you're doing.
Based on the data that we have from Doppler, we can't see customer secrets at all, but we can get an auto report of the names of the secrets they use if they give us that permission. And from the data we have, it's like 90% of the sensitive secrets that we put in Doppler and customers have an ability to say, this is sensitive or this is not, is actually like access tokens.
So it's not encryption tokens. Um, so it's like, if you get that Stripe key, you have it all. Like you have every customer's credit card and bank account in there. There's no extra layer of protection.
Yeah, that's all. I always want to, one of the biggest things that, uh, I always hope for when I'm like getting a new vendor, like access token or whatever it may be, is that they like have some kind of like, you know, whitelist stability with like either their domain or
like a domain key or some kind of identifier that can be like, you know, securely matched when the handshake is made using their service from whatever server that you end up using. In my experience, it's honestly very rare that that actually happens.
That makes a lot of sense, you saying that your customers mostly store access tokens because most vendors don't give you that whitelist ability and handshake process, which is very important. I think it causes a lot of the breaches when it's not there.
Agreed. I do think that the world is changing. It's just going to be very, very slow. So like you can see this in passwords today where we're going from passwords to pass keys. So anyone familiar with pass keys, it's basically like you can sign into a website with like your face or touch ID and you never had to put in a password. It's a passwordless world.
And the way that's working is through cryptography. You have like a fingerprint that leads to a cryptographic signature that can be used to authenticate yourself. And that that thing that's happening in passwords is now starting to happen in secrets with identity based authentication.
And so like the kind of picture that we can start to see happening very, very slowly in like Doppler's job is to like help move this along, help move the industry along is to have like this bridge of connection. So like on one side, you have like your AWS GCP execution environment, and that has an identity already today. And then you can pair that with Doppler.
And now you're authentic by the fact that you're in that EC2 instance or you are now automatically authenticated to Doppler. And then eventually Doppler will have the connection of now that you're authenticated with Doppler, you're automatically authenticated to Stripe. There's no more API key needed.
We are all cryptographically authorizing each other across this chain or this highway of authentication. And that's kind of like the dream state that we're going to be getting to over the next couple of years. It's going to take a lot of work of working with partners, but you don't have to convince Stripe, Twilio and so many other providers to enable this.
And we're going to try to make as easy as possible, but it's going to be it's going to be an uphill battle. But if we can do that, we're in a secret.
Is there an open standard that is following this process or no?
There is a little bit of an open standards, mostly between like AWS and X like Doppler or whatever it may be, but there is no open standard on, or even a standard altogether on the like secrets manager to third-party service like Stripe or Twilio. So I think our goal is we're gonna probably try to create an open standard
that both sides adhere to so that if you're anywhere in this chain, you can directly, you can just plug in very quickly. And I think we'll probably have to build SDKs for like every primary language and Kubernetes and a bunch of other infrastructure tooling as well. So just like works out of the box.
So, I mean, I find this all really fascinating, but I guess my question is, What kinds of mistakes might I be making now? Or, you know, what what kinds of things should I be considering doing now to protect these secrets? Because, yeah, you know, what you're talking about sounds brilliant, but it doesn't exist yet. Yeah.
And I'm just I'm a little worried that, OK, we're going to talk the rest of this time about futuristic stuff and then I'm going to be I'm going to miss them.
All right. So I have four questions that I think every person listening should ask themselves. And it's like basically a very simple sniff test to see if I have a problem. So to highlight what the problem is, it's called secret sprawl. And that is basically all your secrets are scattered across a bunch of different places.
And because of that, you have no way of controlling it or governing it, which is like one of the most important parts about protecting your secrets is being able to govern those secrets. And so the four questions are, can you confidently answer where all my secrets are?
Not just the ones that you know about in a secrets manager somewhere, but all the ones on developers' laptops, all the ones in Slack, in email, in your code somewhere. You got to have a complete picture of where all your secrets are. And you got to be able to make that guarantee that you're not going to miss one by not knowing about it. And I'll talk about how to solve that in a second.
The second is, do you know who has access to all of those secrets? And this gets really scary, especially if you don't have like a centralized system, because like if I have a file, I can just share it to another person and no one on the security team is going to know about that. Can I control who has accessed those secrets when they were accessed and by who? Right.
So do I have access controls and do I have an autolog associated with those access events? And then most importantly, when a data breach does happen, can I stop it very, very quickly? When I say quickly, I mean in the orders of seconds and minutes, not in the order of days and weeks or months.
And so if you cannot answer those four questions confidently, like to the point where your job would be on the line for it because it essentially is on the line for it, then you have a problem. And I can talk about in a sec what it looks like to have a solution there.
I don't have to go into like Doppler's features, but I can talk about like the fundamental properties you'd want to look for in any secrets manager, Doppler or someone else.
Yeah, I think that's important. It's funny because so I kind of have two levels of this one where I'm working for a client and the other one where I'm just working on my own stuff. And on my own stuff, I'm fairly confident that I can answer these questions because I'm basically the only person accessing or using them.
But then the question for me becomes, okay, well, how do I, how do I, you know, let's say I bring in another developer. How do I make sure that I know the answers to those questions as I bring them on board?
Yep. And I think once your surface area grows of team members, infrastructure, Devices, it gets pretty hard to answer those questions confidently, unless you have a dedicated system for it. So I'll be valuable to go into like, what actually does a dedicated solution look like? The first thing is you can centralize all your secrets in it.
So once you know where all your secrets are, you can confidently say, this is the place we're doing. We're not gonna store our secrets in code. We're not gonna send it over email or Slack or Microsoft Teams or wherever it may be. This is the one place you can get all your secrets. Now you have a complete landscape of your secrets.
The next thing you should be able to do in this system is you should be able to can set up access controls and audit logs around them. So you can say you have this develop full stack developers have access to the development environment for these sets of projects for the teams they're on. But the DevOps team has access to production.
So you can start getting access and you have a full audit picture of it. And obviously that audit picture then gets pushed into your auditing tools like Datadog, Simulogic, Amazon, wherever maybe. You should be able to have this infrastructure natively integrated with their developer workflows for local development.
That way developers aren't working around the system and then recreating that risk you're trying to avoid. And it should be natively integrated with your production and staging infrastructure so that When it is time to roll out a secret, you don't have this race condition of the code arriving before the secrets do. The secrets have to arrive first if the code is going to rely on those secrets.
And so then this kind of gives you this whole picture of I know that if I can see the secret in this project, in this environment, on the solution like the secrets manager, then it's guaranteed in my production infrastructure or it's guaranteed on my developers laptops in a secure way. And that basically gives you the ability to remediate secrets pretty quickly.
And so the last part of this puzzle is being able to rotate secrets. Rotation is the fancy way of saying, I'm going to swap out the locks on the door. And you got to do it in a way because if you just like immediately destroy the lock, in this case, you'll just bring yourself down in production. So you have to do it in a way that does not create any downtime.
And it's usually requiring a two key system. So like in a database, for example, you don't ever want to not be able to connect to the database. So you'd have two credentials to the database and you swap them and then you rotate one and then you swap again. And so you want a system that does all that. That should not be done by humans.
That should be done by a system that's running on a schedule where you have an immediate like I'm fucked button. I need to click this thing to get me out of a data breach. And then obviously it orchestrates to the rest of your infrastructure. And so those are the things that you should be looking for in a developer for secrets manager.
So I love this. You have this great secrets for all page on Doppler. I link to and it's really great. Kind of helps you quickly assess your risk exposure based on your team size. You go through kind of like a quick calculation. Oh, we have like this number of projects, estimate number of secrets per project. Then you have a certain number of environments, right? So it's a multiplier.
And then you have, you know, a multiplier over the number of team members too. And so those quickly add up and you make a great point, like, you know, the malicious actor, they only need one, right, of the thousands that adds up to. And so I'm curious, like, you know, from the, like the whole point kind of is like risk assessment and remediation, right? And like, do you find that, you know,
focusing on the that risk aspect is just as important as like kind of the secret storage management mechanisms yeah they go they go hand in hand for sure you need something that developers are going to want to use that is secure when they do use it and you want to think about that risk like
A good way that I kind of like talk about it when I'm talking internally with our employees is the game of offense and defense are very different games. The game of defense is build walls around your perimeter as high as possible, as strong as possible, with no cracks in the foundation, right?
And then the goal of offense of an attacker, of a hacker, is to find one crack in the foundation and hit it as hard as freaking possible that brings down the entire wall, right? They only need one weak link, right? And if secrets happen to be that weak link, well, then you're really giving them the keys to your digital kingdom. Um, that's extra bad.
I'd much rather be the weak link, be something far outer in the perimeter. Like, um, I don't know, some authentication bug or something like that, that, that impacts like one or two users, not your entire users, uh, user base. Um, And so, yeah, managing risk is incredibly important here.
That's the end goal that you're trying to accomplish is, like, make sure you can stay up and orchestrate, like, get all the DevOps and platform tooling you're looking for, and then also manage the risk associated with that.
So I'm curious, like, if you have any tools or, like, what would you do to kind of, like, visualize the, like,
know organizational structure in the in combination like it's more than just like secrets management it seems right uh like how do you how do you personally visualize like the security aspects surrounding the secrets and who accesses them and things like that uh to understand kind of what is happening and where the access points are yeah great question uh truly a great question um
We have tried really hard to build a model that maps very closely to what you'd have with code, so with GitHub or GitLab. So we have projects just like you have repos in GitHub and GitLab. And we have environments which are very similar to branches. And then we have user groups that map in with access control policies. And then we have integrations that connect to those environments.
And so you can have these like this bare bones, hierarchical structure of projects and environments. From there, you can assign users to them and you can assign infrastructure that connects with them. And it's very easy within Doppler to kind of see all that connect together in an extremely visual way. We even have to show kind of how far we've taken it.
When you go into a project, you actually have a Kanban board style user experience that shows all your different environments from development to staging to production, everything in between.
What inspired you to start Dropler? Where does this story start? I'm curious where your background is and how you're like, oh, we need a better secrets manager.
Yeah. It had nothing to do with secrets when it started. I was working on a crypto machine learning marketplace. All the buzzwords in one. It was literally every buzzword you could find jammed together. And man, that was a hard project. I worked on it for about eight months straight and it felt like pushing a massive boulder up a hill.
Every foot forward, I'd slip five to six feet back from exhaustion. It was just hard. And at one point I really got to my breaking point where I was like, I don't know if I can keep working on this thing. And so I decided to take a trip to Cancun.
mexico and the whole goal was like don't think about this project um actively do not and obviously i failed at that miserably uh i was thinking about it the entire time i was there and what i realized is that this pro i was never going to be able to get this project off the ground but that doesn't mean that i had to stop there um i think like there's a lot of people that like like follow like sports players and like they know all the stats of all these players i'm very much the same way with founders i i like i love learning about founder stories um
And I think Stuart Butterfield from Slack is like the best at failing upwards I've seen. Creates a video game out of that video game. The video game fails. Born out of that was Flickr. Comes back a second time. So I'm going to create a video game again. Born out of that was Slack, which we all use today. And so I was like, what can I learn?
from um from my experience with this crypto machine learning marketplace and managing secrets was just one of them that was like super painful in all these weird interesting ways um it was weird from a developer productivity side because i always felt like i was copying and pasting things in but sometimes the code would get into production faster than the secret force at an outage um i had a couple of developers that i was working with on this project i was like really painful to sync all these secrets with them like i would literally slack them and email them
the secrets and I like my heart would cringe every time I do that. Like, oh, that didn't feel right. But that's the only thing I have at my disposal to do. And there was actually one time where we dropped the crypto part of the marketplace because it wasn't working. We just did straight up machine learning marketplace using Stripe.
And I had the Stripe production key in staging and the staging one in prod. And it took us a month to figure out why we weren't doing any transactions. And it was because of that, because the Stripe testing key rejects all live credit cards. It only uses test credit cards. And we didn't realize that for a long time. And so I was like, okay, this is a real problem.
And so I come back from Cancun and I go to this dinner that Stripe was hosting that had a bunch of founders and developers at it. And we were all going around the room talking about what we were thinking about. And I just asked them openly, am I a shitty developer? Is the world broken? I just can't tell anymore. Are you guys having this problem too?
And like 60% of them were like, yes, this is a very big problem for us. And one woman in particular, like literally comes running up to me right afterwards. I thought she was going to trample me. And she says, I've had three outages this week. Have me a solution by Sunday. And I was like, no, no, no, no, no. Like I'm not even started yet. And this is a Wednesday. There's no way that's happening.
She goes, I don't give a fuck. Have it by Sunday. And I was like, okay, there are some interests here. People want it. Um, But we didn't obviously have it by Sunday. That never happened. But what we did is we started doing a lot of research and we found that individual developers are struggling because there wasn't tools made for developers.
Medium sized companies are struggling building their own things and large companies were buying hash core vaults, which really isn't meant to solve this problem and still struggling. So like everyone's struggling. And I was like, okay, we can build something that's developer first instead of just purely security first. And we got our first version working in three weeks.
And we got a set of customers right after that by just dragging them to Chipotle and saying, hey, I'll get you anything on the menu as long as I get a rant at you for about two hours. And that got us our first set of customers and off to the races we were.
Yeah. It's funny how, yeah, you wind up scratching an itch. That's not, how do I put it? It's not like the core thing you were trying to build.
Yeah. Anyway, but turned out to be a good business.
Yeah. It's, it's interesting because I've been kind of in that same space where I've been looking for a solution that does a whole bunch of things that I want for the podcasts. And I've just, you know, so I built little pieces of it here and there. And so I finally just started bolting it together so that I have the full solution and,
Yeah, it's interesting how many times you have, yeah, things come out of that.
Yeah, I would probably index and be like, okay, are other people that I know that also do podcast recording asking for this tool? Does it continuously get better and better the more I mount onto this hodgepodge of a product? And is that value going up exponentially, not linearly?
And if both those answers are correct or yes, and you still feel like there's nothing in the market that does anything close, you have a product. The question is, is that product 10X better than whatever's in the market? Because if it isn't 10X better or 10X different, there won't be enough of a differentiator to create a buying event, I found. Right.
So I'm curious, like, what are some common, like, preventable breaches that you've encountered? And like, How can developers avoid them, right?
Yeah, there's a couple that really come to mind. I mean, let's look at... He laughed like he's got some doozies of some stories. They're all public. I'll share one that isn't public, but I won't say the name of the company, and then I'll share a couple that are public. So Twitch, Toyota, and Twitter all kind of had the exact same breach happen. And they had secrets in code.
The code got leaked somehow, either by them opening up the repos themselves, or by hackers being able to get access to the GitHub account from an employee that got compromised. So like someone... got called and said, hey, I'm your boss and give me access to GitHub. And then the hacker got access to GitHub. And then from GitHub, they saw the code and the code had the secrets in it.
And boom, now the hackers are really off to the races. And that's like that happened multiple times and keeps happening because there wasn't a system in place to make sure secrets never entered code. They should never be in code.
Yeah. So I just want to just jump in here because essentially what you're telling us is just because your GitHub repo is private, does not mean that you can check your secrets into it and expect them to stay safe.
Absolutely. I mean, that is like the number one thing most attackers are going after is like GitHub because they're assuming you're going to be making this exact mistake. And it's very easy to impersonate a boss, especially in the days of AI. So these attacks are going to get more and more sophisticated and feel more and more real.
Like you're actually talking with your boss and all of a sudden, boom, you have access or they have access. And then I'll tell a story. I can't say the name of the company. I can promise you this. You have absolutely heard of the name of the company. You probably have used their product at some point in time. And they're very big. And this is before they were customer of ours.
They had a data breach where they hired a malicious actor in the company. So there was a rogue agent that they had hired. That rogue agent stole all their credentials or all their secrets, about 2000 of them. They detected a bit late in the game, obviously exited that employee. But then the security team was tasked with rotating these secrets.
So again, swapping out the locks, making sure that all those secrets they stole aren't valid keys to the locks anymore. And it took their security team six months across three engineers to rotate all of these credentials. So six months.
That's not just like six months of only doing this and not doing any other projects and a major distraction, but also six months that the attackers got to steal data. Well, if data moves at gigabytes per second through AWS, They have an enormous amount of time to drain all the data, right? Like they had all the time in the world, basically.
Like when you're dealing in data breaches, you're dealing in seconds and minutes, not months. So months is an absolute insane amount of time that the attackers had to basically use all those stolen credentials to get anything they wanted out of the system or compromise the system in any way.
Um, and it was from what I, what we heard, it was quite a bit of a battle between the hackers and like, just because you wrote the key. Well, if they use that key to grant, to create new keys, now you're like, you have this cat and mouse game. Right. Um, and so it was quite a big challenge for them.
Uh, post Doppler, they, uh, have a secret station set up a whole level of orchestration with their infrastructure. And that same thing would have taken about like 30 seconds to a minute. They'd clicked a button and it'd been done. But, yeah, it was quite painful for them. And you can imagine all the users that were deeply impacted by that along the way. It's not fun.
I think that the surprising thing is like that I hear a lot is, oh, this won't happen to me because my company is really big and we had the best security team. Or I also hear the quite another inverse, which happens all the time of like, I'm a small startup. Hackers won't care about my company. Right.
And like you hear this from both from the opposite sides, like the stars are saying one thing that the big companies are saying the other. And the truth is hackers are just looking for any company that they can exploit. Does not matter big and small. They're looking for prices. Sometimes they hit a jackpot million dollar payout. Sometimes they get a 10K payout. Right.
And like our small a small attack or a big attack. But they're just looking to make an exploit. And so really every and that's why the number of data breaches are just rising exponentially every single year. Like if you look at the graph, it's terrifying and it's getting and it's getting cheaper and cheaper for them to do that because the techniques are getting more and more understood.
The AI is getting better and better at doing this. So it's a really big growing problem. So everyone is vulnerable. It's not just if you're a startup, you're insulated because of your scale. Or if you're a large company, you're invulnerable because of your scale. Like Microsoft got hacked. Nvidia got hacked. Samsung got hacked.
Like these are really, really large companies that you'd think would have their stuff together. And still, they got hacked. So it's on all of us. All of us are responsible for keeping these credentials safe so we keep our users safe.
Yeah, I think you make a great point. I mean, being able to respond quickly is definitely like high up on that list and being able to remediate that as fast as possible because it will happen. Like you kind of just like have to go with eventually something will happen and you have to like just stop it as soon as possible.
And in my experience, just having that ability to move quickly to rotate things around has been like the best thing defense in the long run, at least from minimizing damage. Right.
Agreed. Like when you're in a firefight, you shouldn't be strategizing how to fight the firefight. You should just be doing the strategizing. Should it happen before the firefight? Right. You just want to be in execution mode when that happens. You want the execution mode to be as quick as possible. Like you should click a button and it should be done 30 seconds later. And that's it.
Yeah, this just reminds me to a story. I forget what the company was, but some some like, you know, software engineer at a big company had a home Plex server and the Plex server was outdated and it gave like, uh, you know, access to their home network and on their home network was a work laptop that they then were able to gain access to because it was on a shared network.
Uh, and then from there they then hopped to remote systems. Uh, and so it's like, uh, you know, the, you're right. It's getting much more like, uh,
Oh, yeah. Like there's some interesting ones that I've heard of recently of like printers, like old printers that are connected to Wi-Fi getting exploited or like everyone has all these IOT devices in their home that are connected to like the home hub system for Apple or Alexa or whatever. And those are internet connected too.
So like a very small device can then basically get access to the wifi network, which then gets access to every device on that network. And thus the attack starts. So there's a lot of exploits outside that don't have to start with just your computer or just AWS.
So let's say that, I don't know, I'm cheap and I don't want to use something like Doppler. I don't even know what you cost. Maybe it's like, ridiculously inexpensive, but let's just say that, you know, I, I decide for whatever reason I want to do this by myself. I mean, what, what does that look like?
Yeah. Um, well, first off Doppler is super cheap. We're free actually to start. Um, so it doesn't get cheaper than that. Um, uh, yeah, it's free for the first three years users. And then it's basically GitHub priced from there. Um, so if you, if you're comfortable paying for GitHub, you should be comfortable paying for Doppler. Um, uh,
But let's just say you don't want to do that for whatever reason. I think the most important thing you can do if you're not going to use any tooling at all or bare minimum tooling is use like a cloud secrets provider, like AWS Secrets Manager, GCP Secrets Manager, and so on, and then build some tooling around it so that you can fetch secrets locally in development without them touching disk.
So what you need to do is have some wrapper around your application that starts your application. So I'd imagine the workflow would be like fetch secrets from AWS Secrets Manager, inject them into the environment or write them to an encrypted file and then have the decryption key in the environment, either path works.
And then once the application starts, you clean up the environment and then load all your other dependencies. And if you did that, plus having your secrets in AWS Secrets Manager and that being the same process happening for production workloads, at least you've solved most of the risk, in my mind.
Also, when you're doing this and this where things get a little bit more of a burden because AWS managing privileges like IAM roles is quite complicated, in my opinion, is you're going to want to make sure that every individual developer and every developer set of infrastructure has their own identity.
And that way you have robust audit logs of like, okay, the secret was taken by X developer at this time or read by this time. And we, we can, we have this auditor of what happened. Um, so you don't want to basically create one master. I am credential and just share it with your entire team because then you lose a lot of your auditing story along with it. Um,
And you'll also probably want to set up, if you have a little bit of extra time to spend, an auditing story around reads. So every time a secret is downloaded from this endpoint, you're going to want to make sure that you have the right logging and auditing set up in AWS for that. And that you're actually looking at those logs and have analysis over those logs. Gotcha. Absolute bare minimum.
One other thing that I've seen, and I think that I saw this on the Kamal podcast, which is a deployment system that was written by David Heinemeyer Hansen. And anyway, it can pull secrets from like 1Password. Yeah. Have you seen people using 1Password for their secret Spanishmen?
Less and less. So they added something called one password secrets automation. Everything that I've heard from my community is that they are not actively investing in that product anymore. And it's kind of like Shelfware now. Shelfware, I've never heard that. It's there, it's available, you can use it, but it's not actively being invested in. It's on the shelf.
But the amount of people every year that I hear using it is less and less because, A, I think there's first-class products now that really work. And the amount of things that you want out of Secrets Manager is growing far faster than what they're adding to, which is at this pace, from my understanding, it's zero. But you can't do it.
I mean, like Secrets Manager or 1Password is end-to-end encrypted. So like you're getting a lot of security benefits from that. So I'm not going to say, hey, don't ever use it. If it push came to shove, use it. I just think there's like far better things that are also easier to use and far better priced out there that are readily available.
Valentino looks deep in thought.
Is Doppler part of the GitHub secret scanning part of the program? Yeah.
Yep. That's awesome. We are. If there is a Doppler token, a Doppler-issued token found on GitHub, GitHub will contact us immediately. We'll immediately revoke the token, notify you, and work with you to generate a new token. And we're going to be investing next year pretty aggressively in the secret scanning space as well. Like really having a full auto picture there.
Like I'd love to be able to say, hey, we found some secrets in your code. We've automatically revoked them, issued new ones. And those new secrets are now not in your code anymore. Like some really powerful workflows like that can just make sure that even though the tooling is set up so that you'd want to use it, if you actually don't use it and you fall back to bad habits, the system catches you.
Yeah, I'm curious on that. as far as the observability goes, what are the latest trends happening now for observing secret use and maybe detecting bad actors ahead of the breach?
Detection ahead of the breach, at least in the secret space, is almost zero from my understanding right now. There isn't a class of tooling yet for that. But on the scanning side, there's TruffleHawk, there's GitGuardian. GitHub is adding some stuff too. There is a space developing right now where basically you can scan. The goal is to scan the perimeter of your infrastructure, right?
So like Slack, Microsoft Teams, email, your code base.
all of that infrastructure um basically will be scanned in real time for for secrets um that's basically the extent of what we have today the problem is like you have to connect those dots right so like you found a secret what do you do next right especially if you're like an individual developer you don't control the secrets manager that's being used you don't control
Or if they're not using a secrets manager, what the flow is. And so like if you're in, if you like a classic example that I hear all the time is, oh, we added a secret scanner and the secret scanner found all of our ENV files. But the company has decided we use ENV files.
So like now it's a moot point because we know we've always known secrets were in code because they're in ENV files, but there's nothing to get us to the next step of like getting it out of code. So now all you're doing is surfacing a problem that we can't solve. So I think that's really where like we're trying to help move the industry is connecting the dots.
You found a secret and you can immediately take an action or an action is taken on your behalf based on a policy to clean up the mess.
I'm going to have to go back and listen to this and then go through all my stuff. Run your checklist. Yeah. Do you have a checklist, Brian, of like, you know, at least run through this for your projects or your team or do you publish anything around that?
Yeah, we have a couple blog posts that highlight basic secrets management practices. Outside of the stuff I already shared today around working with his cloud provider, I would strongly recommend having a secret scanner. So TruffleHogsfree, use it. It's open source. Or GitGuardian, both are partners of ours.
The other big thing is outside of secrets, I would have a vulnerability scanner, really important on all your major dependency streams. So like Docker images, Ruby gems, if you're using Node, NPM, but basically any dependency you have, you should have vulnerability scanning on it. Super fricking important.
because it's so easy these days for like one dependency that nobody's cared about for 10 years to just get bought for like a couple hundred bucks and then someone pushes an update for it um and all the other dependencies haven't frozen on that older version and so now all these dependencies get that update which means you get the update and that update has malicious code in it and that's like a very common like paradigm for attack these days
Uh, so a freeze your, um, uh, your, uh, your dependencies, meaning like I'm fixed at this version and no amount of them pushing a new update will, will change that I'm on this version. And this version has been like, uh, guaranteed good. No, uh, no vulnerabilities in it.
And that at least can help prevent any issues where like a vulnerable code gets pushed up and you guys immediately get attacked from it. What you really want is you want all your dependencies frozen. And then when a new exploit happens, you go, oh, that was kind of cool. We are using that package, but our version doesn't have that problem. So we're good.
And then you just wait for the fix to come out and then you can do another refresher, another update.
Cool. Anything else that you guys want to dive into? Um, so give us kind of the, the what? Two minute pitch on, on Doppler. Just, just to let people know what it does and how it makes your life better.
Yeah. So, uh, At the end of the day, Doppler makes your life better in a couple of ways. One, it's going to add two hours of productivity per developer per week. So in a way, it kind of pays for itself. On top of that, it's going to make you far more secure from an organization standpoint, managing your secrets. You can sign up today. by creating accounts for free.
And then if you're a much, much larger org, you can talk to our solutions engineering team who can help figure out how this integrates into your complex infrastructure. We have a whole host of great companies that are using us. So you're not alone from Accenture, Crumble Cookies, Puma, Children's Cancer Institute, StockX. travel trip advisors. So there's a number of great companies.
You won't be alone in this. And we manage about 30 billion secrets a month right now. So like at quite large scale. And I think the key thing here, if you decide not to use Doppler is at least manage your secrets. Like at the end day, I think we're all tired of data breaches happening because they keep impacting us and they impact your customers.
And so if you're not going to use Doppler, that's totally fine by me. You can even use a competitor, just use a secrets manager, protect those secrets.
I'm a little gratified you mentioned Crumble Cookies, their local company. Anyway, let's go ahead and do our picks, and then we'll wrap up. Now, I don't know if you had a chance to listen to other episodes of the show, but we just shout out about stuff that we're playing, doing, or that we like. Valentino, do you want to start us off?
Sure. Uh, yeah. I mean, the first thing that comes to mind is the dark net diaries podcast, which is just like an incredible resource of deep diving, uh, exploits.
Uh, and just like people are just so creative out there and, uh, nefarious or not, it's just like wild to see what, uh, what kind of attack vectors there are and how people are using, uh, and exploiting your behavior, uh, which can be patterned. Right. Uh, So I'd recommend if you're interested in any of that kind of stuff, it's, it's an awesome show.
Next to that, I've been following a lot of the rails world fallout, which is just like, I definitely missed out. I don't want it to go to conference. Yeah. There's so much, so much great stuff that I'm just watching from afar come out of that. And it's really exciting. Uh, so I guess just watch the, uh, YouTube channel for updates of, uh, new talks that are coming out of there.
Um, and then the, the last pick I have is, uh, Landon Gray. Uh, he's in the Ruby AI space and, uh, he basically made Streamlit, uh, for Ruby, uh, which I'm really excited to, uh, mess around with. And if you're not familiar with Streamlit, it's kind of like the, uh, you know,
I don't know if it's JavaScript or it's some other language, but it's what a lot of the, you know, machine learning enthusiasts will use to just quickly get up a chat and, you know, fine tune or use LLMs in creative ways. And Langchain has kind of like promoted it for a long time for just quickly iterating through AI products development.
And it's been really incredible just to like quickly get up and running on something new. And so I'm excited to see that come to Ruby in a very like Ruby way. So it looks very much like a, you know, a rack app or something like that, where you can just run it and it has a chat that you can use. And I'm really excited to see that develop more.
Cool. I'm going to throw in my picks. So the first one I always do a board game pick I'm going to pick, keep the heroes out. So the kind of the design of the game and stuff is kind of like a dungeon for like D&D, except simplified. And you're not playing the heroes, you're playing the monsters. And so each player gets to play a different set of monsters and they have different abilities.
And so the way that you play is you take your turn by, you know, playing your cards. So you have a deck of cards. And so it's a little bit of a deck building game and you can do all kinds of stuff. You can move, you can create artifacts. You can do stuff with the artifacts, depending on which room you're in and what actions are available to you. And anyway, you can also spawn new monsters and,
You know, again, depending on what actions you can take. And so then after you do all your stuff, then what happens is then you spawn heroes. And so you draw the card and it tells you what rooms to put what kinds of heroes in. The heroes do their special abilities. And then they'll either attack monsters or try to open the chests.
in the room, or if they can't do either of those two things, then they'll move one step closer to the vault. And the chests are numbered. So if it's a one, then it takes one activated hero to open it. If it's a two, it takes two. You kind of get the idea. So anyway, that's essentially the game. There are a couple of other nuances to it. And of course,
it has a series of dungeons that you can play through. And so, you know, there are different rules for the different dungeons as far as how they work. The way you win is you exhaust the hero pile twice. And yeah, the way you lose is, if the heroes open the chest in the vault, which is the four, uh, chest. So anyway, uh, really fun game takes about, uh, what, 45 minutes ish to play.
Um, the artwork on it is awesome. The different types of monsters are fun. Um, and so, yeah, I've played it. It plays one to four players. I've played it with three and four players. And anyway, it's, it's a fun game. Um, Yeah, so I'm going to pick that. And then one movie I've been wanting to watch for a while, mostly because I just heard it was really good, is Hillbilly Elegy.
Now, I recognize that this is the memoir by J.D. Vance, who's currently Donald Trump's running mate. So politics aside... Right. And I try not to get into it too much on this show. Long time listeners know where I stand on all this stuff, but I don't want to make the show about politics. But the story is amazing.
And, you know, the fact that, you know, this kid can come from basically the kind of place and lifestyle that he did and then become. you know, wind up going to Yale law school and then, you know, becoming us Senator. I mean, I just find that incredibly inspiring and, you know, at the end of the day, then, you know, how you feel about him now, whatever it it's really well done.
It has Glenn close and Amy Adams in it. Um, I haven't read the book, so I, I don't know how closely they stuck to it, but, um, Anyway, there were just several, I think, life lessons that anybody can benefit from. From the standpoint of, hey, look, if you really want these things out of your life, then you can go have them if you're willing to do what it takes.
And so, you know, from that standpoint, yeah. I really liked the movie. There was something else I was going to pick and I just... Oh yeah, it was the Mike Rowe film. And this one is on the Angel Studios app. I think it's called Something to Stand For. And this is more, it's almost a documentary. So if you've listened to the rest of the story by Paul Harvey, it's kind of based on that.
that's also the premise of Mike's podcast, at least when he started it. Now he just interviews people that are interesting to him. But what he would do is he would tell the, tell the story without telling you who it was about. Right. And so at the end, then he, you know, there's the big reveal. And, and so the, the, this person was, you know, Neil Armstrong or whatever. Right.
And, you know, and so that's the rest of the story is kind of the idea. And so he tells a bunch of those from a U S history from moments that, in some way shaped the direction of the country. And anyway, I really, really enjoyed that too. Uh, so I'll put a link to that in the comments as well.
Um, but yeah, and then the last pick I have, so Valentino mentioned rails world, um, and the stuff that came out of it. I haven't watched any of the talks. I haven't really dived into what came out of it, but I've been using Kamal to deploy my stuff and I've gotten really tired of fighting the traffic, um, reverse proxy load balancer.
And so I wanted to upgrade to Kamal 2 so I could get the Kamal proxy put in and make my life a little simpler because it looked like it would make my life a little simpler. And so I did the upgrade and I kind of screwed it up and then fixed it. So it took a little longer and a little more work than I thought it would. Essentially what I messed up was the directions are a little bit
The documentation on Kamal is just not awesome. I'll just put it that way. But anyway, what it does is it tells you to upgrade by upgrading to Kamal 1.9. There's 1.9.1 now, but I think it tells you to do 1.9.0. Anyway, you want to make sure that you can deploy with 1.9, and then you can upgrade to 2.1 and 1.9.
do the Kamal upgrade and then it goes in and it stops traffic and it starts the Kamal proxy and it, you know, does the let's encrypt and all of that stuff for you, um, without having to know all of the arcane magic that you need to know in order to get traffic to do all that for you. Um, and so I'm, anyway, I'm pretty excited about it. It's, it's been pretty awesome so far.
So I'm going to pick that, um, I got a couple of things I still need to figure out on it because I want to do multi-tenancy and allow people to pick their own domains. And so then I want it to do the let's encrypt stuff. And so I'm trying to figure out if I can make Kamal do that for me or Kamal proxy do that for me without having to have it in my configuration.
But for now it's, it's deploying great. So I've been pretty happy with that. Brian, what are your picks? Yeah, those are some good picks. Hard to compete with that.
I think I'm a huge Star Wars fan, so I'll just shout out Star Wars Clone Wars. I feel like not a lot of people know about it, but it's between Episode 2 and 3, and it's so freaking good. I'm a guy that usually doesn't like animated stuff, and it's just blown me away. So if you're a big Star Wars fan, there's a lot of seasons, and it's insanely good, and it's basically like Dave Filoni at his best.
Outside of that, really big fan of the Perplexity app. It's basically replaced any Google searches I do now. For anyone who doesn't know what Perplexity is, it's Google search plus ChatGPT. So ask it a question, and it just gives you the knowledge back. It doesn't just give you a bunch of links. And just like a Google search, it's updated in real time.
So if something comes out yesterday or earlier today, it'll be informed on that. And it is just an incredible job at answering really in-depth questions. Like you can ask it very, very complicated questions that have a lot of nuance that have a lot of requirements on real time data.
Like, I don't know, what's the compare the market cap of Nvidia to its revenue and then compare that to Microsoft's and figure out which one is a better stock to invest in. Like you can ask that question. I'll give you a pretty concrete answer with like charts and graphs included.
So very impressed with like I think that's like a pretty big step up and the ability to like get knowledge very, very quickly. and then the other one is John Coogan I think he does a great it's a YouTube channel that he that's out there and he just does an incredible job of like talking about, I think, tricky topics.
Um, so, uh, like what's happening with Taiwan and chips or, and your old, uh, prime defense, like he just does a good job of telling the story of some event or some company really, really well. And I feel like it's extremely fact-based and not like bias-based. Um, so I, I generally go to him when I'm looking for like really high impact news, uh, breakdowns. Um,
And then if you're also a Star Wars fan, I'll just end with this again because, again, huge Star Wars nerd here. There's a board game called Star Wars Rebellion. It's played with two players. One guy gets to play as the Empire, the other as the Rebellion, and you guys battle it off, and it's quite fun.
It's a very asymmetric game, but I think that's what makes it fun because Star Wars is very asymmetric. The Empire had a lot more resources than the Rebels. But if you like Star Wars, both fun wars and that board game will do you a lot of good.
Awesome. I don't think I've played this one. It's it's pretty highly ranked on board game geek. It has a weight of 3.74, which means it's pretty complicated game.
Oh yeah.
It took us eight hours to figure out. Yeah. It says two to four players and the play time is like three to four hours. And I'm assuming that's if you know how to play.
Yeah. I'd say me and my brother, we play, it took us eight hours, the first game to get it. The second game, we actually did it across two days just because like we were, we were kind of like very slow with every decision and be really methodical. And now we'll play like in an hour and a half. So like once you get comfortable ramps up pretty quickly.
Yeah. But yeah. So the, the rank on it overall is nine. So it's the ninth most popular game on board game geek.
Hey, there we go.
Yeah. And it's, uh, it's the fifth thematic, which means that it has some, you know, some theme to it, like star Wars. Um, so yeah. So, um, next time I'm at a board game convention, maybe I'll see if they have it, if I can play it anyway, I'm, I'm a board game nut. I really enjoy that stuff. So it's a lot of fun. Yeah. So, Brian, if people want to connect with you online, how do they find you?
Brian at Doppler.com for email. Balalunga Brian for LinkedIn and Twitter. Cool.
All right. Well, thanks for coming.
Yeah.
Thank you. All right. Till next time, folks. Max out.