The Changelog: Software Development, Open Source
The world of embedded systems (Interview)
Wed, 15 Jan 2025
Elecia White, host of Embedded.fm and author of Making Embedded Systems, joins us to discuss all things embedded systems. We discuss programming non-computers, open source resources for embedded, self-driving cars, embedded system like the GoPro, Traeger smokers, and even birthday cards. According to Elecia, embedded is going everywhere.
What's up, friends? Welcome back. This is The Change Log. Yes, we feature the hackers, the leaders, and those building embedded systems. Today, we're joined by Alicia White. She runs the awesome podcast Embedded.fm. She's also an embedded software developer. She joins us to discuss...
Programming non-computers, open source resources for Embedded, mentoring in the world of Embedded, self-driving cars, Embedded systems like GoPro, Traeger smokers, and even that musical birthday card you have. According to Alicia, Embedded is going everywhere. A massive thank you to our friends and our partners over at fly.io.
Fly is the public cloud built for developers like you, like me, developers who ship. And you can learn more and deploy your app in five minutes at fly.io. Okay, let's talk embedded. Well, friends, before the show, I am here with a new friend of mine, Scott Dietzen, CEO of Augment Code. I'm excited about this.
Augment taps into your team's collective knowledge, your code base, your documentation, your dependencies. It is the most context-aware developer AI, so you won't just code faster. You'll also build smarter. It's an ask-me-anything-for-your-code. It's your deep-thinking buddy. It's your Stan Flo antidote. Okay, Scott. So for the foreseeable future, AI-assisted is here to stay.
It's just a matter of getting the AI to be a better assistant. And in particular, I want help on the thinking part, not necessarily the coding part. Can you speak to the thinking problem versus the coding problem and the potential false dichotomy there?
A couple of different points to make. You know, AIs have gotten good at making incremental changes, at least when they understand customer software. So first and the biggest limitation that these AIs have today, They really don't understand anything about your code base.
If you take GitHub Copilot, for example, it's like a fresh college graduate understands some programming languages and algorithms, but doesn't understand what you're trying to do. And as a result of that, something like two thirds of the community on average drops off of the product, especially the expert developers. Augment is different.
We use retrieval augmented generation to deeply mine the knowledge that's inherent inside your code base. So we are a copilot that is an expert and that can help you navigate the code base, help you find issues and fix them and resolve them over time much more quickly than you can trying to tutor up a novice on your software.
So you're often compared to GitHub Copilot. I got to imagine that you have a hot take. What's your hot take on GitHub Copilot?
I think it was a great 1.0 product, and I think they've done a huge service in promoting AI. But I think the game has changed. We have moved from AIs that are new college graduates to, in effect, AIs that are now among the best developers in your code base. And that difference is a profound one for software engineering in particular.
You know, if you're writing a new application from scratch, you want a web page that'll play tic-tac-toe, piece of cake to crank that out. But if you're looking at, you know, a tens of millions of line code base, like many of our customers, Lemonade is one of them. I mean, 10 million line monorepo as they move engineers inside and around that code base and hire new engineers.
Just the workload on senior developers to mentor engineers. people into areas of the code base they're not familiar with is hugely painful. An AI that knows the answer and is available seven by 24, you don't have to interrupt anybody and can help coach you through whatever you're trying to work on is hugely empowering to an engineer working on unfamiliar code.
Very cool. Well, friends, Augment Code is developer AI that uses deep understanding of your large code base and how you build software to deliver personalized code suggestions and insights. A good next step is to go to AugmentCode.com. That's A-U-G-M-E-N-T-C-O-D-E.com. Request a free trial, contact sales, or if you're an open source project, Augment is free to you to use.
Learn more at augmentcode.com. That's A-U-G-M-E-N-T-C-O-D-E.com, augmentcode.com.
All right, we are here with Alicia White from Embedded FM. Alicia, welcome to The Change Log.
Hi, it's nice to be here.
Nice to have you. See, you are on the other side of the microphone, so to speak, as a longtime podcaster. I think you've done 491 episodes, if the most recent one on your homepage is in fact the highest number.
I think it's 492 today.
Oh, so it must be a new one out today. What's the most recent?
Octopus Army.
Oh, I see it right there. You know what? I scrolled right past it. Two oscillators oscillating other oscillators, which is a great title.
Like you, we like our titles to be amusing.
I love that. Yes. That is my brand. That's your brand.
I guess so. I never really intended it, but that's what it is.
Well, you started a podcast called Embedded. So at a certain point, you decided to embrace it, at least.
Actually, the podcast was originally called Making Embedded Systems, which is the title of my book. But then everybody just kept calling it Embedded. And we're like, fine, OK. And then NPR came and said, we want to have Embedded, too. And we're like, no, it's ours. But actually, we just said, can we please interview an actual NPR person? Because that would be so exciting. So that's what we did.
Cool.
What was their approach to ask you about having embedded? What does that mean? What's the backstory? Yeah.
Well, okay, picture this. You are just sitting there and doing your work, coding away, and you get an email from NPR and it's like real NPR people and they say, we'd like to talk to you. And you're like, oh my God, it's my big moment. Finally, engineers are going to get to be on NPR and we're going to get to talk about how exciting this all is.
And they're just like, yeah, so we're starting a show called Embedded, and we just wanted to know if you were going to have a problem with that. And we're like, we didn't trademark anything. You've got to be kidding me. Anyway, yes, it was a huge disappointment. And yet they did let us interview Kelly McEvers.
And she had to keep leaving because things were happening in other parts of the world that were actually important.
Like during your interview with her?
Yes, yes. She was in the NPR studio and she was like, yeah, I have to go. People are doing things. So now we're Embedded or Embedded FM.
So they also have Embedded.
Yes.
There's two Embedded podcasts out there.
There are. One is about how engineering is pretty cool and how technology is interesting and how you can make cool devices that interact with the world. And the other is about all the depressing stuff that happens in the world.
Well, that's a sad dichotomy.
I hope ours is the former.
Yeah. I would stop at one. I was interested in that first one. Yeah. It sounded cool to me. You know, what's interesting, I think, what I think embedded is, is this idea of creating, I would say more so in the last, I don't know, maybe since the pie, the raspberry pie came out, like this idea of like, I can do things myself. I can take this small thing,
that is just a single board computer and I can learn to program or I can add servos. I can add, I can 3d print now because those are more ubiquitous and I can start to make my own thing in the world. And the system I put on there is not necessarily connected to the internet or the rest of the world, but it might be through wifi or whatever. Is that, is that kind of what you mean by embedded?
Like this kind of world where you have standalone devices that are not really tethered via wire to the network, so to speak, like how do you define embedded?
I like to say that I write software for things that aren't computers.
Okay.
That pushes the definition onto what is a computer.
Yeah. You got my interest. Go.
If you're working on servers, then developing an iPhone app is embedded. But if you've been doing 8-bit microcontrollers, anything with an RTOS doesn't really feel embedded. So it is definitely a loose definition, but the whole software for things that aren't computers... Actually makes you think about your microwave and your car and your toys if you have electronic toys for kids.
There's a lot there. And it's huge. I mean, there's all the sensors that get interconnected for science and for industry. And there's ICU devices. And I mean, I did actually work on educational toys and that was so fun. So it's a big field.
That's a good explanation of it. I was thinking a very simple one from my recent Christmas holiday. We got my oldest son some e-drums. So he has a drum kit. As you can imagine, an eight-year-old going on nine, banging on some drums is cool. But it's also, as a parent, not always the appropriate time to bang on drums. And we want to encourage the obvious curiosity that is drumming.
However, we also think that e-kits are cool because they're less loud, and they're cool because they have this controller. It's not a computer, but it is a computing device because it has MIDI controllers and all these different sounds for drums. So you can choose the metal kit or the e-rock kit. So it's got all these different drum kits in there.
It's taking the sensing input from the microphone on the e-drum into the controller, into your headphones or your amplifier or to your tracking device. And it's turning that into MIDI control sound.
I didn't think about that being, you know, like, I guess I did, but you know, I didn't think about until this moment that that's kind of like not a computer, but also kind of a computer that somebody wrote software for. Exactly. And maybe, maybe I could even write my own software for it.
If there was a way to, is that part of what you do is embedded is like you start to find ways to add to the non-computer computer.
Well, I mean, I do it as a career. So most of the time people bring me ideas and I help them build their systems. I may have taken apart a few toys.
For sure.
BB-8 was one of my favorite ones when it first came out. Figuring out how to get its IMU data and maybe control it so that it did weird things. That was fun. So I don't do a lot of hacking and reverse engineering except for my own amusement. But the same tools are necessary. The way that I reverse engineer a device is about basically debugging it without the code.
So what are some non-computers that you've programmed?
Well, I have a... autonomous underwater vehicle that I worked on last year. And it wanders around its area. For example, one is in the Monterey Bay in California. And it checks in with all of the various sensors that are on the seafloor. I also have worked on some of the sensors on the seafloor, measuring when we get big rainstorms, it changes how the mud And that is really hard to monitor.
So they dump a couple of smart rocks down there that have to live for years and years. So battery is a big issue. And it's not like I can go down there and push buttons and say, update the firmware.
Smart rocks. Is that what you call them? Smart rocks?
Smart rocks.
Wow. Okay. Never heard of that.
And then when something happens, when there's an event that causes a mudslide or some weird currents happen, it monitors and it reports back to the little vehicle that wanders around the bay and picks up all the data from all the sensors. So, I mean, those aren't entirely my projects. Those are working with the Monterey Bay Aquarium Research Institute, which is awesome.
And they do lots of science and ocean-based technology. Let's see. What else?
You can't top that one, can you?
Well, I worked on a gunshot location system where you sprinkle sensors, microphones, around the city and automatically call the police when there's a gunshot. Oh, that's cool. That was really cool technology and had some amazing applications. I'm kind of glad I left.
I mean, there's the whole, I mean, I am comfortable with the level of privacy associated with it, but I am less comfortable with how the company started being run. Children's toys, that was fun. I got to take some of my toys to a kindergarten, just volunteering. And one of them that I hated, it was a karaoke. And it was the most annoying toy. It would just sing constantly.
And there was a voice changer that did not make anybody sound better. And I thought it was a dumb toy. I was way more interested in this other toy that helped kids learn to write. And I would take the toy to the kindergarten and kids would play with them and I'd watch and it was fun. And then one of the boys who was kind of behind asked if he could take the karaoke home.
And I was like, good riddance. And a week later, he had caught up to his class. He had been really behind. But because singing was the way that worked for him, it totally blew my mind that Something so different than what I could have stood worked. I've worked on DNA scanners twice now, actually. Once pre-1995, 98. Well, anyway, once when DNA scanners were really cool and very hard.
And more recently when you can just make them on your desk. That progress has been just phenomenal and amazing ever. What else have I worked on? I worked on inertial measurement units, which used to be the way, when I worked on them, that I would make anybody I was sitting next to in a plane just stop talking to me. Oh, yeah, I work on inertial measurement units.
I could have said, home insurance, are you interested? No, no, no. Inertial measurement units was so good. People were asleep by the end of the words. But I didn't tell them what I was doing on this airplane. I was going to a racetrack so that we could test them on high-speed vehicles. or airplanes, or any of the other places where inertial sensing becomes really interesting.
This was the day before Wiimotes. And so the whole idea of accelerometers and gyroscopes as being interfaces was very different then. Now you flip over your phone and it turns on, that's what you expect. But how does it even know? I think sensors are probably my favorite area of embedded systems right now.
You've done a lot of cool stuff. How did you get into this world?
By accident? I mean, isn't that how it usually happens?
Usually.
So in college, I took applied CS courses because I couldn't see the point of the theoretical ones and theoretical engineering because I didn't really like any of the actual engineering. And what that gave me was Fourier and signal processing, as well as the ability to develop products.
I went to HP and net servers, big then cloud computers, and kept getting lower and lower into the hardware, monitoring temperature, the things that we monitor now in cloud servers. We were just starting to put in temperature sensors and alerts when things were happening instead of just the bad old days of having computers that ran or they didn't or they caught on fire.
And you found that out by going in. And as I kept going lower and lower in the stack, eventually I wandered over to HP Labs and that was when we made DNA sensors, DNA scanners. And I didn't know what I was getting into. My perspective boss was like, you'll really enjoy this, I promise. And I was just like, HP Labs, that is the best location ever.
And the first time I moved a motor, because I told it to, not just because I powered on something, but the first time it was under my control. It was magical. It was like software can touch the world. Software can change physical things. It's not just zeros and ones. It's move this here, read what's actually happening. That physicality of embedded systems is what got me into it.
And that's what's kept me there. I really like... Not blinking lights. A lot of people get into it for blinking lights. A lot of the hobby things are blinking lights. And I appreciate it. I hate blinking lights. But I appreciate their enthusiasm. But for me, it's touching buttons and having things happen. So that's how I got into it.
But I found out you can actually write software that changes things. And not just other software.
That moment of discovery is the fun part, right? Like when you like get that, oh man, I had no idea that's what that really meant. You know, like when you kind of get that taste and it sparks this like perpetual curiosity, I would say, cause that's, I mean, mine is not as cool as yours.
I was writing CSS and my mom was like, and I was doing a blog cause I was far away from home and she was catching up with me and my life via a blog back and Gosh, 2003, I want to say, 2004. And she was like, you are good at this. You should keep doing it. And because my mom said that, I'm like, okay, this is kind of cool. I should probably keep doing this.
And that's kind of like where my curiosity in software development began was just like learning CSS. But that initial moment is so strong.
Teaching people, it's always amazing to get to see that moment or to see part of it. I like mentoring folks in hopes that I can be there when it hits.
On platforms like the web, it's so easy. You can get to that moment in minutes because you can load up their favorite web page and show them how to open up the dev tools and change the background color or something or delete a line and the title disappears. And you see that aha moment very quickly because And I know in embedded systems, a lot of times now it's the blinking lights, right?
It's the Arduino or whatever it is. And that moment of turning a light on, that's so cool. When you mentor, when you teach, what do you do? How do you get people to that moment? What do you show them? And how long does that take? Because it has historically been considerably more steps in embedded than on platforms like the web.
Yes. I mean, teaching now, there are some simulators. There's one called WOKWI, W-O-K-W-I. And it will simulate an Arduino system. And you can program it under C or Arduino or MicroPython, lots of interfaces. And it takes out that pain of wiring, getting the wiring wrong, blowing it up, all the things that can happen with embedded systems. It also takes away the pain of cross compiling.
But in the end, it's on the web. And I agree, if you want to see that coding can do more than just, it's not just typing, it's doing things, it's doing cool things. Changing web pages and interacting with simulators is totally the way to go. But yeah, it is sadly a little bit of a step function with some of the embedded systems. Trying to get from, I opened a box to I'm doing something.
If anybody wants to try it, like wants to try it today, we've said Arduino. Arduino is really great. It was the system developed for helping artists do installations. And so the whole system is very directed towards art and making things accessible that way. They don't expect you to have 20 years of sea experience. The other one is one of the micro Python boards.
That one, Python's a pretty easy language as far as languages go. It's not C, and it isn't compiled, so you can try stuff on the command line. And you get a micro Python board, you plug it in, and it acts like a thumb drive. You just take your Python file over, put it on the thumb drive, and then there are all these things like motor.run, and suddenly it runs.
And at that point, you do have to have things wired up. But there are a lot of kits for the MicroPython, and it's a good way to get started. And with MicroPython or CircuitPython, which are very similar things, you can get started right away making bigger and bigger things. Arduino, I love Arduino. There's a lot of Arduinos. There's a lot of good things you can do with Arduino.
But it's an old system, and it's really expensive for what it is. So I would recommend MicroPython for the future.
I pulled up the walkway and I'm a couple layers deep while you're talking. And I'm on the walkway for VS Code and I'm like, wow, this is so cool that you can, I mean, you can do it in the web too, but like a lot of people are really, they're happy in VS Code. I'm not unhappy in VS Code. It's not my favorite, but it's certainly a quite usable code editor.
And I think it's a great platform for a lot of people. Yeah. The fact that you can have another tab next to your code in VS Code and it's Walkway and you're kind of like programming in software what could be future hardware and learning, that's so cool.
I've personally been intimidated by, I suppose, hardware coding like this or things that do stuff that isn't software because I don't know how, and I suppose everything's find-outable, right? The word find-outable. But I just haven't found out or had the time to find out. So the burden and the hurdles to start in the way that I want to start seem so high.
This Walkway thing seems so cool that I can, I feel like it might be a bit more accessible now.
It's a great system. Uri Shakad is the developer and he's very responsive and they've got a Discord. It's really fun. And we have an episode about it if anybody wants to look up that. And I feel here for how do I get started without breaking everything. Adafruit and SparkFun are two companies that are for hobbyist electronics, and they both have Boards you plug together.
You're not going to solder these. I mean, you can, but the goal for them is you just click these things together and you don't have to worry about that. Am I providing enough power? Am I going to cause a small fire if I leave this on overnight? So Adafruit and SparkFun are the places to go to find these things out. They both have a really great tutorial system as well.
So if you're thinking about getting into it, It's a good place to go. On the other hand, Wakui lets you do things like have a robot army that you control from an Arduino because you can put in 500 motors and Wakui doesn't care. If you do that in real life, you're talking about a lot of power and figuring these things out. But if you do it on Wakui, it's all just really fun.
I like the idea that, one, it is software, so you can scale, just like you said, which is a great thing. But at the same time, it's like, well, this isn't real. It's software. But the moment I felt comfortable literally buying or spending my cash on some hardware and some wires and a kit or whatever it might be,
I can take the thing that I've been playing with, iterating on and get like every other software developer is doing. And I'm learning the software development skills. But now once I've kind of gotten to the point where I've got my really good version that I want to really make in the real world, I can translate that to, it seems like at least translate that to real world.
Now I've written the software for it. Now all I have to do is just pair it up with real hardware versus this, this software version.
Yes. Yes, indeed.
That's so cool. That's dope.
I had no idea it existed. How big is the chasm between... I'm able to use a simulator in Python or maybe it's even in C. Or I'm even able to get something onto some device and have it do stuff. And your world of I'm writing professional embedded software for autonomous vehicles that go underwater and sense things. How big is that chasm?
encouraging everybody to come to Embedded says, it's not that bad. You just read my book. It's all fine. The real me has to admit that over-the-air updates are never going to be easy. Security is never going to be easy. Low battery.
All these things where you have, like in toys where you have to shave off pennies so you end up with a processor that should not be able to do this much, but you are packing in every piece of RAM, every cycle is spoken for. And that optimization, it's... People bring me their Arduino projects and they say, it works. All the features work most of the time.
and I look at it and see all of the things that isn't going to work. Usually there's like, I just want it to connect to the internet so I can get the data back. Like, okay, so you have to have a database for the data you want, the cloud data, the cloud data of interest. But you may also need to monitor the devices to make sure that they check in?
What if this is a medical device and you need the user to be using it? So if they aren't using it, you need to know. And then what if it's had some faults? What is the fail safe? Can you actually hurt someone? What firmware version is it on? And these are all cloud problems. But these are cloud problems that have to be solved with the device.
And as you scale from one or two or three devices in your house that you're playing with, that you're building, or even 10 devices in your tiny startup office into thousands or millions. I worked at Fitbit, and this was a huge problem as they scaled. Oh, I mean, orders of magnitude. That database became bigger and more people had to access it.
And yet you don't want someone to know where President Obama is right now because he tends to wear a Fitbit watch. That's the sort of data you don't want to get out. And yet you still need to know if every device is up to date and if their batteries are degrading more quickly because on the new firmware, Yeah, there's a lot. There really is a lot. It's not insurmountable.
And there are companies all the time that are trying to make it simpler. Do they succeed? Some of them. Some of them succeed pretty well, but it's never, sometimes they talk about it being Lego blocks. Like I used to have to develop a lot more stuff. But now I get my drivers from whatever repository I'm working. I get them from the chip vendor or from the RTOS vendor.
I don't have to write drivers anymore. And other pieces of code like IMUs, most of that code comes from somewhere else. I used to have to write it. I don't anymore. It's really nice. But there's still this... the shimming of it together to try to get it all to work together given the time constraints or whatever other limitation there is to the embedded system.
I guess that's another way that I have defined embedded systems is that they are purpose-built for what they do. And so they don't have a lot of extra resources.
Mm-hmm.
whether it's needing to live underwater for five years on one battery, or whether it's needing to live in your pocket overnight on one battery charge. They don't have extra stuff. And that causes a lot of problems when you need to do extra stuff.
Well, there's a sideways application of Parkinson's law here, which, you know, Parkinson's law is the one that the scope of a project will expand to match the time allowed for the project. Something like this. That's a bad paraphrase.
And I was just teaching my kids about budgeting this morning and was talking about how if you're not diligent that your lifestyle will expand to match the size of your income, right? Sometimes beyond. It's just kind of like a human nature thing. And I think our software expands to... you know, match the size of the hardware that it's running on to use every last bit.
And has allowed for many of us, web developers like myself included, to be lazy and do things the easy way, the fast way, the cheap way. But in that embedded world where like every resource counts and it's heavily constrained, you must have to know all of the all the little tricks, the way to manage your memory so as not to bloat or leak. Do you like, I mean, you must like that.
I mean, it's literally a constraint. Puzzle. The challenge of, yeah, it's a puzzle. Yeah. And to many of us, that would be a huge bummer, you know, because it's hard and there's things to do. But you must like puzzle solving a lot.
I do. I do. I really like puzzle solving. And I'm not super afraid of math. So if I need to add up where all of my memory is going or be able to look at a memory map file and see hex addresses, those don't really scare me. I know that they're just addresses. They just look funny.
Mm-hmm.
But I really enjoy the puzzle part. And like I said, I really enjoy the motor moving. And when the robot, for a personal project, I wanted to do some higher level software. So I got a Jetson, which is NVIDIA's AI embedded system. It's a Linux system. It's a computer. I used it as a computer for like two weeks.
And I wanted to get a really janky arm, a robotic arm that could go up and down and grab things. And I wanted to make a typing robot. I wanted to have a camera that would see the keyboard and then the robot would hold a little I don't know, pin tooth like thing. And it would punch out hello or whatever I told it to punch out. And it kind of worked.
And I used some machine learning and I did a lot of computer vision, which is an area I'm always interested in. But then one day I'm telling it, okay, type hello. And it types help. And I'm like, done. I'm like, this is the end of this personal project. I cannot make it any better. This is the best. And I forgot why I started this story, but I hope it was amusing.
Well, I'm wondering maybe if it wasn't a Fiona moment. My friends who are big fans of a very awesome TV show called Silicon Valley will know what I'm talking about. Because in this episode, spoiler for those who haven't watched it yet, she asks for help. She is a robot. She needed help. I'm wondering maybe if your robot needed help. It might have.
It certainly did. It was... In the way of how much software can I throw at a problem to make the hardware that is very bad better? It turns out you can throw a lot of software at some bad hardware and it still is bad hardware. But it can be really fun.
What's up, friends? I'm here with Kurt Mackey, co-founder and CEO of Fly. As you know, we love Fly. That is the home of changelog.com. But Kurt, I want to know how you explain Fly to developers. Do you tell them a story first? How do you do it?
I kind of change how I explain it based on almost like the generation of developer I'm talking to. So like for me, I built and shipped apps on Heroku, which if you've never used Heroku is roughly like building and shipping an app on Vercel today. It's just it's 2024 instead of 2008 or whatever. And what frustrated me about doing that was I didn't, I got stuck.
You can build and ship a Rails app with a Postgres on Heroku. The same way you can build and ship a Next.js app on Vercel. But as soon as you want to do something interesting, like as soon as you want to, at the time, I think one of the things I ran into is like, I wanted to add what used to be like kind of the basis for Elasticsearch. I want to do full text search in my applications.
you kind of hit this wall with something like Heroku where you can't really do that. I think lately we've seen it with like people wanting to add LLMs kind of inference stuff to their applications. On Vercel or Heroku or Cloudflare or whoever these days, they've started like releasing abstractions that sort of let you do this, but I can't just run the model I'd run locally on these
black box platforms that are very specialized. For the people my age, it's always like, oh, Heroku was great, but I outgrew it. And one of the things that I felt like I should be able to do when I was using Heroku was like run my app close to people in Tokyo for users that were in Tokyo. And that was never possible. For modern generation devs, it's a lot more Vercel based.
It's a lot like Vercel is great, right up until you hit one of their hard line boundaries, and then you're kind of stuck. The other one, we've had someone within the company, I can't remember the name of this game, but the tagline was like, five minutes to start, forever to master. That's sort of how our pitching flies.
Like, you can get an app going in five minutes, but there's so much depth to the platform that you're never going to run out of things you can do with it.
So unlike AWS or Heroku or Vercel, which are all great platforms, the cool thing we love here at ChangeLab most about FLY is that no matter what we want to do on the platform, we have primitives, we have abilities, and we as developers can charge our own mission on FLY. It is a no-limits platform built for developers. And we think you should try it out. Go to fly.io to learn more.
Launch your app in five minutes. Too easy. Once again, fly.io. I'm curious about the possibilities of embedded systems. Jared, maybe your question regarding the chasm is, if I begin, how far is that dip, so to speak? How far is it from beginner to potentially professional? How big is this embedded world? Is it an underserved world? Is there a lot of opportunity? Is there one place to go to learn?
Is it all in university or college? How do you get to be And if you do become a pro or someone who's professional, earning their money and career via this route, is it big? Is it expansive? Is it growing?
You might have noticed that your home has more blinking lights than it used to.
Yeah. I'm sad about that.
Electrical tape.
There you go.
It is growing. Some things are becoming easier. Some things are becoming more straightforward. But I see more job openings than people looking for jobs. So I would say that's pretty good. How do you get that? There are a lot of paths because, like I said, what is an embedded system depends on your perspective.
There are quite a few embedded Linux projects that still are embedded because they need to be very conscious of their constraints. And yet, to most software engineers, it wouldn't look that different. It might be boring because it's trying to figure out how to provision 100 small sensors with IP addresses. But the actual algorithms and such wouldn't be strange.
It would be the interactions that were a little different, having to deal with actual boxes out there.
Mm-hmm.
But then when you start getting into higher, I guess it's lower tolerance, things where you have to last for five years or have to squeeze out every microamp possible, or even things where firmware update is very hard and if you do it wrong, things go very bad for you. Like firmware update at the bottom of the ocean. Those, you don't start with those.
I think that's what I need to say is there are better places to start. And there are a lot of things that software engineers know that embedded software engineers are still picking up. We are not really that good at CICD. Somebody could just tell us how to use Jenkins instead of making us figure it out each time. That would be great. I mean, that's not, I use Jenkins, but.
Sure.
Some of the software engineers I work with, Jenkins is just like obvious. Every part of it is obvious. And to me, it's like, I can do what I need to do. And when you ask me to set up a new project, I just go home and like, I don't know, turn off my computer for the rest of the day. Design patterns was one of those things that I really wanted to bring to embedded systems.
And I feel like I have kind of talked about it for the last 20 years about how important it is to not keep solving the problems in new ways, but to start solving the same problems in the same ways each time so that we don't have to rewrite all the code. And things like object-oriented programming, it's not new. And a lot of embedded people really understand it.
And yet we mostly program in C still. I know. I know the gasps, the horrifying screams, yes. We still mostly program in C. But we don't usually program in an assembly anymore. We're making progress.
There you go. There you go. Yeah. So what about open source? Has that permeated your crew yet?
Yes. There's open source. I mean, the vendors of our chips have figured out that they don't really want to keep giving us their crummy software in ways that only they can manage it. So like ST and Microchip and Nordic all have big repositories and they have examples of working with different sensors. They have all the drivers there.
And then we have things like Zephyr, which is an operating system that is kind of like you took Linux and you made it much smaller. And then instead of anything you could run at runtime, you hard-coded it at compile time, which has some interesting things. The device tree is one of those things that most people in their first three months of Zephyr experience will start cursing.
And once you have, yeah, so Zephyr is huge and FreeRTOS is now open source. Walkway is open source. Most of it, I think. I think there's just a little bit that isn't. A lot of like the sensor vendors, like the IMU vendors will give us code. And sometimes that's partially open source and sometimes it's not.
But the easy stuff now, the Kalman filters that used to be so complicated, now that's code they just give you. They tell you how to do.
What's that?
A Kalman filter is a way of... I could talk about this for like three hours. I'm trying not to. Okay. Okay, accelerometers and gyroscopes. Accelerometers measure gravity. They measure acceleration too, that's what they say, but really they just measure gravity. Gyroscopes measure spinniness, how fast you're turning.
And it doesn't really matter whether you're spinning around a certain thing or whether you're spinning around part of a thing. So you can measure how your arm moves with a gyroscope. It's not like it has to go around and around. It's just how much is it turning. when you are in a car and you're going around a cloverleaf. You're spinning and you feel acceleration.
You feel acceleration, you know, the outward push of you as you're spinning around. An inertial measurement unit can use both of these sensors and know which one is wrong. And it's kind of like your ears and your eyes. If you are spinning around in a circle and your eyes are closed, you're going to get dizzy much faster than if your eyes are open.
The Kalman filter is a way to combine sensors with different error features, different error characteristics into a way to get a much better response because you can characterize the error when nothing's happening. Or you can characterize the error in certain conditions where you know one is good and one is bad, like on a clover leaf.
Gotcha. And so the vendors of the device would provide like a set of filters for you to use?
They would provide the algorithm, yeah. And they would provide the base code for how good each of their sensors are.
Okay, because they've done those tests and they understand the sensors and their error rates and these other things.
Exactly.
Better than you do.
Better than I want to.
Yeah, I mean, after a while, you may understand them better than they do. But those used to be closed source or they used to be unavailable to you and now they're just provided.
In 1990, it was a patent I worked on.
So you helped invent these things?
Well, no, I helped implement them.
Okay.
It came out of a, and to be fair, a lot of people started doing IMUs and Kalman filters in the early to mid 90s. And it exploded as to the applications, but a bunch of PhDs were done at that time. And so they became very, very popular because they solve the inertial measurement unit problem pretty well. And now they solve a lot of other problems that involve a bunch of sensors.
Usually when we talk about self-driving cars, the GPS has one set of errors, mostly it's really slow. The LiDAR or camera can have another set of errors and it provides a different set of information and the different types of Kalmans can put all of these together. And the idea is just you put the errors here and you put the data here and you try to make the best out of the bad situation you have.
Gotcha. So you mentioned self-driving cars and you mentioned LIDAR. So I have to ask your take as an embedded person on Tesla's proposition that they can get to autonomous or whatever level you need to get to with cameras only. And then Waymo's using LiDARs. Other people are using LiDARs.
How important is the LiDAR, do you think, pure opinion, versus having a bunch of cameras and letting that be your only input or your only data streams?
I can't picture doing it only with cameras.
That's what a lot of people say.
I mean, the argument is humans do it only with cameras.
Right. Like all we have is eyes, right?
But we have a few other things.
Such as?
Our eyes and our inertial measurement units are very well connected.
Okay.
So there is no disconnect that if you jerk the wheel, you know what your vision is going to do.
Okay. I'm tracking you.
And as a human, I just cities, cities and self-driving cars. I worked on autonomous vehicles and not just underwater, but self-driving large off-road vehicles. And. We did it almost entirely with cameras. But part of our goal for the future was dealing with stop signs and pedestrians. And I spent a lot of time with the machine learning systems and the reinforcement learning systems. And it...
It's a hard problem and I don't know how they're going to solve city driving in a way that is truly safe and doesn't have systemic failures. That's one of the problems with medical devices. When we talk about doing embedded systems and medical devices, it's usually there's some element of, oh my goodness, there's so much documentation. There's so much documentation the FDA needs.
And that's primarily to make us understand the ramifications of the points we're making. And the reason for that is because Oh, I've started this in the wrong place. Okay. Moving backwards. If you work on airplane stuff, there are different levels of concern. If you take down a small airplane, a Cessna, it may crash into something important like a school.
And so you have a certain amount of damage you can do with a small plane.
Sure.
You have a much bigger amount of damage you can do with a larger plane. Medical devices are a little confusing because, and safety critical devices, medical or vehicles, have a problem that you can have thousands of them in the field when they start to fail. And so your problem is not that you have hurt one person.
Your problem is that your software has decided that the 29th of February is not a real day and has gone nuts and is now on its murdering spree. And that's what you need to avoid. And so the thing with cars is that we keep trying to make them better in small ways, but I feel like we've kind of reached a local minimum that we're not going to get better with just cameras.
I'm not even sure we're getting it better with just a LiDAR. But I don't know how to build in that feeling of where the car is, the proprioception that you get as an experienced driver. I'm afraid that cars will drive like 16 year olds forever. And I don't know that that's what we want.
Yeah. As an outsider who I'm not sure I would ever even venture into the world of what it would take to be on a team that creates autonomous cars. I'm just I'm a fan from afar. Let's just say of the idea.
Me too.
I like the idea, but I think it's. This is layman. This is from the outside. It seems perpetually flawed because it's not a closed system. There's so many flaws in the system that you can almost never predict. You can predict a lot of the various ones, but you can't predict...
all of them ever, and because it's not a closed system that you control, then it seems like you're perpetually going to be flawed. No matter what, there's always going to be something that changes the game, potentially catastrophically, that you can't circumvent.
As a human, you can't do that either.
Right. Yeah, I think your point about the horizontal scale, Alicia, if that's just the way I'll say it, is that we have crazily bad 16-year-old drivers... But it's not like at a specific moment in time, this percentage of all cars on the road are going to turn into that.
The error that happens spread across a fleet of cars all running the same software, for instance, that makes the same mistake all at once. is catastrophic. Of course, the famous or infamous, I think this was the one that the first Tesla that killed somebody that was in FSD mode was because it came over a hill and the sun was coming up And the sun shone into all the cameras at once.
I'm just giving my depiction of what happened. It's roughly that. And the thing, I mean, it couldn't handle it. Well, of course, a human couldn't handle that either because your eyes are completely blinded. And so likely, maybe the same result in that one instance. But if something like that happens across 50,000 cars at the same time.
that's a that is a catastrophe of course it's a tragedy when one person dies but when this fleet suddenly goes bad i mean the stakes are way higher right yeah i mean i can i've been that human jared that's coming up over a hill with the sun i mean i'm 16 and i can well not even at any age really i mean period as even a good driver i would say a marginally good driver i think i've had
Uh, the most recent accident I had, which is like the one I didn't cause the deer hit me. And this was like a year ago. Some things can't be avoided. And this deer wasn't like random at nighttime when you would think a deer would be out. It was, uh, nine in the morning in town. Fuck. Like where other cars were at. It wasn't the time when deer should be running necessarily.
He was looking for your car. Ran right into me. And then the other time was, geez, probably 20 years ago. And it was just wet. And I just didn't stop in time. Like I stopped, but my car, it's the size of the skid because, you know, that's what happens. Yeah.
Yeah.
And sure, I could have been paying a little bit better attention, but I'm a pretty good driver is the point of the last 45 seconds. That's the point you're trying to get across? I'm a pretty decent driver. Right. But I've been in a scenario where the sun has completely overtaken my vision completely. I do recall in seconds beforehand what was in front of me. And so I went now from literal memory.
Mm-hmm. with confidence of my next driving maneuvers straight. Right. And around the bend a little bit kind of, kind of because I was going from my memory before my site was taken away. And the case of the cameras, you know, the thing freaking out, I don't know what happened there.
Maybe it was like a literal panic mode, but you know, like in my own body, I had a panic version, but I just didn't crash. There was no Adam BSD or BSOD. There was a, it was just a, you know, continued driving. It was safe. Yeah.
One of the reasons why I've been tentatively positive about this progress is because of how many accidents there are and because how distracted and bad we are at driving, even though we are amazing, Alicia, to your point about the actual physicality of observing the world that we're in and that connection between our hands and our eyes being just grounded in, I don't know, biology.
We're pretty amazing computers. But I don't think these cars have to get so good to be better than us because we have so many failure rates already. The failure rate's very high. That being said, it does seem fraught. It's always been, it's 18 months around the corner for the last 10 years. There's progress being made. I mean, Waymo is opening in new cities.
It seems like maybe a city-by-city rollout, and I believe they have LiDAR involved as well, is working. They also, I think they have a call center, I believe. I could be spreading fake news, but I think they have actual humans, not in the loop, but available in order to take over a car if need be. That seems like a good fail-safe. There's progress being made, but it does seem like
I don't want to call it an impossibility, but at this point, I just don't feel like we're close.
Can we go back to my idea of this closed system? Is that even a possibility? To your point, Jared, can we create a closed system?
Freeways and trucks.
Yeah.
Self-driving on freeways and trucks, I think, is a better possibility. Freeways have a much smaller number of things that can go wrong. And it is more of... It's not a closed system.
Right.
But it is a differently constrained system.
Right. And a smaller problem set.
I mean, if they had their own roads, it would be much simpler. Then it would be truly closed. And... In the meantime, I think we can get closer on freeways more than anything. I mean, there have been a lot of, most of the new cars now will have lane keeping and keeping up with the person in front of you. All of the things that the initial full self-driving Tesla did.
Mm-hmm.
we can do now in you know my new bolts it is progressing whether or not we call it self-drive or if we just call it new safety features yeah i think it's getting there and i like the idea of a closed system but at some point you do go back to being a train and so you're laying track Or lane pavement that can only be used by this form of self-driving.
Right. Some sort of guardrail, parameters, boundaries, something that says this is where you should be. And if you're not here, then you can no longer operate or something changes to no longer be the danger possibility, so to speak.
We'll put sensors in the roads. We'll have it monitor the car in front of you. and humidity conditions. And then the sensors in the roads will talk to the cars. Wait, wait, let me scribble this down.
I need to invest in a smart rock company because we're going to be selling a lot of smart rock. Yeah.
Well, as we're knee deep in this conversation, I was just thinking, you know, one of the most Probably globally known, most people have bought one at least one point in their life or want one. An embeddable system is like a GoPro, for example. That dude who founded that company was just surfing. He was like, man, I would love to film this. And he went away and did his thing and now GoPro exists.
And I'm just thinking like, wow, this must be a proving ground for not just innovation, but potential future founders. This seems like a good place to go and not so much get rich, but change the world in dramatic ways. I think GoPro has changed the world in dramatic ways. It's given you, let's just say, a camera view, obviously, of something dangerous. But then you have that GoPro effect.
Like, oh, that doesn't look that steep until you get there. And you're like, that's pretty steep. You know, like that's this embeddable system world seems like if you want to go and invent something that this is a place to go and invent something because it's real. It's physical.
Yes. There are lots of times when it's like I have a problem. I know how to fix it because I know how to wire things together. But I also have that same feeling in software. Yeah. that sometimes I can invent something just pure software. I do origami and I write Python scripts. I write very long and complicated NumPy scripts for doing origami. And I love that.
That's crazy. What do the scripts do for you?
They create crease patterns, which I can then use a laser cutter or Cricut or other device to pre-score paper. Then I can fold along wavy lines, which will give me origami that is not flat, that is patterned in weird and wavy and organic methods.
Does this exist in the world? Can you do that? Can you go buy that product right now?
No.
Well, then you just invented something.
Do you, I mean, there's curved origami out there, yes. Yes, there are other people who do curved origami.
It's just a world that I've not been introduced to, so I don't know what exists. Yeah. But yours is custom curved origami that you want to make. Yes. And so your scripts give you the crease lines.
The scripts give me the crease lines. And they... Let me combine straights and creases in order to get tucks and weaves. And I will go off the rails here. Go off the rails.
No, we're with you. We are with you. I'm listening.
Adam was saying how you can invent things. And I think you can invent things in software too.
Right.
And it's some of the same impetus that you invent them because you want them. I had a stuffed animal that would wifi to another stuffed animal, wifi through the cloud to another stuffed animal. And when you padded it, the other stuffed animal would light up and you could, you know, you could have this, this sort of communication to someone else.
What the other animal do?
I would pat mine and theirs would light up. They would pat theirs and mine would light up.
I see. So there's like a,
It was like a good night, sort of.
Yeah. Maybe for, like, brothers, you know, they're in different rooms. Like, good night, brother, kind of thing. Not a walkie-talkie. It's more like, I'm betting mine, you're betting yours.
Right. You just know that somebody else is there doing something. It's a connection.
Because it's a two-person system. Yeah, it's a peer-to-peer with two peers.
Well, this actually went through the cloud. So it was more like you're in college and you're telling your parents you're going to bed.
Yeah. That's cool.
Yeah. All kinds of just, yeah, two people connecting quietly. Yeah.
I just like, I think I'm liking this idea more than what we're talking about. I mean, I've always liked this idea, but this idea of software paired with hardware that creates these newer experiences that isn't software only, like an iPhone or a computer screen or a browser software, which is like, maybe that's the kind of software I'm thinking of.
But then I'm also thinking like, not just that GoPro, but I use, so if you didn't know this, Alicia, I'm a, I like to barbecue. And so I own a Trigger. A trigger is a smoker, but it's basically a computer. It's an embedded system, right? Somebody wrote software to have sensors that says, how many pellets should I push through to maintain this temperature? What temperature is it in the ambient?
What temperature is it at the burner? This thing that I just push the button, turn the dial, set it to 275 and wait an hour or two, that's a computer, an embedded system that somebody invented. Because they were like, well, doing it with the old wood method was too cumbersome. It wasn't ubiquitous enough because too many people didn't want to tend a fire.
And so now triggers exist as a smoker to give way more people access to push button oven light control of a barbecue system. Exactly. That's crazy. I never really thought about this trigger as this embedded system until this conversation. Obviously it is, but I never thought about the fact that I could have made that.
You could have made it.
If the things were more accessible. And I think that's a lot of this conversation, like how accessible is this technology to, to people, to, to lay folks like Jared and I, or we're seasoned in software development, but not in embedded systems necessarily. Yeah.
But with an Arduino or one of these little micropython circuit Python boards, think about what you would need to do. You would, you need to, okay, so like mentally build the system architecture. You need to push palettes. You need to be able to tell time. And you need to monitor temperature. If you went online and looked up, how do I get a thermistor? How do I tell temperature? Really easy.
That would take you three minutes. Wiring, it's going to be easy. Powering, it's easy. Doing the timing, a little bit of research on that will probably turn up that you don't care whether you're doing two hours in one minute or one hour, 58 minutes. So you don't have a big constraint between really good time versus kind of crummy time either way, as long as you're around two hours.
And now you have to figure out how to push the pellets in in a way that's probably metal because you don't want it to burn in plastic. You're not going to 3D print this. And now you need a motor subsystem. And that's where things get a little complicated. But motors are very common. And you can buy the kit that just says, I want you to move here and I want you to move there.
And so you push the pellets in. Maybe you have another door closer thing. Now you're getting into the mechanical. And I do get intimidated by mechanical, but you have a unit that works. You can start to look at it and see how does this work? What are the mechanical pieces that I need to be able to push and close doors and scoop pellets? It becomes... You want to build this.
So you build each piece and then they all work together and then it's done. And then you want to do firmware update and you call me.
Yeah.
Yes. Love it. It seems like the, you said you get intimidated by mechanical and it seems like the pusher would be the... the hardest part of that, maybe you could utilize gravity to just have like, you know, an opener and shutter versus a pusher. And then you're dropping the pellets in. I'm not sure how triggers work.
I know exactly how it works. I'll tell you. So there's a hopper. Yeah. Right. And in the hopper, you fill it up. It uses gravity to, as you know, drops the pellets. Yeah. As the pellets fall down, it falls into this space that there's an auger and the auger just turns. And so it slowly just moves the pellets along the system. And so, um,
you fine tune the temperature within the firebox based on the speed of the auger. So if you slow down the auger, no pellets feed it. If you speed it up, it speeds it faster. And you probably have some, maybe an algorithm or machine learning of like, how the speed of the auger versus temperature control, and you're obviously maintaining temperature, but it's just literally a turning mechanism.
It's not even smart aside from knowing speed and temperature and how speed affects temperature. And the hopper is just the pellets are falling based upon gravity. So gravity did the work there. Thank you. No pusher necessary. And there is a sensor inside the hopper that will tell me if it's low, but I'm smart enough to know how much I put in there. I know when it's low.
And my cooks are never that long where I need to literally monitor my ever. I've never had to know if it's low. I think it's just a gimmick, really. In terms of time, on the timer, I mean, Trigger does have an iPhone app and an Android app that controls it, and you can set time and stuff like that, and you can tell it to turn off, but that's a rich person's feature. Not everybody needs that.
You really just need the ability to maintain a temperature for a sustained amount of time. Any barbecue person will tell you all I'm teaching you is time and temperature. I'm not teaching you anything special about cooking food. Just time, how much time do you need, and at what temperature. So that's the recipe for cooking is like, you know, time and temperature.
And so the trigger just basically does those two things is time and temperature.
And that mechanical is so much smarter than what I was coming up with.
Okay, friends, have you ever been doxxed on the internet? Maybe you've been stalked. Maybe you've been harassed or you've had your identity taken from you. Privacy matters so much to everyone, obviously. But have you ever wondered how much of your personal data is really out there on the internet for anyone to see? There's more than you think.
Your name, your contact info, social security number, what unlocks your credit history, your home address, or even information about your family. That's terrifying. All this data is being compiled by data brokers and is being sold online. So DeleteMe is one of our sponsors, and they gave me an account to use.
I tried it, and it's helping me remove my personal information from hundreds of data brokers out there. Here's how it works. You sign up, you provide them with exactly what information you want deleted and their experts take from there. They send you regular personalized privacy reports showing you what information they found, where they found it and what they removed.
And delete me isn't just a one time service. It's always working for you, constantly monitoring and removing the personal information that you don't want on the Internet. To put it simply, delete me does all the hard work. It wipes your information, your family's personal information from these data broker websites and gives you peace of mind.
So take control of your personal data and keep your private life private by signing up for Delete.me now at a special discount for our listeners. Today, you can get 20% off your Delete.me plan by texting CHANGELOG to 64000. Again, text CHANGELOG to 64000. Of course, message and data rates may apply. Check their terms for more details. Enjoy.
So, Adam, what do you want to build next?
Uh, geez, let's see. Birthday cards.
Birthday cards.
Say more. Yes. My family is a fan of singing birthday cards. And my son loves to take them apart later. I'll tell you a story. So my son is curious like this. Like he loves to take things apart and figure out how they work. And so he took apart a birthday card. We got him. And obviously you open it up and it starts to play the song or whatever it's supposed to do.
Well, he decided to take it out of the back of the package, discover the sensor, and discover this thing and this mechanism, and he decided to pull the wires out and put them in different places. He made this birthday card do things it wasn't intended to do for us. It was just meant to be open, do the thing, and it's a closed system in that regard.
But he found out you can pull the sensor out or whatever the mechanism is and rewire it, and he made it do something different. He's like, dad, look at this. I'm like, what is that? He's like, it was my birthday card. You know, that's kind of cool. Okay.
Chibitronics. C-H-I-B-I-tronics.
Okay.
This is away from the software, although they have little microcontrollers it works with, so we can get to the music and whatnot. But what they've done is they sell these kits that have copper tape, and you copper tape around, and you can make little buttons by having pieces of paper touch each other. You can make little sensors by having patterns of copper tape do different things.
They sell stickers that are LEDs, so you put your copper tape up to one side and your copper tape to the other, and then you put it to power and you have a little battery. Usually the clips, what are they called? What would you call a clip like this? Oops, nope, that's... Binder clip.
Binder clip.
Put a binder clip with the battery and you make your own card like that. And it lights up or you can have it do music. They have all of these different neat sensors. And this is more on the electronic side. This is how people understand how do you go from battery through a circuit, right? And you don't need to know Ohm's Law and all of that, Kirchhoff's Laws. That's not that important for this.
It's really about you follow the copper tape to the different pieces and you make neat lit things. And you can add a controller, which can then even add a little tiny motor, which can either vibrate or it can move like you do a paper engineering cutout of a butterfly and it will gently wag the wings. Yes, chibitronics. They're super fun. Chibitronics.
They were directed towards women at one point, getting girls interested in chibitronics. storytelling through electronics but i don't think they are as gender specific anymore because i've been seeing lots of their new sensors and new systems um being a little louder okay since we're here then i have to ask you i'll just say one one brand and i want a reaction from you alicia crunch labs nothing
I'm terrible.
Oh, my gosh. Nothing. Well, then. That's great. So I don't know his name. I'm going to Google real fast. Crunch Labs, the fellow that runs this, this is a very massive YouTube channel. But it's not a YouTuber, like a, I don't even know how to describe it.
Oh, I've seen this guy.
Oh, yeah. It's the coolest stuff ever. This guy used to work at NASA, and he is all in STEM and all kids-focused, but he has his subscription box that you can get to build different stuff. And as you're describing this – how do I pronounce this? Chibitronics, I'm thinking like, wow, here's more kids that entertain and educate these. Yeah, Mark Rober. Thank you, Jared.
Yeah, everybody loves this guy. Yeah, I love Mark Rober. He's the coolest. I mean, he is – I don't even know how he does what he does. He's amazing. And it's like he's living his purpose for sure. But my son loves Crunch Labs, is so entertained by all this stuff, has become smarter because of Mark Rober.
I mean, like educational, curiosity, STEM, embedded systems, I'm sure, electronics, Rube Goldberg things, et cetera. Like just all this cool stuff. So I thought you might know Crunch Labs. Yeah.
I don't, but I'm going to sign myself up. I love the little robots, and they had a crossbow. How do I get the crossbow?
How do I get the crossbow? That's great. And these kits, they come on this, like, frequently. So I was thinking, what made me think about Crunch Labs was the monetary accessibility of the chibitronics and, you know, And building these things that no one else seems to be doing to... I mean, at some point, you got to want more people in your field, right?
You're probably lonely to some degree in your field. You want more folks who have the same drive and curiosity as you do. And the only way you get there is by inviting them in.
Yes. And my field was so lucky to have Arduino because it really drew a lot of folks in. to be interested, both in electronics and embedded systems, but also in software. And there are Chibitronics and some of these other robotics kits. FIRST Robotics is just wonderful. That's the competition for middle and high school students. And what the kids learn is just
just want to hire all of the first robotics engineers. Because once they get into the robotics and they start understanding how the mechanical and electrical and software all work together to make the robot move, they get a systems understanding that's really hard to teach.
We've gotten so many amazing links from this conversation. Are there others? If we're like Alicia, what's another cool thing you just showed us? Chibitronics first. Firstinspires.org is the first robots people met or met. Adam pointed us to Crunch Labs. Of course, we have MicroPython, Adafruit, the WalkWe. What else you got?
Hackster.io, Instructables.com, and Hackaday.io.
Oh, Hackaday I'm well aware of. What was the first one you said?
Hackster.io. Hackster. And Instructables.com. And I'm grouping these all together, even though they have very different flavors, they are all places you go and tell people about your projects.
Nice.
As an engineer, it's like, okay, I'm learning to document. This is why I tell people to use these sites, because I want them to learn how to document and reproduce their projects. But as somebody who's doing it, as somebody who's excited about it, it's getting to show off what you did.
we don't always get to go to these hackathons or to these hack meetups where you get to bring your project, but this lets you connect to people who are maybe in the same boat. Maybe there's a competition for Nordic dev kit.
And so everybody's looking at the same set of sensors and you get to see what other people's ideas are, but you also, you know, there's, they all have community aspects where you can talk about what you're doing and talk about,
what you want to be doing, as well as demonstrations where other people have said, this is what I did and these are the steps so you can reproduce it in case they have a neat project.
Love it. Where's this going? You know, like this kind of thing. Showing off is kind of cool, right? I'm looking at this one here on Instructables in the circus section because it lets you see things like crafting and cooking and living. So there's more than just simply this embedded world there. It's a sleep cycle lamp. And I'm seeing like this lamp with lights.
a common shade, probably a similar, or sorry, a very small board beneath it that's powering this thing and probably basing it on time as doing different sleep cycles, colors, tones, et cetera. But I'm thinking like, man, where could this world go where you have infinitely accessible Lego-like things?
Not so much Lego itself, but how you can take this part, pair it with that part, and make a product or create something new or invent something new. Like where... Where can this go? Where's the future going for this?
Like you have this idea and you want to market it?
No, no, no. More like, more like you, where do you think, I mean, given your knowledge of embedded systems and where you've seen it go, what's next for, I suppose, this world of them from the simple hobby project to underwater things with sensors looking at dirt moving around. You know what I mean? Like it's such a chasm, but like, where's, where's this going? It's a big question. I'm sure.
I mean, it's going everywhere. I fought against the term internet of things for so long. But that's what it is. Everything wants to be on the internet, wants to help you be better or do something different. And I don't love all that. I mean, I don't need everything I have to be on the internet. But having two stuffed animals essentially on the internet, talking to each other over distance,
It turns out to be kind of cool. The applications turn out to be really interesting. The ability of my doctor to see if I'm using some medical device they've prescribed. okay, that gives them more tools. If it's not okay with me, then I make it not okay.
But it gives other people the opportunity to engage with us if we want to, or to monitor elderly folks, or to teach kids to read, or... Adam, I have to say, it's going everywhere.
Yeah.
And I'm not sure that's the best thing, but... It's also going into all kinds of science. One of the projects that I worked on a year ago, two years ago, that's going out this year is they're just going to chuck these boxes into various rivers around the country and be able to monitor the environmental DNA. So
They would know if an otter moved into this river before anybody could see if they were actually there yet. And they can also monitor things going wrong, like human DNA being in the water too much is bad. The Internet of Things, that's where we're headed. And it may not be great, but some things will be great.
So embrace it, right? You know, you've lamented your fight against IoT, but just embrace it, basically.
Be aware of your security. Be smart. You don't let strangers into your house just willy-nilly. Maybe look at the names of the companies you're letting into your house. And make sure that they have nice friends and good references and all of that. Because you don't want to create bot networks or allow people to spy on you. So when you do engage with the Internet of Things, do so wisely.
And it's not just people in their houses. It's companies and businesses. UAVs, you need to know where you get your UAV from because some of them report back to their home countries. And that's kind of scary depending on what you're doing. If you're using it to take pictures of the beach, who cares? If you're using it to map your new mining prospect, I don't know if you care.
Yeah.
But you should think about it. Security is unfortunately the downside of IoT. It's also the S in IoT, which is to say you really need to think about it for yourself.
And privacy too, which those two are kind of eternally linked, aren't they now? Because being able to throw a smart rock into a river and monitor things is awesome. But being able to throw it into my pool and monitor my swimming, maybe not so much. Just for instance, you know?
Yes, there are a lot of downsides to this.
The Internet of Things. And also, for IoT makers out there, you know, just leave us physical buttons. You know, like give us, let us opt out every once in a while and just use the dishwasher without having to launch an app. Oh, yes.
Yeah.
For instance. Yes. I take it your dishwasher requires you to use an application or something, Jared? No, that was just a for instance. I'm sure somebody's does. Yeah. Well, I have LG washer and dryers, and I can tell you when my washer is done and when my dryer is done. And I can tell it to start drying something right now if it's on. So that one safety, security and safety, maybe the similar S's.
is that I can't control it from afar or remotely if it's not on currently. I can't power it on and power it off. I can only instruct it if it's been previously human-touched, like recently kind of thing.
And would you have paid for those features?
Apparently I did.
I know. We did too, but I would never meant to pay for those features.
No. I never want almost any. Sometimes you're like, oh, this actually is kind of nice. Once you start using the feature, you're like, yeah, I kind of appreciate this, but it was never.
When the dryer's off, it's kind of nice.
Yeah, it never has sold me on any piece of appliance.
So the one thing I would say about that that I've appreciated, because I try to be not the person who only complains. I try to be the person who appreciates pretty much any scenario. I do my best. I'm always the greatest at it. Is that it's a different interface to a simple interface that doesn't make any sense.
And so the iOS application actually makes a bit more sense to use its advanced features like... Uh, like the dry clean version, there's a dry clean setting on the dryer. For example, it hooks to the water and it can humidify the air kind of thing. It can do some cool stuff. That feature is not easily accessible in the, on the device itself.
I mean, you can, it's just harder to like map your brain to like, okay, push button go. Whereas the iOS app is a bit more easier to use. I think of it like a different interface to this thing. The notification is, I turn that off in most cases. Like I actually don't have it on unless I'm washing clothes personally. And when I say that, I'm like, I don't usually wash our clothes.
So I'm that person who gets to just sit back and enjoy life for the most part. You better not be complaining. Until I have no more underwear and then I'm like, oh my gosh, I have to wash my own clothes now. And you're checking the status at all times. That's right. Notifications on. Thank you.
I mean, the app does give you more information than could possibly fit on the front of your washing machine. It does.
Yeah, so that's how I look at it. It's like, I wouldn't have paid for that feature. I think I did because I bought it, obviously. I didn't buy it because of that feature. And I'd say that it's a nice to have, not a need to have, but it is nice to actually have because I've used some of the advanced features via the iOS app.
And it's just a different way to interface with the device that's actually better and easier.
But you know, those... Those bits of information that they're showing to you can also just seep out of your local area network and into the hands of somebody.
Adam is washing his clothes again. Get him.
You know, someday you're going to get an email from a hacker and he's going to be like, Adam, you're out of underwear.
Yeah. Listen, hey, listen, you probably should wash your clothes. It's been about two months since your dryer's drying anything. Yeah. Clearly you're out of underwear at this point. I've checked your Walmart checklist for your underwear or wherever you buy your underwear. I don't buy my underwear from Walmart, but I just said that. TMI, dude. Just so you know.
Okay? All right, Alicia. I assume the website is embedded.fm. Where else can folks connect with you?
You can get embedded from any of your regular podcast sites. And if you are faced with one that is embed in all black and looks scary, you're the one with the robot.
Look for the one with a robot.
It's like a little robot head slash radio, old-timey radio. So that's us.
Nice.
You can get my book. It's called Making Embedded Systems. And it's in second edition. It's from O'Reilly. It's on normal Amazon and bookshop sorts of places. If you're interested in making embedded systems, the goal is for hardware engineers to understand more about software and for software engineers to understand more about hardware and how to design and make embedded systems.
If you want to contact me for consulting, I think the answer to that is please don't. Please don't.
Please don't? Is that right?
We're booked. We're booked for like a year. It's great.
Congrats.
And yet I always love to hear about neat things people are making. So if you want to tell me about your embedded system, hit the contact link on embedded.fm or email me at show at embedded.fm. And I'm just fascinated by the technologies coming out.
Very cool. Any conferences or like IRL places? I know you mentioned several digital ones, which we've definitely collected and we'll put in the show notes. But is there anywhere that's like common? Like is there an embedded conf or something like that where it's, you know, international or accessible to folks? Like the one you can't miss kind of an event? Yeah, like you must go there.
If you're really getting into it, you must go to this conference kind of thing.
This is another one of those areas where there is a good hacker subset and there's a professional subset and they don't quite overlap. I will be keynoting the Embedded Online Conference in May of 2025. Please don't ask me what my talk is about. I don't know yet, but I will really soon.
Embedded things.
Probably sensors. And for the hobbyist or... See, it isn't just hobbyist. I've gone to Supercon from Hackaday. I would happily go to Teardown. There's Open Hardware Summit. There are a ton of really fun places where people will show you how to get into embedded systems or hacking devices or making cool things.
There aren't as many professional conferences unless they're about certain topics like Zephyr or security. Hardware.io, H-A-R-D-W-A-R-E.io is one of the international conferences that's pretty popular and more security based. I can't think of any other ones, but I know I will when we get off. So I'll send you email and you can provide links.
For sure. I know IRL is important to folks. We can obviously shell out lots of digital locations to hang, but IRL can't beat that, right? Real humans, real people, real fun.
Really?
I think so.
Really?
Maybe not. Maybe not. To each their own, as they say. I'm kind of an indoor cat. We'll have to get some sensors for that and maybe a motor to judge it.
There you go. Or one of those things you can touch that we know like Alicia is still there. She can just send a sensor to the conf. Still here. Like light up a light, you know, blinking lights. We hear you like those.
Yeah, for sure.
As long as they're blinking for other people.
Well, Lisa, thank you for your curiosity, too. I know that sometimes you're going alone, right? Sometimes you're just in the dark, but now you're not. Obviously, your book's solid with your business. You've written books. You've come on this podcast, and you're doing cool stuff. So thank you for sharing all that with us. Appreciate it.
Thank you for having me. Your podcast is a lot of fun.
Well, thank you. Well, the world of embedded software is going everywhere. Treggers, GoPros, birthday cards, smart rocks. I mean, the possibilities seem to be literally endless. And so maybe this is for you. Maybe this is your new field. Maybe this is your invitation into the world of embedded. And on that note, there's a ton of links in the show notes. So make sure you go check those out.
Wookiee, Crunch Labs.com. chibitronics so much to explore and i think you should do it so there's your push and if you haven't yet make sure you check out alicia's podcast embedded.fm i'm a listener and you should check it out of course a big thank you to our friends and our sponsors for this episode today augment code check them out augmentcode.com
This is the developer AI for teams, large code bases, the ask me anything out there. So level up your copilot and go with Augment Code. Check them out, augmentcode.com. And to our friends over at Fly, you know we love Fly. Fly.io. The developer cloud for everyone, for those who ship like you, like me. Fly.io. And, of course, to our friends over at Delete Me, text CHANGELOG to 64000.
Get 20% off and delete your information off the internet. Again, text CHANGELOG to 64000. And a big thank you, of course, to Breakmaster Cylinder for those awesome beats, those banging beats. We've got some new beats planned for this year, 2025. Stay tuned, of course. But in the meantime, thank you, BMC. You're awesome. Okay, we are now shipping full-length episodes, chaptered and all.
to YouTube. So if you've ever want to see the behind the scenes, the, the never saw before the full length, not a clip, but a full length show on YouTube, our new workflow is video first. And we have a lot that we're exploring this year on YouTube and through this video first flow we have going on. So explore with us, join us, youtube.com slash changelog. And yes, cpu.fm is in full effect and,
We have things happening behind the scenes. They're not on the .fm yet, but soon. Very, very soon. We'll have a brand new super feed for you to subscribe to. And there you go. That's all I'm going to say. That's it. Okay. This show's done. We'll see you on Friday.
Have you been tempted to build any sort of systems for your pets or your animals to know where they're at or to warm them up in their beds kind of thing? Have you ever done anything like that, like personal projects for your... Not for the pets.
She's got an air tag on her, even though she's a tiny dog and the air tag is like half her weight.
Yeah.
um heaters no because i'm really neurotic about setting the house on fire but what if it was water cooled or water warmed warm water yeah smart rocks i'm just i never got over that i'm like he just said smart rock and i'm like that's that's new to me you're dumb as a rock no you're smart as a rock i know it's like a whole new world where rocks are smart cats and dogs living together
I'm inspired by this. I mean, I feel like we should quit podcasting, Jordan. Just build some smart rocks, man.
Or something. I don't know. See well enough.
I'm still entertained by that desire from you. You wanted to put a sensor in your freezer. Remember that?
Yes.
And I think you ended up doing that. Did you end up doing that?
No.
No. I forgot about it.
But that'd be so cool. Oh, yeah. I have the... I still have the pie. Okay. and I was using it for something. I was using it as a, what's that thing called, the blocker pie? The pie hole. Yeah, I was using it as a pie hole, and then I quit using that, and I'm using XDNS instead. And so the pie is just sitting down there, and it's relatively close to the freezer.
I just want to make sure, at least here we have, we buy like half a cow once a year kind of a thing, and like that's very expensive.