Alex Stamos
Appearances
Search Engine
The Dave and Buster's Anomaly
Hi, my name's Alex Stamos. I'm the Chief Information Security Officer of Sentinel-1 and a lecturer in computer science at Stanford University.
Search Engine
The Dave and Buster's Anomaly
It means I used to be a hacker and now I'm a corporate sellout. That's what it means.
Search Engine
The Dave and Buster's Anomaly
It's fine with me, man. There's plenty of podcasts about the top 100 problems.
Search Engine
The Dave and Buster's Anomaly
So if you're looking for that podcast, turn this one off now.
Search Engine
The Dave and Buster's Anomaly
What? That's the back door? It's not like Xi Jinping looks like Winnie the Pooh or a sequel query or something. It's Dave and Buster. Let's do this.
Search Engine
The Dave and Buster's Anomaly
Yeah, I get three dots and it's still three dots. Yes. This is giving me anxiety just watching. And it doesn't show up. That's crazy.
Search Engine
The Dave and Buster's Anomaly
They now have this feature, which is they do AI, where they listen to the message and do real-time transcription of it.
Search Engine
The Dave and Buster's Anomaly
I think it's supposed to be on the phone. I mean, this is one of the things that Apple advertises is that their AI is supposed to be more private because it's running on your phone.
Search Engine
The Dave and Buster's Anomaly
A lot of people have thought this new iOS is kind of like their worst release ever. Here I am getting uninvited from any Apple event ever again. But like they demoed all this stuff, like the summaries. Have you seen all the summaries people have shown? You get broken up over text and it gives you the Siri summary of like... Sorry, they can't. Don't hate them.
Search Engine
The Dave and Buster's Anomaly
Right. So it just makes me think, like, they kind of rushed a bunch of this AI stuff. So my first thought is, is the AI model, for whatever reason, choking on the Dave and Buster's ampersand?
Search Engine
The Dave and Buster's Anomaly
We do this thing called escaping, where you basically say, hey, treat this as something that's displayed as an ampersand to a human. Don't interpret it as an ampersand. And so maybe it's just forgetting to escape this ampersand in this one particular circumstance for whatever reason, if it's Dave and Buster's.
Search Engine
The Dave and Buster's Anomaly
I've got an iPhone here that's hooked up to my computer and I could try it here. We could see whether or not we can get to throw an error that we can see. So you see, I've got just this test phone that's got nothing on it.
Search Engine
The Dave and Buster's Anomaly
Yeah, it's a development phone. It's a totally clean iPhone 14 with a brand new install of 18.3.1. It's set to developer mode. And so it's hooked up to a tool called Apple Instruments, which allows me now...
Search Engine
The Dave and Buster's Anomaly
Yeah. And so we're going to see maybe if something crashes, we might get lucky. And as it dies, it might say, oh, I'm dying.
Search Engine
The Dave and Buster's Anomaly
And so I am transcoder agent. This is probably a pretty good one for us to look at, right? Mobile SMS looks like a good one. There's info debug.
Search Engine
The Dave and Buster's Anomaly
I'm not a professional in this. I just want to point that out.
Search Engine
The Dave and Buster's Anomaly
Exactly. Yeah. Don't do this. That's it. There are people who do this all day, every day.
Search Engine
The Dave and Buster's Anomaly
One of the things they've done is they've created security protections where if something bad happens, they can try to protect that if you hack one of those subsystems, that hopefully they can contain that hack and keep them from taking over the entire phone.
Search Engine
The Dave and Buster's Anomaly
So there's a company in Israel called NSO Group that sells these hacks and is very good at it. You know, the Russians do this, the Chinese do this. So the Chinese government's actually quite good at this.
Search Engine
The Dave and Buster's Anomaly
So there's what are called interaction and non-interaction hacks. So there have been ones that are so bad that the message is delivered and in the background, your phone parses the message. They take over your phone and then they delete the message. You don't even know you got hacked. And then they plant the malware on your phone.
Search Engine
The Dave and Buster's Anomaly
And now they can read all your messages, read your email, and even in some cases, turn on the microphone, turn on the camera, track your GPS location and such. And that's been used against democracy activists, it's been used against journalists and such. It's really bad.
Search Engine
The Dave and Buster's Anomaly
I mean, it's possible. It is possible. This is this, that this is, if you pull this thread, you find a big hole in the sweater on the other side. You know, this kind of bug, it's not super likely, but it is possible. I have seen more minor things than this turn out to be a highly exploitable condition. So yeah, it is possible.
Search Engine
The Dave and Buster's Anomaly
This feels like around seven seconds here. But back to Alex's tests. There's a bunch of errors getting thrown around the IAM transcoder agent.
Search Engine
The Dave and Buster's Anomaly
None of the errors here, it gives us a perfect, but this is pretty good.
Search Engine
The Dave and Buster's Anomaly
You know, there's a saying that you hear software people say, it's turtles all the way down, right? Like, you know, something I tell my Stanford students is that security is an incredible field to get into because it's the only part of computer science that gets worse every year, right? Like every part of CS just magically gets better, right?
Search Engine
The Dave and Buster's Anomaly
Like graphics and compute and storage, but systems get more complex, right? less understandable and more important every year. And so as a result, systems get less safe and there's more need for people to break them and make them safer. And I think AI has just massively multiplied that. I mean, this is one of the weird things about AI. Just...
Search Engine
The Dave and Buster's Anomaly
Theoretically, AI systems are supposed to be what's called deterministic, right? So a deterministic system is a piece of software where if you know the inputs, you can predict what the outputs are. In practice to human beings, modern AI systems are non-deterministic. We have no fricking idea why they work.
Search Engine
The Dave and Buster's Anomaly
Like we just build these things and we train them on these huge training sets and then they just kind of happen. Right? Like, they just kind of do things.
Search Engine
The Dave and Buster's Anomaly
Like, we are building software systems that are beyond human comprehension. And we're throwing them in our pockets and then building our lives around them.
Search Engine
The Dave and Buster's Anomaly
And this is another thing I tell my students. It is the most exciting time to be in security since the late 90s because, once again... new kinds of vulnerabilities and bugs are being discovered every day of new entire classes of issues, right? This is what it was like when I was young.
Search Engine
The Dave and Buster's Anomaly
You'd go to a security conference, you'd go to a DEF CON or Black Hat, and you'd go to a talk, and somebody would get on stage and they would talk about some new research. You would leave and you'd be like, wow, I think every single product on the planet is vulnerable to that bug because nobody's ever heard of it, right? And that's what AI is like right now, is that
Search Engine
The Dave and Buster's Anomaly
You know, somebody might say we built a secure AI system and you're like, you can't make that promise because nobody knows what the vulnerabilities are in these systems yet. Like just the fundamental research hasn't been done yet. And so it is like both a terrifying and really fun time to be alive if you're in this field.