Last week, Google announced a fundamental change to how the site will work, which will likely have dire effects for the news industry. When you use Google now, the site will often offer AI-generated summaries to you, instead of favoring human-written articles. We talk to Platformer’s Casey Newton about why this is happening, why publishers are nervous, and about a secret new internet you may not have heard of, a paradise to which we may all yet escape. Support the show at searchengine.show! Search Engine - How do we survive the media apocalypse? (Part 1) Platformer - Google's broken link to the web 404 Media - Why Google is shit now To learn more about listener data and our privacy practices visit: https://www.audacyinc.com/privacy-policy Learn more about your ad choices. Visit https://podcastchoices.com/adchoices
Surge Engine is sponsored by Viore. Viore is a new perspective on performance apparel. It's perfect if you're sick and tired of traditional old workout gear. Everything is designed to work out in, but it doesn't feel or look like it. It's extremely comfortable. You'll want to wear it all the time. I promise you it is more comfortable than whatever you're wearing right now.
The product is incredibly versatile. It can be used for just about any activity, running, training, swimming, yoga, but it's also great for my favorite form of exercise, which is lounging on a sofa. Also, Viore is 100% offsetting their carbon footprint. They're using better sustainable materials for their products to empower your best active life. Viore is an investment in your happiness.
For our listeners, Viore is offering 20% off your first purchase. Get yourself some of the most comfortable and versatile clothing on the planet at viore.com slash pjsearch. That's Viore, V-U-O-R-I dot com slash pjsearch. Not only will you receive 20% off your first purchase, but you'll also enjoy free shipping on any US orders over 75 bucks and free returns.
Go to viore.com slash pjsearch and discover the versatility of Viore clothing. This is Search Engine. I'm PJ Vogt. No question too big, no question too small. Each week, the small staff of our show meets in a sunny office in one of the tall buildings in New York City's least charming neighborhood, and we try to decide what we should pay attention to.
It could be anything, which is sort of tricky, actually. often we settle for trying to understand and explain very recent history. Stories that have unfolded in the past few years, which, with the benefit of hindsight, we can now understand more clearly. The rise of fentanyl, the fall of Sam Bankman Freed.
There's one story, though, we keep bumping into this year, a story that we're in the beginning or maybe the middle of, which I find myself too curious about to resist trying to understand as it unfolds. A couple months ago in March, we spoke to journalist and fellow podcaster Ezra Klein. The question we posed to him was, how do we survive the media apocalypse?
At the time, all these online news outlets were dying. BuzzFeed News had been killed. Traditional newsrooms like the Washington Post and the LA Times were shedding staff through layoffs and buyouts. And as a person who loves reading human-written, fact-checked sentences on the internet, who depends on those sentences, I felt alarmed.
I wanted to understand this moment, and I wanted to hear ideas from smart people about how to prevent it. Ezra had insights. He had suggestions for how readers could push back. If you haven't listened, please check that episode out. But since then, our apocalyptic moment, it has just kept rolling on.
The scenes from this apocalypse are so bizarre and spectacular, I sometimes can feel myself disassociating, like while I was watching this video last week. In California, on a psychedelic stage, a YouTuber slash DJ was crawling out of an oversized coffee mug while wearing a rainbow kimono. The DJ then started howling the name of the company whose event he was opening.
Google! Time to get up, you silly little nerds. Wake up!
This is how the world ends. Not with a bang, but with a DJ set from an internet personality. This was Google's annual developer conference, Google I.O., the event where every year, Google announces which technological breakthroughs the company has in store for us. 21 minutes later, after the DJ had tuckered himself out, the show began in earnest.
Welcome to Google I.O. It's great to have all of you with us.
The moment that shook me, that shook a lot of people, it came after CEO Sundar Pichai had made his opening remarks. He introduced Google's head of search, who walked on stage to funky elevator music.
Thanks, Sundar. With each of these platform shifts, we haven't just adapted. We've expanded what's possible with Google Search.
Liz Reid explained that Google search was about to fundamentally change in a way unlike anything anyone had seen in the last quarter century.
Now, with generative AI, search will do more for you than you ever imagined. So whatever's on your mind, whatever you need to get done, just ask. And Google will do the Googling for you.
Google will do the Googling for you. These seven words, I'm not kidding. They made me feel deeply uneasy in a way that announcements at tech conferences rarely do. What Liz Reid means when she says Google will do the Googling for you is that from now on, frequently when you ask Google a question, what's the best Bluetooth speaker or what's happening with the war in Ukraine?
Instead of being sent links to articles written by humans, the AI will read those articles and just provide you with its own summary. There will be links in the footnotes, but you can skip them. On its face, a totally useful feature, but as we watched the media apocalypse arrive, this seemed like a pretty obvious accelerant.
Almost without exception, every website on the internet depends on Google for traffic, for audience. Google now seems to be saying, that highway we've built, we'll be closing the exits.
A report from the Wall Street Journal suggested that online publishers, the average of which is already limping and coughing like a 20-year-old cat, could now expect to lose as much as 40% of their remaining traffic. I wanted to talk to someone who could explain all this, how we got to a place where Google defines so much of the internet, and what to make of this new change.
So, of course, I called up Casey Newton, the genius tech reporter behind the Platformer newsletter, co-host of the Hardfork podcast, I wanted to talk to Casey because I knew he actually had some very different ideas about possible solutions to this apocalypse. And besides, if I was going to watch the world begin to burn, I knew who I wanted to sit next to. Hello? Casey, how's it going? I'm good.
Am I coming through okay? Yeah. You're coming through loud and clear. I think it's even recording your voice. Good, because it's one of the most important parts of podcasting that I've learned. Without it, it's just purely a theater of the mind.
If this airs, will I be the first returning three-time champion on Search Engine?
Not only will you be the first three-peater, Kelva and Ezra... I mean, Kelva's been very vocal about wanting to come back three times before Ezra, and I don't think he saw you coming in from the other lane and knock him out.
I'm a dark horse, just like that one Katy Perry song.
So, okay, before we get to this week's news, can you just give me the prehistory? Like, can you tell me the story of Google search? Yeah.
So PJ, when the internet was young and exciting, it was just a series of webpages, famously delivered over a series of tubes. And these webpages were so vast in their number that to find them, we needed a box we could type into. And there were many boxes with names like Excite and Hotbot and Infoseek.
But one day, a couple of Stanford grad students come along with this thing that is better at searching these webpages than anything we've seen before, and it's called Google. And basically from the minute anyone sees it, people are saying, this is the one. They've come up with some really clever stuff that helped them find webpages better than anything else.
And the story of the next 25 years is Google gradually wrapping its arms around the web until it essentially became synonymous with it.
And why? I mean, I remember, like, I'm really not enjoying how often I find myself saying I'm old enough to remember in a non-ironic way. But I am old enough to remember the other search engines. Like I remember AltaVista. I remember Ask Jeeves. I remember using like AOL search.
And I remember the feeling I got when I first used Google, if I remember it correctly, was almost the feeling you get with like a good like chat GPT type product. We're like, oh, this is better. This feels different. What was happening under the hood that made Google work better than what preceded it? Yeah.
They did a bunch of things, but the most famous is something called PageRank. PageRank, named after one of the co-founders of Google, Larry Page. And the idea was really simple. It was just that as they created this index of all of the web pages, they would look to see which web pages were linking to other web pages.
And if a bunch of webpages link to that webpage, that was a really strong signal that it was valuable. So if as they're crawling, they see 100 different webpages linking to the New York Times, that's going to rise up in search results as people search for the New York Times. And in fact, it is the website of the New York Times. And so everybody sort of gets what they want.
So that was a really good and useful thing. And it enabled them to become the default search engine for most of us. And after that, it turned into basically the greatest advertising business that anyone's ever seen. And why did it turn into the greatest advertising business that anyone's ever seen? Because it turns out that what search engines do is they capture intent and desire.
If you are typing in new car, you might be in the market for a new car. If you're typing in shoes, you might be in the market for new shoes. And so really quite easily, Google could just go out to people who wanted to advertise to people who were in the market for various products and services.
And it started, it actually had this idea of running an auction so that advertisers could bid to be above all of the search results. And it just worked incredibly well. And I would argue it was actually just a really fair bargain for anyone. If you are looking for shoes, it probably actually doesn't hurt you too much to see one ad for shoes on top of a list of links to other websites.
So I think it's important to say that the first decade or so of Google, while it had various problems, it just fundamentally felt like a good bargain for everyone. People got to the webpages they were going to, it was paid for by ads, and that seemed fine.
Yeah, it's funny. Now we have so many—I think it's not true, but there's that idea that Eskimos have so many words for snow. We have so many words for either technology making us feel bad or capitalism behaving in ways that we feel conflicted about.
And people talk about extractive models, and there's all these web products where it's like, you like it, but it's doing something to someone that's bad. Right. It's offering you something, but it's like pulling something out of your back pocket while it's doing it.
And you're right, in the early stages of Google, in the first chapter of the company's history, it's like, this is great for everybody, actually.
Yeah, and we should say it really helped the web grow and establish itself. Google made the web much more useful, and the more useful the web became, the more people rushed into it. Google started showing ads on other websites.
And so if you were a publisher or even just a blogger that had decent traffic, you could just run ads that Google would manage, and you could begin to make money on the web as a creator. So you just see this huge rush of talent and capital into the web as Google leads the charge in making it more useful for all of us.
And so another question that I've always wanted to ask someone who would know, there was a moment where it was like there's a bunch of search engines and Google is the superior one. What happened that Google became, like, I know not literally the only one, like one could use Bing, but why did Google pull out so far ahead and never get caught?
A big reason is just that the more that people used Google, the better that it got. So for example, I used an example earlier of somebody trying to find the New York Times website, and Google starts out with this thing, PageRank, that says, we actually have a pretty good idea of what you're looking for right now.
Well, then think about all of the people who start visiting Google and they search for the New York Times and they click the link and they go to the New York Times and they don't go back to Google. And Google says, aha, we served them the right link. And it starts feeding that model. And it does that across every category of search for every single thing.
And so all of a sudden, Google has the most accurate index by far of any of the search engines. And the longer that it goes, that just becomes more and more true. So it starts to gain this momentum that nobody else can really match.
And at what point does the news industry, does the media industry start to enter into this relationship with Google?
Yeah, so from the start, people were trying to figure out how do we optimize our webpage so that it floats to the top of these Google search rankings? Because as Google becomes... A default place to start your day on the internet, one of the things people are doing is searching for news.
And so publishers, they're changing the HTML, they're talking with people at Google about what exactly are you looking for, and it becomes this dance. And there are some players in the game, like I think probably most of the publishers were, that were pretty good actors.
And then there were a bunch of unscrupulous fly-by-night characters that were just trying to sell you a vacuum or whatever and wanted to swarm every keyword you can imagine just on the off chance that maybe their webpage would get to the top of the search results. And so it becomes this very competitive adversarial thing.
And an effect of that was Google just became increasingly powerful because basically it's not just the publishing industry. It's like every industry is beating down a path to Google's front door saying, hey, how do I get to be the top link on the thing? And that becomes like one of the sort of main drivers of Google eventually taking over the web.
It's just such a strange thing. Like it happened and so it seems normal, but it's weird to contemplate the idea for how infinite the internet is that really the most normal experience you would have on it is you search something on Google and you visit one of three to five links or you go on one of the handful of social media websites and then that's it.
Yeah, I mean, Google did do things to put itself at the center of the news conversation. The first thing it did was it had a product called Google News, where it just started to aggregate headlines. And you can visit Google News and get a rundown of what was happening around the world.
Another thing that it started to do, and this happened much later in the mobile era, but eventually by the time you Google something on your phone, even before you search for anything, Google would know your previous searches and they would show you news stories that you might be interested in. And all of a sudden, that was starting to send a flood of traffic people's way.
A third thing that happened was that publishers just started to pay attention to what people were searching for. There are various tools that let people understand, oh, wow, a lot of people are searching for the Game of Thrones trailer. It's going to take us four seconds to embed the Game of Thrones trailer in our website.
Let's just go ahead and do that so that when anybody searches Game of Thrones trailer, maybe we'll rise to the top and we'll be able to gain that ad revenue. And here's where I do think the publishers just made a mistake because there was a lot of easy traffic that was available. The...
output wasn't actually that high in journalistic quality, but the revenue that was coming in from all those visits was like pretty good. And so this dynamic was just created where these big digital publishers just saw this ocean of traffic available to them.
And all they had to do is figure out what are people searching for and what's the like cheapest webpage that we can quickly get up to take advantage of the traffic.
Yeah, and you know, it's like sometimes as a person who has worked in online media for my entire adult life and spends most of my time thinking about what tech companies have done wrong and not what media companies have done wrong, it is funny how much in that chapter of internet, how much of what got published would just be every single website, whether they're a video game review site or a national newspaper or a blog or whatever, would just be like, hey...
The people on the highway of the internet today want to look at the trailer for the movie. Let's slap that on our website. Like, it was so undifferentiated. Everybody posts, like, the John Oliver clip. Like, everybody posts Saturday Night Live. Like, just everyone's selling the same product with very little differentiation.
The example I always think of is, it felt like a decade ago, every single news site was writing articles just called, What Time is the Super Bowl? Do you want to tell that story? Yeah.
Yeah, one of the most popular queries on Google, as you might imagine, is what time is the Super Bowl? Because that is a day, I'm told, when people who do not ordinarily watch football games will watch a football game. And they don't know what time it is, PJ. Without looking, do you know what time football games are on?
Is it 4 p.m.?
Is it 5 p.m.? When's the kickoff show?
Nobody has any idea.
So the Huffington Post realizes that it can write an article that answers the question, what time is the Super Bowl? And it will be a traffic bonanza akin to the Super Bowl itself. The what time is the Super Bowl post is the Super Bowl of SEO traffic for the Huffington Post.
We can just put that on the web too.
And you can probably guess what the ultimate conclusion of the story, which is that Google says, we also know what time the Super Bowl is. We're just going to start showing it on top of search results.
And it is that shift, Google sort of realizing if what people are looking for from us are just answers, we don't have to leave it to the Huffington Post and all these other hangers on to answer people's questions. We can just start doing it for them.
And if you are the frog in the pot of water that the entire media industry has been for the past 25 years, this is when the temperature went up by five degrees.
After the break, the temperature keeps rising. As the quality of search results declines over the years, as websites become generally crappier in an effort to get noticed by Google, the death spiral continues. More death spiral after these ads. Search Engine is brought to you by Fresh Direct. I really like FreshDirect.
When I go to the grocery store, for some reason, my brain goes into screensaver mode, and it takes me forever, and I get lost, and I come back with half of the things I actually need. So I end up saving hours a week just using FreshDirect. FreshDirect is farm to kitchen, food sourced directly from farmers, fishermen, and ranchers, and delivered straight to your door.
Because of this, I'm paying for the quality of food, rather than just paying someone to go to a store for me. FreshDirect says they are seven days fresher than the grocery store. And honestly, you can tell the difference. The convenience is unbeatable. You can grocery shop from your office or your couch whenever you want. The most recent thing I ordered from FreshDirect was a fennel bulb.
I mean, I had other stuff too, but the fennel was really good. For over 20 years, Fresh Direct has been delivering the freshest fruits, vegetables, and meats to the tri-state area. Don't take my word for it. Try to believe it with $50 off your first order. Go to freshdirect.com and use code SEARCHENGINE. That's freshdirect.com, code SEARCHENGINE, for new customers to save $50 on their first order.
Terms and restrictions apply. See site for details. Search Engine is brought to you by Rosetta Stone. There's a lot of reasons to learn a new language. One of my favorites, cognitive benefits, improved brain function. My brain has not been very good. Studies have shown that learning a new language can improve memory, problem solving skills, and even delay the onset of dementia.
It might be too late for me, but it's not too late for you. Rosetta Stone is the most trusted language learning program available on desktop or as an app. It truly immerses you in the language you want to learn. Rosetta Stone has been a trusted expert for 30 years with millions of users and 25 languages offered.
Lifetime membership has all 25 languages for any and all trips and language needs in your life. Don't put off learning that language. There's no better time than right now to get started. Search Engine listeners can get Rosetta Stone's lifetime membership for 50% off. Visit rosettastone.com slash search engine. That's 50% off unlimited access to 25 language courses for the rest of your life.
Redeem your 50% off at rosettastone.com slash search engine today. Welcome back to the show. So Casey had told us the story of the what time is the Super Bowl era, the chapter in which august American news outlets were competing against each other to be at the top of a predictable annual Google search.
I remember when Google changed its website so that Google itself could just tell you what time the Super Bowl was. And I remember thinking, that makes sense. I'd understood why the publishers had wanted the web traffic, but a news industry designed to tell you what time the Super Bowl is is just not that healthy of an industry. So no big deal.
I was not savvy enough, however, to notice what it might mean as Google gradually put more and more of the information that would have lived on various websites onto its own front page. When you saw them start to answer those questions themselves, as someone who studies the power that platforms have relative to the industries that depend on them, did you make note of that shift?
Yes, but only in the sense that I thought, well, here is a place where Google's power is increasing. I've been writing about Google for more than 10 years, and I would say the whole time they've been trying to figure out how can we answer more people's questions on what they call the SERP or the Search Engine Results Page. It's an acronym.
And to me, one of the most interesting statistics about Google over the past two decades is the rise of what they call the zero-click search. which is the search that does not result in any outbound traffic to anything. You sort of flash back to the first days of Google.
I would guess that almost every search resulted in a click to somewhere because Google itself didn't know anything except for maybe where the webpage was that you were looking for. But then you get into the 2010s, and all of a sudden, it's not just answering what time is the Super Bowl, it's pulling snippets out of Wikipedia. It is telling you what movies actors are in.
It's telling you what movies directors have directed. And all of this is appearing in various little boxes and carousels on top of the classic 10 blue links that have always been the heart of Google.
And so, yeah, what I noted over the past decade was every year, there's another box, there's another widget, there's another answer, and there's one fewer what time is the Super Bowl bonanza for publishers to count on.
I mean, I feel like the other experience you could have in the last 10 years on Google was that...
Sometimes when you search something that could appear in one of those boxes but didn't appear in one of those boxes, you would end up on a website that gave you that information but had been so designed for Google that the experience of actually landing on that website... The example that I see people refer to a lot is recipes, where for whatever reason the Google algorithm decided it liked longer articles.
The most privileged link would not be a recipe for tomato soup. It'd be like... a five-page essay about what tomato soup meant to someone and their grandmother who gave them the soup recipe and blah, blah, blah, blah, blah. And you're scrolling down and you're like, why is this written this way? This is completely insane, but it's written that way for Google.
And at that point, you're like, could Google just please tell me what tomato soup is made out of? I'm pretty sure it's tomatoes.
Right. And it is a great example because we all ran into it and we were all annoyed by it. And this was just one of many things that Google did to take over the web experience. They also created the Chrome web browser. The Chrome web browser helps to dictate HTML standards, how web pages are built, how browsers interpret them. It's able to exert pressure in that way. So it's not just...
Like in what order do links appear on webpages? Like Google is actually dictating the shape of the web itself through all these different things.
It's weird. It's like, I can't, I always try to come up with metaphors for things. And I'm like, I want it to be like, okay, like the American highway system, like the highway system, which is just meant to connect towns. Eventually, you know, it's like people put up stores on the highway and the highway itself reshapes it.
But I feel like what happened with Google and the internet is more than that. Like, I feel like it's like as if, the map became the thing instead of the thing it was describing. Do you know what I mean?
Yeah, they built like the greatest highway system that the internet had ever seen. And then over time, it is just shrunk to the size of a parking lot. And anybody who searches it is just like driving around in a circle in the parking lot. And why did that happen?
Because like everyone has experienced that, but what made that happen?
I mean, I think my answer to that would be that Google just wound up being arguably the biggest economic victor from the internet in terms of certainly the amount of digital advertising revenue that they were able to generate from the internet. And digital advertising revenue is like the single biggest category of revenue, I think. Is that true? Well, I don't know. We should look that up.
We'll look that up.
We did look this up. We're professional podcasters. Alphabet, Google's confusingly named parent company, made $307 billion last year. Google Search alone accounted for $175 billion. The New York Times, by comparison, $2.4 billion.
The publishing industry, it was really amazing when it was just like newspapers, but nobody was making $100 billion a year, right? Like Google was able to just sort of go out and over the years, more and more of the advertising revenue just accrued to Google. Like Google just became like the most powerful thing and publishers just became... disempowered. They laid people off.
They scrambled, you know, whatever they could do to like get up high on those search results, they would do. It would work for a time. Then the algorithm would change. Then, you know, more people would be laid off. Like all of this just had like a downward pressure on the quality of things. Like people couldn't afford to take big swings anymore. They couldn't afford to to hire big staffs anymore.
And so you just get more of these like generic websites telling you about that week's movie trailer. So like basically Google got too much of the money and like the rest of the digital media ecosystem, in my opinion, did not get enough. And there's a very solid narrative that has unfolded over the past few years that Google just isn't as good as it used to be at searching for things.
In part, that is just because there are so many more ads now on the sort of high-value searches that people often do on Google. A predictable way that Google has added revenue over the past couple years when they need to show growth to Wall Street is they'll literally just add one more sponsored link to mobile search results.
So maybe it used to take five links before you would see what they call an organic result, so a result that has not been paid for. Now I think it's up to seven, right? Now maybe most people don't even realize that those are sponsored links and they're perfectly happy to click on ads all day.
But for people who are a little bit savvier and who just kind of wanted to see a web that wasn't totally corrupted by commercial values, that just feels like it is harder to find. And it is in part because people are not making as much stuff for the web as they used to because there is not as much money in it.
Right. It's funny. It's like Google's really two businesses that even within themselves are sort of in competition. There's like the search part of it where we're serving people who want to find stuff on the internet.
And the advertising part of it, which is like we want people who are trying to find stuff on the internet to get distracted on their way and stop at our store or stop at the advertiser's store. And it's like I wonder if even within the company they feel like search and ads are in combat. Yeah.
They always have been. I mean, this is kind of what led a former Googler in the early days to coin their famous catchphrase, don't be evil. This was what don't be evil was about. It was about not compromising the integrity of what they were doing by reaching for the easy revenue.
And over time, I think that they have just reached more and more for the easy revenue and have not thought enough about the health of the broader web ecosystem that ultimately they do depend on.
So tell me about this most recent news. What happened?
So this week at Google I.O., they laid out some changes to the way that search results will work. And there's the way that it will work in the near future, and then there's the way that it will work in the medium term.
The way that it's going to work in the near-term future, and in fact, a lot of people have had this feature already in preview, I've had it for several months now, is when you search for some things, Google will just show you an AI-generated summary of the results. So if you say something like, what's the best laptop I can buy right now?
Before this feature rolled out, you would see a list of links to sites like Wirecutter that had done a lot of rigorous testing of laptops. Now, with what Google is calling AI overviews, it'll say like, here are some of the best laptops of 2024 as judged by experts. And they'll sort of look at
50 different companies that have written a page like this and they will summarize it and they will sort of show you in little footnotes maybe who wrote that story. But most people, of course, are not going to click on those footnotes. They're just going to see a little summary. So why does that matter? Well, This is one of the places where publishers are still making money.
They're able to put affiliate links if they do these sort of wire cutter style tests of products. And if people buy something because they read that web page, then the publisher gets a little bit of a kickback. now those kickbacks are probably gonna start going away too. And so this is just, again, one more place where publishers aren't gonna see revenue.
But it's actually much bigger than that because the real idea here, PJ, is that whereas browsing the web used to be considered something of a pastime to older folks like you and me, now it's being sort of presented as a chore, something that you shouldn't have to do. something that you should just let Google read the web for you, show you a bunch of results, and you'll never have to leave Google.
So the reason that this is so important is this is really the first step toward you not having to visit the web anymore because Google is going to read the web for you. And like,
I feel like what you just said is, let me put it this way. Put it this way. Recently, I had seen a different search engine. I think it was Perplexity that was sort of doing the same thing. And we talked about this on Search Engine. We were talking with Ezra Klein. It was like, this seems kind of bad. They're like taking the journalism, but they're not paying for it.
What's going to happen to underlining journalism? That seemed like a moment. Google unveiling this functionality. Would you say this is a bigger deal?
Yes. So I hated what Perplexity was doing. I hated what ArcSearch, another company, was doing that was basically exactly the same thing. And the reason I hated it so much was that I knew that Google would do it. because in some ways it is a better user experience, right? There's a reason that people really like asking ChatGPT questions.
And it is that they do not get a big research project back when they say, show me the best shoes, right? ChatGPT will just say, oh, if you're a man, here's like 10 kinds of shoes that should be in your wardrobe. Google will show you 4,800 links to websites. It's clear to me what is the better user experience, right?
So I knew when I saw what Perplexity and Arc and some of these others were doing that Google was going to feel pressure to do the same thing. But still, it had to happen. And then this week it happened.
I wondered if Google wouldn't do the same thing because I feel like, okay, as a journalist, I understand why I don't want this to happen because it makes us less valuable. It makes our jobs more precarious. As an internet user, I understand why people... I have used perplexity. I have found perplexity to be useful, but I also feel like I'm like...
pirating music, except for the thing I make, so I feel worse about it. Sorry, musicians. I thought Google might not do it because I'm like, yeah, in the short term, you're giving people a better user experience. But if you roll this thing out, and I don't know how to estimate how many journalists lose their jobs from this, but 10%, 20%, 40%, you're killing the input for the machine that you need.
And it seems that Google needs that machine more than a perplexity or an arc. So I thought maybe they wouldn't.
Yeah, and I do think that they will move a little cautiously here because to some degree that is almost certainly true. If truly every publisher in the world went away and restaurants stopped creating websites and dry cleaners stopped posting their phone numbers online, this does create a problem for Google.
I'm just not sure, one, that it's as big a problem for them as a journalist I would like it to be. And two... they are going to be in control of this entire process, right? Like they have their fingers on the knobs and the levers. And so they can just tweak it like 5% this way or 10% that way. They can see what happens.
And if nothing really breaks for them, then they can dial it another 5% or 10%. Every other business on the internet might be kicking and screaming the whole time, but there is almost truly nothing they can do because Google is in control.
So to me, what this moment has meant is that on stage this week at Google IO, the company essentially put the web into a state of managed decline where they said, without saying it, that the web was really useful for 25 years. but we don't need it anymore.
Because with generative AI, we'll be able to tell you anything that the web could have told you, and you're not even going to have to leave Google to get the information.
You're not somebody who is like... One of the things you and I talk about sometimes, we're not talking into a microphone, but I guess also talking into a microphone, is that like... To be clear, we mostly talk to each other via microphones.
Mostly via microphones. Because that's just the relationship you've set up for us.
But I feel like we're both... Tech journalists in our generation, for the most part, tend to be incredibly skeptical of tech companies, incredibly paranoid about what they're releasing, and with good reason. That's earned skepticism, earned paranoia. I feel like you and I are a little bit unusual in that we're still sort of stubbornly- We have some optimism in us. We have some optimism in us.
My feeling about AI has been like, I'm not just going to try to go destroy the machines with an ax. I want to see how this is good. I want to see how this is bad. And maybe this is just my solipsism where it's like, well, you weren't worried about Dolly, but now you're upset about this. But like, it's unusual for me to hear you talk about things in this dire way.
Yeah, and I'm a little nervous that I am over-rotated here, right? And yet, if you look at the trajectory of the journalism industry since I got into it in 2002, it pretty much just is a line falling off a cliff.
I want to say, correlation is not always causation.
Not always causation. And I'm not saying it's all the internet's fault. I'm not even saying this was Google's job to fix this necessarily. It just did become the economic engine that powered the web. And so the moment when it says, This honestly just is not that important to us anymore.
Regardless of what you think of whether that is good or bad or what Google should have to do, it just is a big deal for publishers. There's been some reporting on this in the Wall Street Journal. And analysts believe that publishers might lose between 20% and 40% of their traffic over the next year as this stuff rolls out, right? Because we should say, what happened this week was Google...
took this AI overview experience that they've been testing. They've now rolled it out across the United States. By the end of the year, they say a billion people around the world are going to have it. So it's gone from this very small test to now a billion people are going to have it by December. And
Once that happens, if people are really losing 20 to 40% of their traffic, we're just going to see so many more publications go out of business. Last year, a bunch of publications went out of business. BuzzFeed News, Vice as we know it, The New Gawker, Protocol, sites that just kind of disappeared.
And when I think about the few and the proud big publishers that remain, if you walked into any of their C-suites and were like, what's your plan to have 40% less traffic by December? I don't think anybody has a really good plan for that.
After a short break, if Google shutting down a huge chunk of traffic to news sites is as big a deal as Casey fears, what are the possible solutions here? How can someone putting their work on the internet safeguard their ability to make a living? That's after these ads.
First, the bad news.
SAP Business AI won't help you generate cubist versions of your family's holiday photos, but it will help you understand which supplier is best to help you roll out your plant-based packaging in Southeast Asia, identify the training your junior project manager needs to rise up the ranks, and automate repetitive tasks while you focus on big innovations so you can be ready for the next opportunity.
Revolutionary technology, real-world results. That's SAP Business AI.
Search Engine is brought to you by Seed. Whether you're off to the pool, hiking, or traveling this summer, you're bringing your microbiome with you too. The 38 trillion bacteria that live in and on you, especially your gut, are essential to whole body health. Seed's DS01 Daily Synbiotic benefits your gut, skin, and heart health in just two capsules a day.
Your body is an ecosystem, and great whole body health starts in the gut. Your gut is a central hub for various pathways through the body, and a healthy gut microbiome means benefits for digestion, skin health, heart health, your immune system, and much more.
With clinical trials and breakthrough research published in top scientific journals, Seed's probiotic research and development makes DS01 a product you know you can trust. Support your gut this summer with Seed's DS01 Daily Symbiotic. Go to seed.com slash search and use code 25search to get 25% off your first month.
That's 25% off your first month of Seed's DS01 Daily Symbiotic at seed.com slash search, code 25search. Search Engine is brought to you by Mint Mobile. You know when you discover a new binge-worthy show or a song that you bump on repeat and have to share with your friends so they can experience just how awesome it is?
That's kind of what it feels like when you discover that Mint Mobile offers premium wireless for $15 a month when you purchase a three-month plan. It's such an awesome deal, there's no way to keep it to yourself. Normal cell phones cost over $100 a month for a plan. 15 bucks a month is a great deal. Friends don't let friends overpay for wireless.
Say bye-bye to your overpriced wireless plan and switch to Mint Mobile. To get this new customer offer and your new three month premium wireless plan for just 15 bucks a month, go to mintmobile.com slash search. That's mintmobile.com slash search. Cut your wireless bill to 15 bucks a month at mintmobile.com slash search. $45 upfront payment required, equivalent to 15 bucks a month.
New customers on first three month plan only. Speed slower above 40 gigabytes on unlimited plan. Additional taxes, fees, and restrictions apply. See Mint Mobile for details. Welcome back to the show. I have to say, I was feeling unusually defeated hearing Casey describe what he thought was about to happen with the internet. I wanted him to walk me through the possible solutions here.
How other people were thinking about the problem of these AI bots chowing up the work of human beings and spitting out good enough summaries. Most things are fixable. I wondered what the possible fixes were. For instance, the most tried and true American response to anything. Lawsuits. Couldn't journalists and publishers just sue Google or sue whoever's napstering their work?
So there is a big lawsuit. filed by the New York Times against OpenAI, essentially for the same reasons, and that is now unfolding. It is an open legal question whether it can be permitted for a company like Google to go in and look at all of the articles that a publisher like Platformer has ever published and use those to train a large language model.
The case for it being illegal is, hey, you're stealing my work. Knock it off. Don't do that. You're just like taking my labor and using it to make something valuable. I don't want you to do that. So maybe it's illegal. The case that it is legal, though, is, I don't know, all of us are allowed to go and read webpages and form thoughts and do other things based on that information.
And these LLMs are not reproducing what they are ingesting perfectly in most cases. And so are you really going to tell computers that they can't read the internet? Because guess what? Computers are already reading the internet in all sorts of ways. So this is just going to have to get litigated.
But of course, all of the big tech companies are just making the bet that courts are going to side with the big corporations here and the publishers are going to be out of luck.
Right. And it seems like of all the possible futures you could imagine, there's ones where the courts are like, you do kind of have to pay them some money. There's ones where they say, no, no, no, this is aggregation. This is what you guys were doing with humans. But there's no world where they say, turn off the machine. Right.
Some people do think that. Some people will say that ChatGPT and Google Gemini are like the fruit of a poisoned tree, that because you used all this copyrighted material in the creation of them, you will actually have to destroy them. But vanishingly few people I've spoken to think that is a likely outcome.
You're a person who publishes on the internet. You're a publisher. You're not an enormous company, but you're someone who is swimming in the same ocean that you're worried is being destroyed. What does it mean for you?
I can't figure that out yet. I had an opportunity back when Platformer was still on Substack. If you want to know why we're not there, that's another great story that you can Google. For now. And they had some kind of toggle where I could say, don't train your LLMs on Platformer, you rogues. And I didn't switch it on. Because number one, Platformer publishes like three articles a week, okay?
We're a very low volume publisher. We've never, frankly, even relied on Google for very much of our traffic. I started a newsletter because I didn't want to have to fight platform algorithms for the rest of my life. I just wanted to write about them. So I've never thought that... whatever happens in Google is necessarily gonna be like curtains for a platform.
In fact, sometimes I think I'm weirdly worked up for this given how little I actually expect this to affect me in the near term. Except that I love the web. I grew up on the web. The web brought me like everything that I have.
And Platformer is much better when there is more journalism for me to read and have thoughts about and inspire questions in me and, you know, send me chasing stories on my own. I don't think this is going to have a huge effect on me directly. But like indirectly, it feels like the only story.
Yeah.
Which is, what is the internet going to look like in five years? And you think it's that fast. Yeah. This stuff is moving very quickly. Every few weeks, it seems, one of these AI makers comes out with a more efficient version of a model or a model that has been tuned for some specific purpose. Like maybe it's better at education or it's better at science or it's better at healthcare.
And we're just kind of in that liftoff stage. And this stuff is starting to... accumulate. These chatbot assistants are getting better and better. A lot of people probably watched the demo that OpenAI had this week where they've made this assistant that has this uncannily emotional voice. It's like actively flirting with you now. I was watching their videos. It's crazy.
They have We did a story like last year about Andrew Leland, interviewing Andrew Leland about his blindness. And he was saying how he uses this app called Be My Eyes where you can connect with a human being and they will tell you like, hey, your shirt doesn't match your pants or whatever.
And one of the things OpenAI demoed was this video where Be My Eyes is now AI-powered, and they were showing a person who did not have sight walking down the street holding their phone, and their phone was saying, the cab's here. Put your hand up. Buckingham Palace is in front of you. The flag indicates the king is there. And I'm like, I'm not against this. This is progress. This is amazing.
I just want the work I love and the web I love to exist in the future.
Yeah, another way I would put that is just like, I want the benefits to be a little bit more evenly distributed, right? I know that the majority of the spoils here are going to go to the companies that do the most innovation. And I'm basically fine with that. But again, when you flashback and think about what Google was like in the early 2000s, I just feel like we had a better bargain.
We got a guide to the web that was really fast and easy to use and reliable. They got a bunch of advertising revenue. And there was a rising tide that was lifting all boats. And what I'm worried about is that tide has now sort of, I don't know, come in and washed out a lot of what was on the shore. And Google is going to be the last boat. boat standing or whatever.
I've sort of lost track of this metaphor. I don't know if Google is a boat now. It's gone completely away from me. But if you just sort of imagine them as a large thing that survived whatever I was talking about, that's what it would be.
When we talked about this on Search Engine last, and it's weird, I really feel conflicted where I'm like, when you said that this feels to you like the only story in some ways, that's how I feel. And I'm like, I don't want to belabor people's patience with my curiosity with this, but it really feels like... Global warming's a bigger deal. Climate change is a bigger deal.
But it feels like climate change for the thing I love, and it feels like it's happening so quickly, and it's very hard for me not to think about it a lot. When we spoke to Ezra about it, I was like, what do you do? And Ezra, being an ethical person who believes that people should act ethically, was like, look, if you love journalism... it is incumbent on you to pay for it. And I agree.
People should pay specifically for search engine. And if they have money left over in their budget, they should pay for platformer. Or maybe search engine twice and platformer once. I pay for platformer. Thanks, BJ. Of course. I pay for search engine. Thank you. We're modeling good behavior to the internet. That also feels like, yeah, people should do that. But like...
It feels like the problem is larger than $5 a month. What do you think the solutions are?
Well, I mean, set the journalists aside for it. I mean, this is something else I think is important to say so people don't just think we're navel-gazing about our own industry. Google does not only deliver traffic to publishers. It's how people discover all sorts of businesses, right?
You move to a new town, you need a dry cleaner, you need a dog walker, you want to know some cool restaurants or cool bars. Right now, imperfect as it is, all of those things can like jockey for a position. They can go to the search engine optimizers and they can get some tips.
And hopefully if they're a really good restaurant or a really good dry cleaner, they'll pop up to the top of search results. And they might not even have to buy a Google ad, right? Like they can compete just by being really good.
We are talking about the beginning of a future where all of those web pages, whatever all those other businesses are doing to sort of wave their hands and say, hey, we exist. That is all just getting subsumed into an even more complicated and mysterious set of algorithms that's just going to be spat out. And you're just going to be told, yeah. Here's the three dry cleaners in your town.
And hopefully, one of those will be good. So I don't want to overly romanticize the present state of affairs because I do think that SEO has ruined a lot of things. But... It still seems preferable to me to a world where there is just this kind of mystery AI giving you the answer to everything and actively discouraging you from visiting websites to make up your own mind.
And I just keep going back to like at this conference this week, they just kept coming back to this phrase, let Google do the Googling for you. And what they were telling us was, searching the web is a chore, using the internet is a chore, Google is now the thing that stops you from having to do that chore.
Google is just going to be the Star Trek computer, it's going to tell you whatever you need to know, don't worry about visiting the web anymore.
And while they protested, that's not really what we meant, and we're still going to send lots of traffic, and we believe in the web, at the end of the day, I was like, no, you're telling us what you want to do, you've been building it for 20 years, and now you're really close. So I just think it's time that we take them seriously about that.
What does it mean to take them seriously except for to worry?
I have the best, worst, dorkiest answer to that question, PJ, which is that we have to finish building the Fediverse. Really? Yes. You're already so upset that I'm making you talk about this, and that's fine.
You should be.
We should all be upset that we have to talk about the Fediverse, but that's where Google has driven us to.
Talk about the Fediverse. But in a way that my mom can understand it.
Yeah. So... Fediverse is a way for people to take back the internet for themselves. It's a way to have an identity and connect to other things that are important to you online and just not worry about having to fight through a Google algorithm or a Facebook algorithm. In fact, you can bring your own algorithm if you want to. I'm already doing such a bad job of explaining what the Fediverse is.
My mom is kind of a false flag in this because the truth is like, here's what I understand about the Fediverse. I understand that there were people who watched Twitter go up in flames and said, never again should one man be able to control the algorithm. And so from now on, among Twitter clones, of which there must now be 1,000, each one bad in its own specific way,
The Fediverse means that you will be able to have an account that is not linked to any one of those sites, and that I could post a boring post on threads, but it could be read by angry people on Blue Sky. The idea that these things are federated amongst each other, but not centrally controlled.
But honestly, if you said, keep explaining the Fediverse for five more minutes, I'd be like, I've run out of steam. I don't really get it.
But that's pretty good, right? It's a collective term for these various web platforms that use open source and decentralized protocols to let different platforms communicate and interact across these different hosting services. That's probably about as technical as you need to go. But the way I think of it is... It's just like a way to bring some humanity back to the internet.
It's a way to sort of rest it back from these giant mega tech platforms. It's a way to personalize things to your own liking, to like sort of customize them. And so it is starting with these social platform. Mastodon was the first thing in the Fediverse. Threads, which is actually now much bigger than Mastodon, is a meta product, but it is part of the Fediverse. Flipboard is joining it.
WordPress is joining it. Ghost, which is this hosting provider that I use for Platformer, is going to join it. And so someday you might just have an app on your phone. And instead of just going to Google to see what's the news of the day, you just open up your app that links you to the Fediverse. And you might be following some publishers there. You might be following some creators there.
There might be some ads in it so those folks are getting money. Maybe you do pay a subscription to some of the publishers in there so you get to see all of their paywall posts and they just kind of show up right in your feed. And while there's a lot to figure out in terms of how do you create a good user experience, how do you make that kind of more fun and useful than Google?
That just kind of feels like the direction to go to me because instead of one giant walled garden that is just keeping you there, keeping all the revenue for itself, it is a way of rebuilding a web where there's just a lot of organic connections between people and publishers who like each other and have ways about how we can make and share money with each other.
And so if it works, we're going to have something, I think, that feels much better than the world we have today.
But could it work? It feels like this is like so under-informed and I should not be saying it into a microphone and putting it on the internet and maybe I won't. But like, it feels like one of those ideas where you're like, yeah, it'd be nice. But like, you know, like things that are civic and volunteer and like parks that end up being trashed and then everybody just goes to the mall.
Do you think it could really work?
Here's my case that it could work.
Okay. Yeah.
Threads is an app that has 150 million monthly users. It is 10 months old, and it is part of the Fediverse. So that means, as hard as it is to believe, 150 million people every month are in the Fediverse. For the most part, they don't know about it, and they don't care. And that's actually a great sign. Because as we've just established through our tortured explanations of the Fediverse...
Nobody wants to understand what it is or how it works. So we're already working on one of the biggest problems with the Fed. And the thing is, PJ, I'm not the only person who's worried about this. You know, yesterday... I met with two folks. One is a guy, Eugene Rochko, who's the founder of Mastodon. The other guy is Mike McHugh, who's the co-founder of Flipboard.
And these two guys are running at this Fediverse stuff at 100 miles an hour. And the main thing they wanted to let me know was just how many other people are building this stuff with them, right? There's a lot of old timers and even young people around
who remember the early promise of the internet, who remember how exciting it was that we were going to have this thing that was decentralized, that was open, that shared the wealth with a lot of people. And they're going out and they're picking off these name brand websites like WordPress and Tumblr, you know, The Verge, the site where I used to work. They're pushing into federation.
So at this moment, is it a crazy band of insufferable, obnoxious rebels? Absolutely not. But I ask you, PJ, what movement in the history of the world has not begun with a band of insufferable and obnoxious rebels?
I can't think of one. Okay, so maybe the Fediverse saves us.
It might be the Fediverse. I mean, look, a lot of people are going to use Google. Again, one of the reasons why I'm so mad, PJ, is that this is going to work, okay? It's like, I'm mad because it feels like game over. I'm mad because most people are going to be totally happy to get the sort of Star Trek computer answer and not give two thoughts to...
any of the labor that went into producing the answer. And I'm sure Google will have a great business for itself. But like some of the people that worked at Google in the early days were really idealistic about what the web could be. And I believed in that optimism and I'm not ready to give up on it.
So if that means that I have to learn what the Fediverse is and explain it to other people, that's what I'm going to do because a better world has got to be possible here.
Casey, you're making me feel things.
tell you this. I felt things talking to Eugene and Mike yesterday. I was honestly shocked at how emotional I was at Google. It felt weird that I was as upset as I was walking around this developer conference.
And I think I was upset because I felt gaslit, honestly, because nobody at Google would just stand up and say, we actually do have a long-term plan to replace most web visits with our wild garden. So that's kind of why I was mad. But I was also really just pessimistic about the future. Then I sat down with those two and they were like, here are the next three things that we're going to build.
And here's like the next three big platforms that we're going to go after and get them to federate. And I'm like... This might work too because the thing about Google taking most of the winnings of the internet for itself is there's a lot of other people on the internet that would also like to eke out a living. And they're highly motivated to make it work for them.
So there will always be a rebel alliance and I would not count them out because companies that are old and have a lot of money, they get really lazy. and they can't move as quickly as sometimes they need to adapt to the future.
So, you know, if the Fediverse folks can build a better future that is truly more fun to use, it'll be really small for a while relative to the size of Google, but there's no reason why it couldn't grow very large in the end.
All right, Casey, I'll see you on the Fediverse.
Do you have a Fediverse account? Are you on mastodon.social?
Okay, so I started a Mastodon account and then I forgot the password. Yeah, that's the single most common story about the Fediverse, by the way. Really? Pretty much. I went on threads and I was like, this is very boring. You do post there. I see your threads post. Don't pretend you're above threads. I post episodes of Search Engine, but I don't like hang out and make funny jokes.
Well, maybe you should try it. Maybe you'd enjoy it. Maybe I'll try to make a joke on threads and see how it feels. Casey, thank you for talking about the past and future of the internet we have loved. You're welcome. Casey Newton. His newsletter, Platformer, is essential to understanding our quickly changing internet.
You can also find him on the wonderful weekly technology podcast, Hard Fork, with his co-host, Kevin Roos. Did we do a good enough job explaining the Fediverse this week? I feel like I'm still a little confused by it. So maybe you're also still a little bit confused by it. As far as I can tell, one way to think of the problem of Google search is that it's a problem of monopoly.
If the internet had more than one popular search engine, the entire web wouldn't have been somewhat corrupted by trying to appeal to Google. And Google wouldn't then have had to replace many of its useless web results with AI summaries. On a federated internet, people with followers can take their followers with them from platform to platform. So maybe that internet resists monopoly more easily?
I don't know. Honestly, I'm still a little confused how that fixes search and AI. You know what? If people want more Fediverse talk, email us, and we'll consider revisiting this in a future episode. If you want less Fediverse talk, email us and let us know too. How in the weeds should this show go? It's a question for you to help us answer. Journalism is a service industry.
You can reach us at searchengine.show. After the break, we have a podcast recommendation, which has to do with the themes of this episode. Stick around.
Thank you.
So as we've been thinking about Google's effect on the internet and about small publishers trying to survive, we've been closely watching the launch of one of my favorite new news websites, 404 Media. They cover the internet, they are brilliant reporters, and they're a small, independent outfit like us. they feel like a canary to me.
Like, if they can make a business succeed online, I'm a little bit less worried about the future. And I know other journalists who are watching 404 with the same question right now. What's relevant here, though, is that the team at 404 are very much at the mercy of Google's algorithms.
And recently they posted an episode where they just talked candidly about what it's like for a small group of humans to try to survive while fighting AI websites that are constantly scraping their work. You should go listen to that episode. It's called Why Google is Shit Now. I found it really fascinating, and it just goes way more in-depth to answer the question a lot of people have.
How come even before this AI thing, Google search just seemed to mostly stop working? You can find out there. I'm going to put a link in our show notes. Also, as we mentioned last episode, we are right now heading towards the end of season one of Search Engine. We're doing a board meeting with all of our paid subscribers on Friday, May 31st, to discuss show business.
That's business about the show, not news about Hollywood. The meeting, Friday, May 31st, 1 p.m. Eastern time. We will be sending out a Zoom link to join week of. This is only for our paid subscribers, people who are members of Incognito Mode, If you're not signed up, there's still time. Go to searchengine.show.
You can also send us questions there that you'd like to hear answered at this board meeting. And if you sign up, you'll get a lot of other stuff, which you can read about on our website. Again, that URL is searchengine.show. If you're a paid subscriber, look out for an email with a link next week and mark your calendar, May 31st, 2024, 1 p.m. Eastern.
Search Engine is a presentation of Odyssey and Jigsaw Productions. It was created by me, PJ Vogt, and Shruthi Pinamaneni, and is produced by Garrett Graham and Noah John. Fact-checking this week by Holly Patton. Theme, original composition, and mixing by Armin Bizarrian. Our executive producers are Jenna Weiss-Berman and Leah Reese-Dennis.
Thanks to the team at Jigsaw, Alex Gibney, Rich Perrello, and John Schmidt. And to the team at Odyssey, J.D. Crowley, Rob Morandi, Craig Cox, Eric Donnelly, Kate Hutchison, Matt Casey, Maura Curran, Josephina Francis, Kurt Courtney, and Hilary Schaaf. Our agent is Oren Rosenbaum at UTA. Follow and listen to Search Engine for free on the Odyssey app or wherever you get your podcasts.
Thanks for listening. We'll see you soon.