Hi Google, please stop 💩 the bed: a desperate plea from the independent web.

Served with humor, as I cry into my goldfish bowl of boxed Chardonnay.

Every 3 weeks, I share my notes on building Shepherd.com.

Join the rebellion and help me create the indie book platform readers and authors deserve! Three hurrahs for our 818 Founding Members who keep us independent and fund new features. Readers and authors can become Founding Members (both get special perks, and more perks are coming).

---

Dear Google,

Please stop 💩 the bed.

You are destroying the independent web. 🤪

What is Google doing?

Google is killing independent websites by burying them in the search results, even when we perfectly follow their guidelines.

Nobody" is sure what is going on, but their search engine started breaking down about 16 months ago.

Some people believe they have lost control of their AI ranking systems, while others believe that the McKinsey alumni who run Google Search are purposefully doing this to increase ad earnings (DOJ leaks from the ongoing trials hint at this).

(I have zero doubts that many engineers at Google want to create a good search engine, but I don’t know if they are in an environment where they can. From many interactions, it does not seem that they understand the websites that make up the web. )

Let me show you how Google is 💩 the bed…

---

Here is a graph of Google traffic to Shepherd…

Over the last 16 months, we’ve seen an 86% decline from Google to the most-loved section of the website.

This section has been incredibly popular with Google for years and it has sky-high engagement stats from visitors.

Google changes all the time, but a drop like this is insane. This is destroying many websites you know and love.

Google's promise to website owners was simple: create great content, and Google will rank those pages if those pages satisfy users.

This promise has been broken; let me show you how.

First, visitors love these pages.

Why did Google stop loving them?

I am a data nerd. I love building good UX and work constantly to provide my visitors with an amazing experience.

I even do anonymous user video testing with every feature to ensure it is easy to navigate and that readers love the experience.

• 60% of our visitors read 75% or more of the pages in this section.

• On average, visitors spend 5 minutes and 1 second on pages in this section (the real number is higher, as this only updates if an action is triggered on the page).

• 12% of visitors click to learn more about a book or go to a bookstore to learn more. This is an excellent sign that the reader found something that intrigued them enough to dig deeper.

• 8% of visitors click to visit another book list that we recommend at the end of the recommendations.

• And more…

My point is that this section is incredibly popular with visitors, and Google used to see and reward that (hypothetically). This is happening to hundreds of thousands of websites, and Google is not saying anything.

Here is a specific example…

I asked Kevin Miller to share his five favorite books about the Battle of Midway and why he thinks each of those is a great read (click the below to read that).

Kevin is an expert.

Kevin has written a book on the Battle of Midway, is the Executive Vice President of the Naval Aviation Museum, and is a former U.S. Navy Fighter Pilot.

He is exactly the person you want making an expert-led book recommendation about the Battle of Midway.

Where does Kevin’s page appear in Google?

Since it was published, Kevin’s page has been in the top 3 results for Google searches related to the “best books on Battle of Midway.”

…

Then Google 💩 the bed.

….

Kevin’s page now appears on the 3rd page, or not at all, for all searches.

How does Kevin’s page engage readers?

• On average, visitors spend 7 minutes and 16 seconds on his page (the real number is higher as this only updates if an action is triggered). And 55% of visitors read to the 75% mark or more.

• 15% of visitors click to explore a book or go to a bookstore (to learn more about a specific book). This is a very good sign that the reader found something that intrigued them.

• 15% of visitors go within Shepherd to a related topic, genre, or book on this page.

• 7.5% of visitors check out a recommended list at the bottom of the page or related topic.

My point is that this page crushes it for a large chunk of readers who are looking for an answer to their query on Google. And even worse, what Google is ranking instead is not great…

What ranks instead of Kevin’s page?

The page ranking #2 is one I created to test how badly Google is 💩 the bed.

It is a list I created on Bookshop.org that has 4 books, no personal details, no expertise, and nothing of value. It even links to Kevin’s list on Shepherd since it is a terrible derivative of that list.

What ranks in the top 10 if Kevin does not?

• There is a Goodreads list of all books tagged Battle of Midway. They are not sorted by rating, and there is no expert explaining why they are a good resource. It is not the worst list, I guess, but it is just a collection of the 100 books on the battle that Goodreads has in their database.

• A 3-year-old Reddit thread with only one book recommendation. I generally like Reddit, but this thread isn’t very helpful.

• A Museum e-commerce page selling a book about the battle for $35 dollars. There is zero curation, and it just has the default book description selling the book.

• A Quora post with only 2 answers. The first answer recommends a single book, and the other answer can only be viewed if you pay for Quora Plus.

• Google’s book widget, which shows book covers that match the topic? Wee! Zero curation or expertise or usefulness.

• An Amazon link to a single book about the Battle of Midway. How is that helpful?

• Another Goodreads list with 829 books about “Midway.” The 3rd book recommended is Dune, and the fourth is Little Women. Is that helpful?

• This is a decent result from the Midway Library (except it hasn’t been updated since 2016). However, they do have some good recommendations and are curated by experts (although I am not sure who). The collection also movies and other media.

• A 2010 post on a Board Game forum asking for good books about the Battle of Midway. This post is 14 years old and has zero expertise. Google loves UGC right now, but this is frustrating.

• A B&N link to a single book about the Battle of Midway.

My point is that Google has elevated bad results that do not help users and buried independent websites like ourselves.

In many cases, we are seeing websites that have served visitors for years lose 95%+ of their traffic from Google and not even rank for their brand name. This is a shadow ban where something in Google’s system has effectively blocked that website from existing. Google refuses to acknowledge it is happening or speak about it.

Meanwhile, search engines like DuckDuckGo and Bing rank Kevin’s page in the #1 and #3 spot, as they should.

---

Shepherd is luckier than most.

I am lucky because while Google Search is breaking, I am receiving increasing traffic from Bing, DuckDuckGo, social media, and other sources. That has allowed us to stay alive along with the support of our Founding Members (who financially support the website for the long term).

I have many friends and acquaintances who are going bankrupt and shutting down fantastic independently run websites. It is sad to see.

The independent web is in danger, and Google is destroying it.

What do I want Google to do?

Get the 💩 out of the bed!!!

I used to love Google.

It was magic compared to Altavista.

I could type in what I wanted, and you would return a result that was what I wanted. At the very least, I had to try a few results before finding what I was specifically looking for.

Now, it is just a wall of ads, unhelpful results, spam from big publishers like Forbes, and Reddit (I like Reddit, but you do have to vet them still). I won’t even mention the AI overviews… which could be useful but are instead plastered on every surface you can find.

Google, you have to fix search so it works again.

It has gotten so bad I moved to the Kagi search engine, and my wife switched to using ChatGPT 75% of the time.

What else? Oh ya → Talk to us!!!!!

Google has released several core updates since this started, and not a single website has really recovered.

It has also not provided enough good information about what is going on. Google has search liaisons who are amazing human beings, but they seem unable to share real information or help anyone. They just say the same corporate speak without saying much of anything.

• Dear John Mu and Danny Sullivan (work), I have to do a shot of Tequilla every time you say, “Just make good content,” so please stop; I am begging you… Please provide real information and help the website owners you are destroying. I know you want to, as you are both amazing people.

Other ideas:

• I’d love to see a partner program where Google has website owners embed code on their websites to pull in anonymous data about engagement so they can better reward good websites. I HATE spam and SEO spam, and I think this could help fix the problem. Google has something similar on YouTube, given their control of that platform, and I think this could work by helping them identify who wants to build a better web.

• Build real help into GSC. Tell website owners what needs to be improved and why. Why are you hiding this information behind a mysterious black box?

---

I am an inarticulate clown.

I put off writing this because I am not the best writer.

Here are articulate breakdowns by articulate people on why this is so dangerous for the web:

• HouseFresh explains how Google is killing independent websites like theirs. This one is especially important as it shows how spam now ranks above them.

• Here is how Google killed RetroDodo, which is one of the best retro video gaming websites on the net.

• Healthy Framework explains how Google is breaking the web and why soon, nothing will be left but spam and Reddit. Make sure you read part 2 to see how this is getting dangerous.

• And a summary from RePlay on how Google has destroyed an entire swath of websites.

• A great overview of the pain Ready Steady Cut is in as well as other sites.

• A fantastic set of notes that Rutledge sent Google before the Oct 28/29 creator summit that Google is throwing.

• Mike’s post about Google destroying a 17-year-old outdoor website that is his life.

I have a lot more, but those are the best overviews of the damage Google is doing.

I also recommend this video. It isn’t a perfect explanation, but he explains why Google is incentivized to create terrible search results.

Thanks for listening Google,
Ben Fox (founder of Shepherd and inarticulate clown)

P.S. Here is a great breakdown on the problem straight from someone who Google brought to advise them on this disater.

And another from an attendee at the Google summit.