Google

Google Warns CDN Moves Can Block Googlebot and Drop Pages From Search

Moving a site behind a content delivery network can briefly flood the origin server with crawler requests, and the instinctive response can be worse than the problem. In a Google Search Central blog post published on November 11, 2026, titled Moving to a CDN without overwhelming your origin (or blocking Googlebot), Google explains that faster edge responses encourage its crawlers to request more pages, while a new CDN's empty cache sends many of those requests straight back to the origin. The post arrived alongside the fifth episode of Google's The Non-Canonical Cabinet video series.

Why a CDN move spikes crawl traffic

The mechanism starts with what Google calls the crawl capacity limit, also known as hostload. Google's crawlers watch how fast a site responds and raise their crawl rate when responses get faster. A CDN makes responses faster, so the crawlers ask for more. At the same time, a new CDN's caches are empty, so the first requests for each page cannot be served from the edge and fall through to the origin server. The result is a surge of requests hitting the very server the CDN was meant to protect.

Google's post describes a CDN as a major upgrade for any growing site, and the spike is a normal part of the transition. The danger is in how a site owner reacts to it. An admin who sees a surge of automated requests in the origin logs, without knowing why it is happening, can easily conclude the site is under attack and switch on the CDN's blanket bot protection.

The bot protection trap

That reaction is the trap Google's post is really about. Blanket bot protection treats all automated traffic the same, including the traffic from crawlers that a site owner cares about. When a CDN or web application firewall meets crawler requests with a challenge such as a CAPTCHA, Google's crawlers fail the test. Google puts the consequence plainly: the site appears deliberately restricted or broken, and if the challenge stays in place, its crawlers will back off, and pages will begin to drop out of Search.

The accompanying video goes further, saying the same block can make products disappear from Google Shopping and stop ads products from working. A firewall rule that turns crawler requests into challenge pages reaches a similar end from the other direction: the products are there, but Google cannot see them. The fix is an exemption, not an off switch: allow verified bots in CDN and firewall settings, using Google's published IP ranges to verify its crawlers, and target the bots that are actually unwanted by name.

What Google recommends instead

Google's advice has three parts. First, allow verified bots in the CDN and firewall settings before the migration, so Google's crawlers are never challenged in the first place. Second, pre-warm the caches by programmatically fetching the highest-traffic and upcoming peak-traffic pages, such as promotional or seasonal launches, so the edge can serve them from the start. Third, if the origin is still struggling, return a 429 or 503 status code as a temporary brake for a short window, a few hours or one to two days at most, as Google's guide to reducing crawl rate describes.

The post gives no figures for how often the problem occurs, how large the spikes get, or how long they last. It describes a mechanism, not a dataset, which means there is no threshold to watch for: any CDN migration can produce the spike, and any blanket bot rule switched on in response can block the crawlers. The check is a setting, not a number, and it should be verified before the migration, not after the traffic drops.

Why it matters for SEOs and link builders

A CDN migration is one of the most common technical changes a growing site makes, and it usually happens for good reasons: speed, reliability, and capacity before a launch or a seasonal peak. This warning lands in the middle of the November core update rollout and two weeks before Black Friday, which is exactly when e-commerce sites are most likely to be migrating, warming caches, and watching their servers. A bot rule switched on in a panic this week could knock pages out of Search during the biggest shopping weekend of the year.

For link builders, the warning matters because a page that drops out of Search stops passing value, and a placement on a page Google can no longer crawl is a placement that has gone quiet. Before pitching a site, or before judging a placement's performance, it is worth checking whether the site recently moved behind a CDN and whether its firewall has a verified-crawler exemption. A sudden traffic drop after a migration is more likely to be this trap than a penalty.

What to watch next

Watch the CDN and firewall dashboards after any migration for a bot-challenge or CAPTCHA rule running without a verified-crawler exemption, and check the server logs for crawler requests hitting challenge pages. The recovery, once the exemption is in place, is a matter of Google re-crawling the affected pages, which takes time. The cheapest fix is the one done before the migration: exempt verified crawlers, warm the caches, and keep the 429 or 503 brake short.

Sources

Sources: Google Search Central Blog, November 11, 2026; PPC Land, November 12, 2026.

Related reading on this site: November Core Update Collides With Black Friday, and a Clean Read Will Not Come Until December, Core Update Volatility Started Days Before Google Announced It, November Search Console Click Spikes May Be Bots, SEJ Reports, Black Friday SEO Starts Eight Weeks Out, and November Is Already Late, SEJ Pulse: Core Update, ChatGPT Business Hub and AI Mode's New Model, and Google Explains How to Find Search Traffic in Analytics and Server Logs.

More from Linkbaits

Turn this playbook into links

Linkbaits helps you create content that earns citations and backlinks. Start with the toolkit and build your first linkable asset this week.

Start free trial