AI Search

Cloudflare Signals Helped OpenAI Find Hundreds of Millions of New Pages

Signals from Cloudflare's network helped OpenAI's search crawler discover and index hundreds of millions of new web pages for ChatGPT search, according to early results from a research pilot the two companies have run since July 2026. The pilot covered only about 1% of Cloudflare's network, yet it still surfaced pages ChatGPT had never seen before — and gave the chatbot more sources to draw on when it answers questions.

What the pilot found

Cloudflare and OpenAI launched the research pilot in July 2026 to test whether signals from Cloudflare's network could help ChatGPT users discover new content more efficiently. The early results, shared in a joint blog post on October 15, 2026, say the answer is a clear yes. The pilot brought previously unknown pages into OpenAI's search index, giving ChatGPT more sources to draw on for relevant, timely information. The post was co-written by Cloudflare's Matthew Conroy and Jack Galilee and OpenAI's Henry Aspegren and James Balfour. The examples in the post show the kind of pages the signals surfaced: the user guide for a particular washing machine, the latest information on passport requirements and fees, and a restaurant's menu. These are useful, specific pages that a crawler working from links alone can easily miss, especially when they sit deep inside a site or change often.

How the signals work

Cloudflare's network sits in front of a large share of the web, and the pilot uses what it sees to point OpenAI's crawler in the right direction. The signals flag which pages have changed recently, which addresses lead nowhere or only redirect elsewhere, and which pages carry quality traffic. That helps the crawler spend its visits on pages worth fetching and skip the dead ends. The companies stress that the research uses metadata about the web for search and discovery, not page content for model training. A signal about a page does not grant permission to access it. Your robots.txt rules and your Cloudflare settings still decide whether OpenAI's crawler can reach a page, and OpenAI independently evaluates and ranks the pages it indexes based on how relevant and helpful they are.

What the results do not promise

The numbers come with limits the companies state openly. The post calls the results preliminary and says another phase of the pilot is planned. It does not report how often the newly discovered pages were actually cited in ChatGPT answers, or how many visits they sent to the sites behind them. The "hundreds of millions" figure is an order of magnitude, not an exact count, and there is no public figure for the size of OpenAI's index before the pilot. Cloudflare says it intends to offer broad access to the discovery capability at some point during the remainder of 2026, but there is no setting to opt into the pilot today, and the post does not describe one.

Why it matters for SEOs and link builders

For site owners, the pilot closes a discovery gap that has quietly shaped AI search. Pages that no crawler ever found could never be cited, no matter how good they were, and this program shows that gap is being filled from the network side rather than the link side. That raises the value of keeping useful pages crawlable and current: freshness signals and real page changes are exactly what the pilot rewards. It also means blocking OpenAI's search crawler has a clearer cost, since a page that cannot be fetched cannot be discovered this way either. Discovery is only the first step, though. OpenAI still decides what to rank and cite on relevance, so the pages that win will be the ones that are genuinely useful, clearly written, and backed by the kind of trusted links that mark a source worth quoting.

Sources

Sources: Search Engine Journal, November 12, 2026.

Related reading on this site: Cloudflare CEO: AI Agent Optimization Is the New SEO, Google's Search Volume Is About 5x ChatGPT's, OpenAI Says, ChatGPT Sends Just 0.62% of Google's Organic Traffic, Google: Technical SEO Remains Key for AI SEO, Brave Sells Its Search Index to AI — Your Site Is Probably In It, and AI Assistants Hit 76 Billion Monthly Sessions, Graphite Estimates.

More from Linkbaits

Turn this playbook into links

Linkbaits helps you create content that earns citations and backlinks. Start with the toolkit and build your first linkable asset this week.

Start free trial