AI agents are already a measurable share of web traffic, and two companies account for almost all of it. Search Engine Journal reported on November 14, 2026 that Meta and OpenAI agents made nearly 90% of the agent requests observed in one security firm's bot data over a month.
The finding comes from bot traffic data covering a month of agent requests, reported by Search Engine Journal on November 14, 2026. Two companies' AI agents, Meta's and OpenAI's, made most of the requests observed, which means the agent web is far more concentrated than the search web. A site deciding how to treat AI agents is, in practice, deciding how to treat two companies' crawlers and agents first and everyone else second.
The requests were not spread evenly across site types either. Most went to product and search pages rather than checkout, which matches how agents are actually used today: researching, comparing, and shortlisting, with the purchase still usually happening through a human. That pattern matters because it tells site owners where agents will show up first, and where blocking them would hurt most.
An agent request is not the same as a search crawler visit or a human visit. Agents arrive with a task, follow links to complete it, and may return repeatedly as they refine a comparison or check a price. They can also arrive in bursts, and they do not behave like a browser session, which is why bot management tools are starting to count them separately from both crawlers and ordinary visitors.
That distinction is already showing up in how sites respond. United Airlines updated its terms this month to let search engines index its public pages while requiring written permission for AI agents, drawing a line between traditional search crawling and agent access. The airline's move, reported by Search Engine Journal on November 14, is the clearest example yet of a major site treating agents as a separate category with its own rules.
For SEOs, the concentration of agent traffic is both a warning and an opportunity. The warning is that a bot policy written for search crawlers may now be letting in, or blocking, traffic it was never designed for. Blocking AI training crawlers is a different decision from blocking retrieval bots, and blocking agents is a third decision again, and sites that conflate the three will get at least one of them wrong.
The opportunity is that product and search pages are where agents spend their time, which is exactly where structured data, clear pricing, and accurate availability matter most. An agent comparing products cannot use a page that hides its price behind a script or buries its specifications in an image. The sites that agents can read cleanly are the sites agents will recommend, and with Meta and OpenAI accounting for nearly nine in ten agent requests, optimising for those two ecosystems covers almost the whole market.
Agent traffic will only grow from here, and the next milestone to watch is whether other companies' agents gain share or whether Meta and OpenAI extend their lead. The more interesting question for site owners is policy: more major sites are likely to follow United Airlines in drawing a line between search crawlers and AI agents, and the terms pages of large publishers and retailers are where those lines will appear first. For now, the practical step is to check the server logs for agent traffic, see which pages the agents visit, and decide deliberately, page by page, whether those visits are welcome. A policy written by default is still a policy, and it is the one most sites currently have.
Sources: Search Engine Journal, November 14, 2026.
Related reading on this site: Google Launches Gemini Decisions API, Powered by New Gemini Decision 1 Model, Anthropic Opens Private Beta for Self-Hosted Claude Managed Agents, Claude Suffers Major Login Outage as Microsoft's Outlook Outage Drags On, SpaceX's AI Group Lays Off Around 60 From Grok Imagine Team as Imagine Tab Vanishes, Chicago Public Schools May Ditch Google Search Over Unblockable AI Overviews, and AI-Generated Images Are Returning Inside AI Overviews.
Linkbaits helps you create content that earns citations and backlinks. Start with the toolkit and build your first linkable asset this week.
Start free trial