September 2026
If your website sits behind Cloudflare, three settings now decide how AI tools interact with your content. Cloudflare calls them Search, Training, and Agent, and in September 2026 it replaced its old all-or-nothing "Block AI Bots" switch with these three controls. If you ever turned that old switch on, your settings were migrated automatically, and the new defaults may not be what you want.
What the three controls mean
Search covers crawlers that index your site so it can appear in search and AI answers. Blocking these makes you harder to find everywhere. Almost every site should allow them.
Training covers crawlers that collect content to train future AI models. Blocking these is a legitimate choice and does not meaningfully change whether AI assistants can find and cite you today. If keeping your content out of training data matters to you, disallow training and your day-to-day AI visibility is largely unaffected.
Agent covers AI tools fetching your pages live, at the moment someone asks a question. This is the one that surprises people. When ChatGPT, Gemini, or Claude answers a question and cites sources, it often fetches the page right then, as an agent. If agents are blocked, your site can drop out of exactly the answers you want to appear in, even though "search" is allowed.
The trap in the new defaults
Sites migrated from the old Block AI Bots switch received Agent: Block on pages with ads. If your site runs ads and you were migrated, AI assistants may be turned away at answer time right now, and nothing on your site will look different to you.
The two-minute fix
Log in to Cloudflare, open Security settings, and find the AI crawler controls. Set Search to Allow. Set Agent to Allow. Set Training to whatever matches your comfort with your content being used to train models; it is the one setting here that is genuinely a values call rather than a visibility call.
Then check what AI can actually reach
Firewall settings are only one of the things that decide whether AI can read, trust, and quote your site. A free scan shows you the rest: your 0 to 100 AI Visibility Score, what is holding you back, and the exact updates to make. If our scanner meets a firewall challenge on your site, AI assistants usually meet the same wall, and your results will say so.