CLOUDFLARE REPLACES AI BOT BAN WITH GRANULAR CONTROLS

AI DESK■ 1 MIN READ

MON, JUL 6, 2026

■ AI-SUMMARIZED FROM 1 SOURCE ▸ TIMELINE

Cloudflare is rolling out granular AI bot management tools that let site owners control Search, Training, and Agent crawlers independently instead of blocking them all at once. Starting September 15, 2026, Training and Agent bots will be blocked by default on ad-supported pages.

The shift moves away from Cloudflare's previous blanket approach to AI bot blocking. Site owners now gain flexibility to allow search engine crawlers while restricting training and agent bots—or manage each category according to their specific needs. The September 2026 default settings represent a middle ground for ad-supported sites, where Training and Agent bots will be automatically blocked unless explicitly permitted. This addresses growing concerns about AI companies scraping content without permission while maintaining access for search indexing. Cloudflare's granular controls apply to all customers, not just enterprise accounts, expanding access to bot management tools across its user base. The change reflects broader industry pressure to balance AI development with content creators' rights and site owners' control over their data.

■ SOURCES

► The Decoder

■ SUMMARY WRITTEN BY AI FROM THE LINKS ABOVE

■ MORE FROM THE AI DESK

P641AI CHATBOTS AUTOMATE DEBT COLLECTION

Startups like Altur are deploying AI chatbots to handle debt collection calls, automating a process traditionally done by humans. Y Combinator has backed six debt collection and settlement startups over the past six years.

JUST NOW— AI Desk

P639CERF DEVELOPS STANDARD TO IDENTIFY AI AGENTS ONLINE

Vint Cerf, co-inventor of TCP/IP, is creating a framework to identify and track artificial intelligence agents operating on the open internet.

JUST NOW— AI Desk

P630AI FILLS RELIEF GAP AFTER VENEZUELA EARTHQUAKES

Following recent earthquakes, Venezuelan developers and citizens deployed AI-powered websites and apps to locate missing persons and coordinate disaster relief as government response lagged.

2H AGO— AI Desk

P625ALBANESE ESTABLISHES AI OFFICE, PLEDGES CREATIVE PROTECTION

Prime Minister Anthony Albanese has created a dedicated AI office and committed to protecting Australian creators from copyright infringement by artificial intelligence companies. The government rejected plans to grant tech firms free access to Australian data.

3H AGO— AI Desk

◄ BACK TO NEWS

CLOUDFLARE REPLACES AI BOT BAN WITH GRANULAR CONTROLS

■ MORE FROM THE AI DESK

■ SUBSCRIBE TO THE DAILY BRIEF