CLOUDFLARE REPLACES AI BOT BAN WITH GRANULAR CONTROLS

AI DESK■ 1 MIN READ

MON, JUL 6, 2026

■ AI-SUMMARIZED FROM 1 SOURCE ▸ TIMELINE

Cloudflare is rolling out granular AI bot management tools that let site owners control Search, Training, and Agent crawlers independently instead of blocking them all at once. Starting September 15, 2026, Training and Agent bots will be blocked by default on ad-supported pages.

The shift moves away from Cloudflare's previous blanket approach to AI bot blocking. Site owners now gain flexibility to allow search engine crawlers while restricting training and agent bots—or manage each category according to their specific needs. The September 2026 default settings represent a middle ground for ad-supported sites, where Training and Agent bots will be automatically blocked unless explicitly permitted. This addresses growing concerns about AI companies scraping content without permission while maintaining access for search indexing. Cloudflare's granular controls apply to all customers, not just enterprise accounts, expanding access to bot management tools across its user base. The change reflects broader industry pressure to balance AI development with content creators' rights and site owners' control over their data.

■ SOURCES

► The Decoder

■ SUMMARY WRITTEN BY AI FROM THE LINKS ABOVE

■ MORE FROM THE AI DESK

P452FERMAT'S LAST THEOREM NOW FORMALLY VERIFIED

Anthropic researchers have completed a formal mathematical proof of Fermat's Last Theorem, translating Andrew Wiles' decades-old proof into machine-verifiable code. The achievement marks a milestone in computational mathematics, ensuring the theorem's logical foundations are beyond dispute.

2H AGO— Industry Desk

P446OPENAI LAUNCHES GPT-6 ASTRA, ADVANCES TOWARD AGI

OpenAI has released GPT-6 Astra, its most advanced model to date, marking a significant stride toward artificial general intelligence. The company has implemented new safety guardrails due to the model's powerful cybersecurity capabilities.

2H AGO— AI Desk

P445AI SHRINKS ENTRY-LEVEL TECH JOBS AS DEMAND SHIFTS

The US information sector lost roughly 23,000 jobs in August as AI reshapes hiring patterns. Research shows junior software developer positions are declining while employers increasingly demand experience and advanced skills from entry-level candidates.

3H AGO— AI Desk

P444ROLAND LAUNCHES MELODY FLIP, ENTERS AI MUSIC GENERATION

Roland has released Melody Flip, a generative AI music tool available as a DAW plug-in. The tool offers 250 themed musical palettes and can generate melodies, chord progressions, basslines, and drums from scratch or based on reference tracks.

4H AGO— AI Desk

◄ BACK TO NEWS

CLOUDFLARE REPLACES AI BOT BAN WITH GRANULAR CONTROLS

■ MORE FROM THE AI DESK

■ SUBSCRIBE TO THE DAILY BRIEF