It’s Content Independence Day: Cloudflare, along with a majority of the world's leading publishers and AI companies, is changing the default to block AI crawlers unless they pay creators for content.
Introducing pay per crawl: Enabling content owners to charge AI crawlers for access
Pay per crawl is a new feature to allow content creators to charge AI crawlers for access to their content.
Control content use for AI training with Cloudflare’s managed robots.txt and blocking for monetized content
Cloudflare is making it easier for publishers and content creators of all sizes to prevent their content from being scraped for AI training by managing robots.txt on their behalf, and allowing targeted blocking of AI crawling on sites that serve ads.
Content Independence Day, one year on- building the business model for the agentic Internet
One year after declaring Content Independence Day, a dynamic market for monetized content has officially emerged. In this report, we examine how the rise of autonomous AI agents is upending traditional search referrals and detail the new infrastructure required to support a sustainable web economy.
Cloudflare's Matthew Prince has a plan to get Google and the AI oliigarchs to pay for your content even though many are used to getting it for free. He might have enough leverage to make them.
AI chatbots are blowing up the 30-year economic relationship publishers have had with search engines. Many think this will kill the web if not addressed. Google needs to change first. It's resisting. Cloudflare's Matthew Prince is going to try and make them.
Exclusive: Multiple AI companies bypassing web standard to scrape publisher sites, licensing firm says
Multiple artificial intelligence companies are circumventing a common web standard used by publishers to block the scraping of their content for use in generative AI systems, content licensing startup TollBit has told publishers.
Data Streaming for AI: From Extractive Training to Sovereign Infrastructure DWeb Camp 2026
AI systems are consuming the world's content without compensating its creators. This session explores data streaming as a new paradigm — where content flows to AI in real time, with built-in rights management, usage tracking, and fair compensation — and asks what it would take to make this infrastructure decentralized, sovereign, and governed by the communities it serves.
Patreon Blocks Crawlers From Stealing Creators' Work for AI Training
“Creators deserve credit, compensation, and consent. If that's not on the table, the crawlers can stay the fuck off Patreon," CEO Jack Conte wrote on Thursday.
Nobody Wants to Pay for Your AI
AI is everywhere today. It writes social posts, suggests email replies, recommends your next binge-watch, and quietly powers tools you use without even noticing.
AI is eating website traffic, websites are blocking AI – and reliable information is getting harder to find
AI has broken the economic bargain behind the web, and everybody is losing.
Cloudflare gives AI agents an identity and a wallet | Cloudflare
New tools mean businesses will be able to see who's behind an AI agent, and those agents will be able to pay for things safely on their owner's behalf.
A Strategy for Protecting Content from AI
Because that’s the world we live in these days.
Content Signals
An up-to-date guide to the IETF's proposed new AI Preferences (aipref): a new way for website publishers to control how automated systems use their content.
Tracking AI-enabled Misinformation: 3,749 AI Content Farm sites (and Counting), Plus the Top False Claims Generated by Artificial Intelligence Tools
Coverage by McKenzie Sadeghi, Dimitris Dimitriadis, Virginia Padovese, Giulia Pozzi, Sara Badilini, Chiara Vercellone, Natalie Huet, Zack Fishman, Leonie Pfaller, and Natalie Adams | Last Updated June 23, 2026
The AI-Powered Web Is Eating Itself
Without a framework of “Artificial Integrity,” AI search platforms risk collapsing the information commons that made the web possible.
Cloudflare CEO warns AI and zero-click internet are killing the web's business model
It's been known for some time that the web is changing into the Zero-Click Internet, the name for when users no longer need to click on links...
"Build protocols, not platforms"
Exploring the 'Protocols, Not Platforms' paper alongside a curated digest of recent AI crawler policy shifts and developer experiments.
