







Today, we're announcing our plans to support a shared AI training control in robots.txt. This will help websites express their AI training preferences across multiple AI providers in a single step and is based on work from the AIPREF working group at the Internet Engineering Task Force (IETF). This builds on our long history of providing website owners with clear controls, including through Google-Extended, the AI training control we've offered since 2023.
Making it easier for websites to express AI training preferences
OpenAI will honor a shared robots.txt signal to help website operators communicate their AI training preferences to participating providers.

Content Signals
An up-to-date guide to the IETF's proposed new AI Preferences (aipref): a new way for website publishers to control how automated systems use their content.
browser-use/browser-use
🌐 Make websites accessible for AI agents. Automate tasks online with ease.
Control content use for AI training with Cloudflare’s managed robots.txt and blocking for monetized content
Cloudflare is making it easier for publishers and content creators of all sizes to prevent their content from being scraped for AI training by managing robots.txt on their behalf, and allowing targeted blocking of AI crawling on sites that serve ads.

AI controls are coming to Firefox | The Mozilla Blog
AI is changing the web, and people want very different things from it. We’ve heard from many who want nothing to do with AI. We’ve also heard from
Google for Developers Blog - News about Web, Mobile, AI and Cloud
An open specification for finding and verifying tools, skills, and agents across the web.Agents are ...

Google for Developers Blog - News about Web, Mobile, AI and Cloud
An open specification for finding and verifying tools, skills, and agents across the web.Agents are ...

Machine-Readable Training Preference Signal
This document describes how a service operator binds usage preferences (e.g., "no AI training") to paths by writing ordinary robots.txt groups keyed by a per-purpose policy token in the User- Agent line. It introduces no new directive names and no new grammar: the path language, longest-match precedence, and group structure of robots.txt are not changed.
Machine-Readable Training Preference Signal
This document describes how a service operator binds usage preferences (e.g., "no AI training") to paths by writing ordinary robots.txt groups keyed by a per-purpose policy token in the User- Agent line. It introduces no new directive names and no new grammar: the path language, longest-match precedence, and group structure of robots.txt are not changed.
Designing responsible AI for learning
We’re bringing Guided Learning capabilities to Search and Google Classroom, and evolving our approach to school controls.

Workshop: "AI Agents & Web Browsers" · Issue #507 · w3c/strategy
NOTE: This is a tentative workshop draft — Dom and Roy are working on a CFP: Introduction This W3C Workshop aims to gather stakeholders to explore how AI agents can interact with and be integrated ...
Download | Tiles
A local-first, collaborative AI assistant that works for you. Built with open models and decentralized protocols.

Next Level ChatGPT? Auto Mini AGI Agents That Run in your Browser!
Search generative AI control - Search Console Help
Note: As of August 31, 2026, we’ve rolled out this control to all websites worldwide. Use the Search generative AI control to manage your site's inclusion in the following generative AI features
Making a Website in 2026
Pete Millspaugh has a great post over on the Val Town blog where he documents his attempt at making generating the same website across a variety of AI tools. I…
Google Unveils Project Mariner: AI Agents To Use the Web For You - Slashdot
Google today unveiled Project Mariner, its first AI agent capable of autonomously navigating web browsers, operating through a Chrome extension that controls cursor movements and form-filling to replicate human interactions online. The Gemini-powered prototype, developed by Google's DeepMind divis...
