Publishers who want to keep their work out of AI training have spent years juggling a different switch for every company: one token for Google, another for OpenAI, another for Anthropic, and so on. On Friday, October 30, the four largest Western AI builders moved to end that patchwork, announcing they will all honor a single shared signal in a website's robots.txt file.
The signal is a policy token called __usage_ai-training. A site owner who wants out adds two familiar lines to robots.txt, User-Agent: __usage_ai-training followed by Disallow: /, and every participating company reads that as an instruction not to use the site's content for training. Publishers can also carve out exceptions with Allow lines, for example opening a press kit while closing the rest of the site.
The proposal comes from the Internet Engineering Task Force's AI Preferences working group. It is set out in an individual Internet-Draft titled Machine-Readable Training Preference Signal, authored by Kevin Kelley of Anthropic and dated October 30, 2026. The draft adds no new directives or grammar, so existing robots.txt parsers can simply ignore a group they do not recognize. It is still only a draft, though: the IETF datatracker notes an individual submission has no formal standing, and this one expires on May 3, 2027, unless revised.
The companies are moving on different schedules. Anthropic, which wrote the draft, says it has already begun implementing support. OpenAI says it aims to implement the signal in the coming months and that its existing crawler controls will keep working alongside it. Microsoft targets February 1, 2027. Google says it aims to support the token in 2027, and Google's John Mueller said it will help websites express their AI training preferences across multiple providers in a single step. Where a site opts out through the shared token without naming Google-Extended, Google plans to treat that as if the publisher had disallowed Google-Extended, its training control since 2023.
Microsoft went furthest. In a post by Krishna Madhavan, a principal product manager for Bing, the company said publishers should not have to choose between being discoverable in Bing and deciding how generative models use their work. Alongside the shared token, Microsoft is adding a page-level training setting and a separate AI Grounding setting that decides whether a page can be pulled into AI answers in Bing, Copilot and Microsoft 365, plus a limited option allowing its own products but not the grounding services it sells to others. Temporary controls arrive in Bing Webmaster Tools on December 15, 2026, and publishers using the older NoArchive and NoCache tags have until March 1, 2027, to switch.
The limits matter as much as the promise. The shared token governs training only: it is not a crawling directive, so bots may still fetch a page, and on its own it does not keep content out of AI-written answers, which is the use currently costing many sites referral traffic. It is also voluntary. OpenAI frames the mechanism as a coordination mechanism rather than a security control under RFC 9309, which means scrapers that choose to ignore it can do so, and adding the lines today does not remove material already collected into existing datasets or models.
Even with those caveats, the coordination itself is the news. Rivals that compete fiercely for training data have agreed to read the same opt-out the same way, turning a dozen vendor-specific rules into one short entry any site owner can write. Whether it becomes a durable standard depends on the draft surviving the IETF process, and on the companies honoring the schedules they have put in writing.