What happened
On October 30, 2026, Kevin Kelley of Anthropic submitted an Internet-Draft to the IETF's AI Preferences working group titled "Machine-Readable Training Preference Signal." It describes a way to state a preference against AI training inside a normal robots.txt file, using a "policy token" on the User-Agent line: User-Agent: __usage_ai-training Disallow: / Allow: /press-kit/ That example says no to AI training everywhere except the press kit folder. The draft adds no new directives and no new grammar to robots.txt. The same day, three companies published their own posts. OpenAI said it will honor the common signal and aims to implement it "in the coming months." Google said it aims to support the control in 2027, and plans to treat a training opt-out as if the site had disallowed Google-Extended when Google-Extended is not specified. Microsoft said it will honor the new preferences as it crawls starting February 1, 2027, and will stop reading its older NoArchive and NoCache tags as training or grounding preferences after March 1, 2027. The draft is an individual submission. It is not an IETF standard and has no formal standing in the standards process.
Why it matters for business owners
Until now, a site that wanted to say no to AI training had to name each company's crawler separately and keep that list current. A shared line lowers that work. If it is adopted widely, one entry could express one preference to every participating provider. It also forces a decision many businesses have never made on purpose. Your website, your guides, your case notes, and your product pages are content other companies may use. Whether you want that is a business choice, and today most sites have made it by default, by saying nothing. Microsoft's post adds a second choice that matters more to many businesses: grounding. Grounding means an AI answer draws on your live page when it responds. Microsoft treats it as separate from training and says that blocking it will reduce your brand's visibility in AI experiences.
What owners should not misunderstand
It is not live. As of the announcements, none of the three companies honors the signal. Microsoft gave a date, Google gave a year, and OpenAI gave a rough window. Adding the line today does not change what any of them does today. It is a preference, not a lock. The draft says the token is not a directive for crawling. It does not stop a crawler from fetching your pages, and it does not keep you out of search results. It also says private agreements can override a preference expressed this way. Your current robots.txt probably says nothing about it. Under the draft, a wildcard group (User-Agent: *) does not count as matching the token, and a file with no group for the token expresses no preference. A site that blocks all crawlers has still not said anything about training through this mechanism. It covers training only. The draft defines AI training as using content to modify the learned parameters of a generative AI model. It does not say anything about whether an AI answer may cite or draw on your page. Microsoft handles that through a separate page-level grounding setting. Old settings do not disappear. OpenAI says the new signal will not replace its existing crawler controls. Google says existing controls will keep working alongside it. Microsoft is the exception: it is retiring NoArchive and NoCache for these uses. Opting out has a cost. Microsoft says that choosing to disallow grounding means reduced brand visibility in AI experiences, and that neither choice guarantees inclusion, traffic, or payment. If AI assistants are one way customers find you, blocking everything may hurt you more than it protects you. This is not legal advice. If your content includes licensed, client, or regulated material, ask counsel what your contracts and obligations require.
The operational lesson
A website is an input to other companies' AI systems, and you are allowed to decide what happens to it. Make that decision on purpose, per type of content, instead of leaving it to defaults. Training and being cited are two different questions. A business might be comfortable letting its public guides inform AI answers, because that is how new customers find it, and still not want that same content used to train models. Another might want its premium or member content kept out of both. There is also a lesson in how fast this area moves. Dates differ by company, the draft can change, and the rules on your own site are one more setting someone has to own.
What a serious business should do next
Find out who owns your website settings. Someone should be able to open your robots.txt file and say what it currently covers. Sort your content into groups: public marketing pages, expert guides you want AI to cite, gated or client material, and anything licensed from others. Each group may deserve a different answer. Decide your position on training and on AI answers separately, and write it down in a sentence or two. Check whether you use a CDN or website platform that edits robots.txt or bot settings for you. Cloudflare, for example, has changed what its "Block" settings do before, and a platform setting can override what you intended. Do not rush to paste the line in. Wait for each provider to say it honors the signal, then add it knowing what it will and will not do. In the meantime, the existing named controls still apply. If you use Microsoft's NoArchive or NoCache tags today, plan to review them before March 1, 2027, since Microsoft says they will no longer be read for training or grounding after that date.
The Atlacis view
Atlacis helps owners slow down, understand the workflow, map the risk, and choose the right approach before spending or changing anything. A robots.txt line is a small technical step, but the question behind it is bigger: which of your content and data is meant to be used by AI systems, and by whom. The same thinking applies to the data your own team feeds into AI tools. If you want help sorting out what should be shared, protected, or kept inside your own environment, a short conversation can be a good start.
The short version
- On October 30, 2026, OpenAI, Google, and Microsoft announced plans to honor a shared robots.txt signal for AI training preferences.
- None honors it yet. Microsoft says February 1, 2027, Google says 2027, and OpenAI says "in the coming months."
- The signal comes from an individual IETF draft. It is not a standard.
- It states a training preference. It does not block crawling and does not remove you from search.
- Training and AI answers (grounding) are separate choices, and blocking grounding can reduce your visibility in AI experiences.
- A robots.txt file that blocks all crawlers has not said anything about training under this draft.
Where ATLACIS can help
Sources
- IETF: Machine-Readable Training Preference Signal (Internet-Draft, October 30, 2026)
- Google Search Central: Supporting a shared AI training control for websites (October 30, 2026)
- OpenAI: Making it easier for websites to express AI training preferences (October 30, 2026)
- Microsoft Bing: Your content, your choice: clearer preferences for training and grounding (October 30, 2026)
- The Verge: The biggest names in AI are backing a new way to opt out of scraping