The New York Times publisher AG Sulzberger slammed AI companies (Photo/Canva)
AI giants OpenAI, Google, Microsoft, and Anthropic are looking to adopt a new standard that would let publishers collectively signal that their content must not be used for AI training purposes.
The opt-out standard known as ‘policy token’ is a shared signal proposed by Kevin Kelley, a technical lead for standards at Anthropic, as part of the Internet Engineering Task Force (IETF). Publishers will be able to add the policy token to their website’s robots.txt file, which has long served as a protocol for which web crawlers can and cannot scrape a site, whether for search indexing, archiving, or AI training.
However, the policy token is slightly different as it lets publishers indicate their AI training preferences across multiple AI companies, instead of doing it separately for each AI company.
The move comes against the backdrop of a long-running tussle between publishers and tech platforms that has only intensified in the…