A new signal is coming to websites against artificial intelligence

Leading AI companies have announced a new standard to simplify how publishers manage bot access to their content. OpenAI, Google, Microsoft, and Anthropic will implement a 'policy token' in the robots.txt file of websites, allowing AI systems to recognize permissions more efficiently. This aims to address issues related to server traffic costs and the unauthorized use of publishers' intellectual property for AI training. Microsoft has committed to respecting these new signals, although it will not affect Bing's indexing of the sites. Anthropic has already adopted this system, while Google and Microsoft plan to support it next year, with OpenAI expected to follow in the coming months.