Key takeaways

We’ve updated our Akamai Bot Directory to split the single AI Bots category into three distinct types: AI training crawlers, AI search crawlers, and AI fetchers and agents.

Businesses can now move beyond blanket “block or allow” rules and apply precise security policies based on what a bot actually does (e.g., restrict training scrapers but allow search indexers).

Differentiating real-time user agents from background crawlers enables sites to safely facilitate AI-driven customer interactions, research, and transactions.

The update pairs traffic visibility and control with optimization strategies, helping organizations safely protect their IP while maintaining brand presence in AI search results.

AI bots have quickly become a regular part of internet traffic, but not every AI bot interacts with a website for the same reason. Some crawl content to train AI models. Some index content for AI-powered search.…