ai.txt adoption

ai.txt is a permissions file at /ai.txt, proposed by Spawning as a way for sites to declare whether their content may be used for AI training. It predates most robots.txt AI-crawler tokens and remains a niche convention.

0.18% publish ai.txt 52,354 of 28.64 million domains that answered our crawl

Data for Aug 2026, from our monthly crawl. Every percentage divides by the base the first tile names.

Notable adopters

Ranked by Open Page Rank. The per-domain page shows each site's full AI crawler policy.

Where this standard actually stands

ai.txt never reached broad adoption and most AI crawler operators do not document reading it. The robots.txt tokens on the AI Crawlers page are where the real policy action is; we track ai.txt because the files still exist and some tools still write them.

Counting method and definitions are on the methodology page.

Look up any website for free

No paywall, no lead-gen, no account. Paste a domain and see what powers it.

No account neededHistory since 2018