Content-Signal adoption

Content signals are machine-readable usage preferences carried inside robots.txt: search, ai-train, and ai-input, each set to yes or no. The format was introduced by Cloudflare in 2025 and can be injected automatically for sites using its managed robots.txt.

4.11% declare content signals 844K of 20.54 million domains with a readable robots.txt 3.42% say no to AI training 703K domains declare ai-train=no 0.02% say no to AI input 4,455 domains declare ai-input=no, no AI answers from their content <0.01% say no to search 312 domains declare search=no, rare by design

Data for Oct 2026, from our monthly crawl. Every percentage divides by the base the first tile names.

Adoption over time

Domains in each monthly crawl declaring content signals.

Notable adopters

Ranked by Open Page Rank. The per-domain page shows each site's full AI crawler policy.

Where this standard actually stands

Content signals are new and their reach mirrors Cloudflare's: a large share of declarations arrive through Cloudflare's managed robots.txt rather than hand-edited files, so adoption measures platform rollout as much as individual publisher choice. Whether AI operators honor the signals is a separate question that a robots.txt file cannot answer.

Counting method and definitions are on the methodology page.

Look up any website for free

No paywall, no lead-gen, no account. Paste a domain and see what powers it.

No account neededHistory since 2018