Websites blocking Applebot-Extended
Apple's robots.txt control token for AI training. Disallowing it opts a site's content out of training Apple's foundation models while leaving Applebot's search functions, such as Siri and Spotlight, unaffected.
Operated by Apple · Training crawler · operator documentation
* block
2.52%
restrict it by name
627K domains name it and close some of its paths
8.94%
name it in robots.txt
2.23 million domains give it rules of their own
Data for Aug 2026. All four percentages
divide by the same 24.91 million domains. Blocked counts both
explicit User-agent: Applebot-Extended blocks and blanket
* blocks the crawler inherits.
Historical mentions, HTTP Archive
High-authority sites blocking Applebot-Extended by name
| Domain | OPROpen Page Rank | Blocking since |
|---|---|---|
| fbcdn.net | 9.40/10 | Aug 2026 |
| nytimes.com | 9.28/10 | Aug 2026 |
| tumblr.com | 9.27/10 | Aug 2026 |
| bbc.com | 9.15/10 | Aug 2026 |
| cookielaw.org | 9.10/10 | Aug 2026 |
| usatoday.com | 9.08/10 | Aug 2026 |
| tripadvisor.com | 9.05/10 | Aug 2026 |
| bizjournals.com | 9.04/10 | Aug 2026 |
| msn.com | 9.04/10 | Aug 2026 |
| mayoclinic.org | 9.04/10 | Aug 2026 |
| pexels.com | 9.03/10 | Aug 2026 |
| theatlantic.com | 9.01/10 | Aug 2026 |
| investopedia.com | 9.01/10 | Aug 2026 |
| hugedomains.com | 9.00/10 | Aug 2026 |
| uk.com | 9.00/10 | Aug 2026 |
| patreon.com | 8.99/10 | Aug 2026 |
| academia.edu | 8.99/10 | Aug 2026 |
| healthline.com | 8.98/10 | Aug 2026 |
| ebay.com | 8.95/10 | Aug 2026 |
| science.org | 8.95/10 | Aug 2026 |
| usgs.gov | 8.95/10 | Aug 2026 |
| quora.com | 8.92/10 | Aug 2026 |
| utexas.edu | 8.92/10 | Aug 2026 |
| huffpost.com | 8.91/10 | Aug 2026 |
| newyorker.com | 8.90/10 | Aug 2026 |
| mashable.com | 8.87/10 | Aug 2026 |
| francetvinfo.fr | 8.87/10 | Aug 2026 |
| jsfiddle.net | 8.86/10 | Aug 2026 |
| nami.org | 8.86/10 | Aug 2026 |
| smh.com.au | 8.86/10 | Aug 2026 |
Sites whose robots.txt names Applebot-Extended (or a legacy
alias) with a full block, ranked by Open Page Rank. Sites blocking every crawler with a blanket
* rule are counted above but not listed here. Since dates start at our first
observation of the rule.
Restricting some paths
- medium.com
- statista.com
- hbr.org
- jotform.com
- squarespace.com
- frontiersin.org
- iubenda.com
- theverge.com
- nbcnews.com
- vox.com
- newsweek.com
- buzzfeed.com
Crawler first seen in the wild around Jun 2024. How to block it and what each state means is on the methodology page.