Websites blocking OAI-SearchBot
OpenAI's search crawler. It builds the index behind ChatGPT search features and is separate from GPTBot, so sites can allow search visibility while blocking model training.
Operated by OpenAI · Search and AI assistant · operator documentation
* block
0.61%
restrict it by name
151K domains name it and close some of its paths
1.62%
name it in robots.txt
400K domains give it rules of their own
Data for Oct 2026. All four percentages
divide by the same 24.68 million domains. Blocked counts both
explicit User-agent: OAI-SearchBot blocks and blanket
* blocks the crawler inherits.
Blocking trend
Historical mentions, HTTP Archive
High-authority sites blocking OAI-SearchBot by name
| Domain | OPROpen Page Rank | Blocking since |
|---|---|---|
| nytimes.com | 9.28/10 | Aug 2026 |
| yahoo.com | 9.27/10 | Oct 2026 |
| cnn.com | 9.16/10 | Oct 2026 |
| amazon.co.jp | 9.16/10 | Sep 2026 |
| amazon.de | 9.13/10 | Oct 2026 |
| sagepub.com | 9.07/10 | Oct 2026 |
| cbc.ca | 9.03/10 | Oct 2026 |
| upenn.edu | 9.00/10 | Oct 2026 |
| naver.com | 8.99/10 | Oct 2026 |
| uol.com.br | 8.92/10 | Oct 2026 |
| huffpost.com | 8.91/10 | Aug 2026 |
| claude.ai | 8.87/10 | Sep 2026 |
| mashable.com | 8.87/10 | Sep 2026 |
| francetvinfo.fr | 8.87/10 | Aug 2026 |
| moz.com | 8.83/10 | Oct 2026 |
| radiofrance.fr | 8.82/10 | Aug 2026 |
| wikihow.com | 8.81/10 | Aug 2026 |
| pcmag.com | 8.80/10 | Aug 2026 |
| nikkei.com | 8.79/10 | Oct 2026 |
| technologyreview.com | 8.78/10 | Oct 2026 |
| iop.org | 8.75/10 | Oct 2026 |
| epfl.ch | 8.74/10 | Oct 2026 |
| mirror.co.uk | 8.73/10 | Oct 2026 |
| rfi.fr | 8.73/10 | Aug 2026 |
| metro.co.uk | 8.72/10 | Aug 2026 |
| colorado.edu | 8.72/10 | Oct 2026 |
| ctvnews.ca | 8.72/10 | Aug 2026 |
| billboard.com | 8.69/10 | Oct 2026 |
| mainichi.jp | 8.69/10 | Sep 2026 |
| overheid.nl | 8.68/10 | Oct 2026 |
Sites whose robots.txt names OAI-SearchBot (or a legacy
alias) with a full block, ranked by Open Page Rank. Sites blocking every crawler with a blanket
* rule are counted above but not listed here. Since dates start at our first
observation of the rule.
Restricting some paths
- linkedin.com
- flickr.com
- linktr.ee
- npr.org
- hbr.org
- jotform.com
- wsj.com
- usatoday.com
- time.com
- go.com
- lnkd.in
- bbb.org
Crawler first seen in the wild around Jul 2024. How to block it and what each state means is on the methodology page.