What powers deserethistoricalarchives.org?
Last seen Oct 2026 · 9 technologies across 8 categories
Databases
Documentation
Font scripts
Miscellaneous
Operating systems
Programming languages
Web frameworks
Web servers
AI Crawlers
Blocks 23 of 33 AI crawlers we track by name in its robots.txt. Checked Nov 2026.
Training crawlers
- Applebot-Extended Blocked
- Bytespider Blocked
- CCBot Blocked
- ClaudeBot Blocked
- cohere-training-data-crawler Blocked
- Diffbot Blocked
- Google-CloudVertexBot Blocked
- Google-Extended Blocked
- GoogleOther Blocked
- GPTBot Blocked
- meta-externalagent Blocked
Search and AI assistants
- Amazonbot Blocked
- Applebot Blocked
- Claude-SearchBot Blocked
- OAI-SearchBot Blocked
- PerplexityBot Blocked
- PetalBot Blocked
- YouBot Blocked
User-request fetchers
- ChatGPT-User Blocked
- Claude-User Blocked
- meta-externalfetcher Blocked
- MistralAI-User Blocked
- Perplexity-User Blocked
The other 10 AI crawlers we track are not mentioned and stay allowed by default.
- robots.txt found
- llms.txt not found
- ai.txt not found
Content Signals:
search=yes ai-train=no declared in robots.txt
AI Crawlers
23 of 33 AI crawlers blocked
Standards: Content Signals
Audience
United States
Where the site's Chrome visitors come from, largest estimated audience first.
Core Web Vitals
| LCP | 0.3 s |
| INP | 25 ms |
| CLS | 0 |
| TTFB | 100 ms |
Real user p75, Chrome UX Report, Sep 2026.
Page
- Language: English (United Kingdom)
- Weight: 0.1 MB · 10 requests
- Structured data: OpenGraph
Homepage as crawled by the HTTP Archive, Oct 2026.
Infrastructure
- Hosted on Cloudflare (104.21.21.43, IPv6)
- DNS: Cloudflare DNS
- Email: no MX record returned
- SSL: Let's Encrypt, ECDSA P256, wildcard, expires Jan 2027
DNS records as swept Nov 2026, certificate as seen Nov 2026. How the web is hosted