What powers ericspencer.us?
Last seen Sep 2026 · 6 technologies across 5 categories
Caching
CDN
Font scripts
Miscellaneous
PaaS
AI Crawlers
Blocks 13 of 33 AI crawlers we track by name in its robots.txt. Checked Oct 2026.
Training crawlers
- Applebot-Extended Blocked
- Bytespider Blocked
- CCBot Blocked
- ClaudeBot Blocked
- cohere-training-data-crawler Blocked
- Diffbot Blocked
- Google-CloudVertexBot Blocked
- Google-Extended Blocked
- GoogleOther Blocked
- GPTBot Blocked
- meta-externalagent Blocked
Search and AI assistants
- Amazonbot Blocked
- PetalBot Blocked
The other 20 AI crawlers we track are not mentioned and only inherit the catch-all rules that apply to every crawler.
- robots.txt found
- llms.txt not found
- ai.txt not found
Content Signals:
search=yes ai-train=no declared in robots.txt
- Oct 2026 ccbot added to robots.txt
- Oct 2026 claude-web added to robots.txt
- Oct 2026 claudebot added to robots.txt
- Oct 2026 bytespider added to robots.txt
- Oct 2026 Content Signals changed
- Oct 2026 amazonbot added to robots.txt
AI Crawlers
13 of 33 AI crawlers blocked
Standards: Content Signals
Open Page Rank
1.13 / 10 · ranked #10,667,213 of all domains
2 referring domains · OpenPageRank
Infrastructure
- Hosted on Cloudflare (104.21.96.81, IPv6)
- DNS: Cloudflare DNS
- Email: Cloudflare Email Routing
- Email security: SPF (soft fail), DMARC monitor only
- SSL: Google Trust Services, ECDSA P256, wildcard, expires Dec 2026
- Common Crawl link graph: rank 6,809,224, 3 hosts
- Verified with: Google
DNS records as swept Oct 2026, certificate as seen Oct 2026. How the web is hosted