What powers castlelabs.io?
Last seen Oct 2026 · 24 technologies across 15 categories
Analytics
Blogs
CDN
Development
Issue trackers
JavaScript frameworks
JavaScript graphics
JavaScript libraries
Miscellaneous
Performance
Programming languages
Security
UI frameworks
Web frameworks
Webmail
AI Crawlers
Blocks 3 of 33 AI crawlers we track by name in its robots.txt. Checked Oct 2026.
Training crawlers
- Bytespider Blocked
- meta-externalagent Blocked
- Applebot-Extended Allowed
- CCBot Allowed
- ClaudeBot Allowed
- Google-Extended Allowed
- GPTBot Allowed
Search and AI assistants
- Amazonbot Blocked
- Claude-SearchBot Allowed
- DuckAssistBot Allowed
- OAI-SearchBot Allowed
- PerplexityBot Allowed
User-request fetchers
- ChatGPT-User Allowed
- Claude-User Allowed
- Perplexity-User Allowed
The other 18 AI crawlers we track are not mentioned and stay allowed by default.
- robots.txt found
- llms.txt found
- llms-full.txt found
- ai.txt not found
Content Signals:
search=yes ai-train=yes ai-input=yes declared in robots.txt
- Oct 2026 gptbot added to robots.txt
- Oct 2026 applebot-extended added to robots.txt
- Oct 2026 ccbot added to robots.txt
- Oct 2026 bytespider added to robots.txt
- Oct 2026 chatgpt-user added to robots.txt
- Oct 2026 llms.txt appeared
AI Crawlers
3 of 33 AI crawlers blocked
Standards: llms.txt, llms-full.txt, Content Signals
Open Page Rank
1.20 / 10 · ranked #9,897,181 of all domains
5 referring domains · OpenPageRank
Search traffic
0 est. monthly organic visits (US)
1 keywords in Google's top 30
Page
- Language: English
- CDN: Cloudflare
- Weight: 0.9 MB · 67 requests
- Structured data: JSON-LD, OpenGraph, Twitter Cards
Homepage as crawled by the HTTP Archive, Oct 2026.
Organisation
From the homepage's own structured data (Oct 2026).
Infrastructure
- Hosted on Cloudflare (104.21.91.28, IPv6)
- DNS: Cloudflare DNS
- Email: Google Workspace
- Email security: SPF (soft fail), DMARC quarantine
- SSL: Google Trust Services, ECDSA P256, expires Dec 2026
- Common Crawl link graph: rank 7,589,659, 2 hosts
- Verified with: Google, LinkedIn
DNS records as swept Oct 2026, certificate as seen Oct 2026. How the web is hosted
Subdomains
| Host | Techs |
|---|---|
| research.castlelabs.io | 17 |
| castlelabs.io | 15 |