What powers sub-agents.directory?
Last seen Oct 2026 · 20 technologies across 11 categories
Analytics
Caching
CDN
Development
Font scripts
JavaScript frameworks
Miscellaneous
PaaS
Performance
Programming languages
UI frameworks
AI Crawlers
Does not block any of the 33 AI crawlers we track in its robots.txt. Checked Oct 2026.
Training crawlers
- AI2Bot Allowed
- Applebot-Extended Allowed
- Bytespider Allowed
- CCBot Allowed
- ClaudeBot Allowed
- cohere-training-data-crawler Allowed
- Diffbot Allowed
- FacebookBot Allowed
- Google-Extended Allowed
- GPTBot Allowed
- ImagesiftBot Allowed
- img2dataset Allowed
- meta-externalagent Allowed
- Webzio-Extended Allowed
Search and AI assistants
- Amazonbot Allowed
- Applebot Allowed
- OAI-SearchBot Allowed
- PerplexityBot Allowed
- PetalBot Allowed
- YouBot Allowed
User-request fetchers
- ChatGPT-User Allowed
- meta-externalfetcher Allowed
The other 11 AI crawlers we track are not mentioned and stay allowed by default.
- robots.txt found
- llms.txt found
- ai.txt not found
Content Signals:
search=yes ai-train=yes ai-input=yes declared in robots.txt
AI Crawlers
No AI crawlers blocked
Standards: llms.txt, Content Signals
Open Page Rank
1.76 / 10 · ranked #4,561,427 of all domains
5 referring domains · OpenPageRank
Page
- Language: English
- CDN: Fastly
- Weight: 0.8 MB · 51 requests
- Structured data: JSON-LD, OpenGraph, Twitter Cards
Homepage as crawled by the HTTP Archive, Oct 2026.
Organisation
- Sub-Agents Directory
- Profiles: GitHub
From the homepage's own structured data (Oct 2026).
Infrastructure
- Hosted on Fastly (185.199.108.153)
- DNS: Cloudflare DNS
- Email: no MX record returned
- SSL: Google Trust Services, ECDSA P256, expires Nov 2026
- Common Crawl link graph: rank 31,804,577, 2 hosts
- Verified with: Google
DNS records as swept Oct 2026, certificate as seen Sep 2026. How the web is hosted
Full technology history →
Report an incorrect detection
Recent changes
- dropped HSTS Apr 2026
- dropped Vercel Apr 2026
- detected Cloudflare Browser Insights Mar 2026
- detected HTTP/3 Mar 2026