What powers adityaarsharma.com?
Last seen Nov 2025 · 23 technologies across 13 categories
Analytics
CDN
CMS
Databases
JavaScript libraries
Miscellaneous
Performance
Programming languages
Security
Webmail
Widgets
WordPress plugins
WordPress themes
AI Crawlers
Restricts 22 of 33 AI crawlers we track by name in its robots.txt. Checked Oct 2026. A recent check saw a policy file missing; awaiting confirmation.
Training crawlers
- AI2Bot Restricted
- Applebot-Extended Restricted
- Bytespider Restricted
- CCBot Restricted
- ClaudeBot Restricted
- cohere-training-data-crawler Restricted
- Diffbot Restricted
- FacebookBot Restricted
- Google-Extended Restricted
- GoogleOther Restricted
- GPTBot Restricted
- meta-externalagent Restricted
- Webzio-Extended Restricted
Search and AI assistants
- Amazonbot Restricted
- DuckAssistBot Restricted
- OAI-SearchBot Restricted
- PerplexityBot Restricted
- PetalBot Restricted
- YouBot Restricted
User-request fetchers
- ChatGPT-User Restricted
- meta-externalfetcher Restricted
- MistralAI-User Restricted
The other 11 AI crawlers we track are not mentioned and only inherit the catch-all rules that apply to every crawler.
- robots.txt found
- llms.txt found
- llms-full.txt found
- ai.txt not found
Content Signals:
search=yes ai-train=yes ai-input=yes declared in robots.txt
- Oct 2026 ccbot: Restricted (was Allowed)
- Oct 2026 claude-web: Restricted (was Allowed)
- Oct 2026 claudebot: Restricted (was Allowed)
- Oct 2026 chatgpt-user: Restricted (was Allowed)
- Oct 2026 diffbot: Restricted (was Allowed)
- Oct 2026 cohere-ai: Restricted (was Allowed)
AI Crawlers
22 of 33 AI crawlers restricted
Standards: llms.txt, llms-full.txt, Content Signals
Open Page Rank
1.29 / 10 · ranked #8,729,858 of all domains
7 referring domains · OpenPageRank
Infrastructure
- Hosted on Cloudflare (104.21.28.213, IPv6)
- DNS: Cloudflare DNS
- Email: Google Workspace
- Email security: SPF (hard fail), DMARC quarantine, DNSSEC signed
- SSL: Google Trust Services, ECDSA P256, wildcard, expires Dec 2026
- Common Crawl link graph: rank 1,989,245, 4 hosts
- Verified with: Brevo, Google, OpenAI
DNS records as swept Oct 2026, certificate as seen Oct 2026. How the web is hosted