What powers listdefender.com?
Last seen Oct 2026 · 29 technologies across 17 categories
A/B Testing
Analytics
CDN
CMS
CRM
Databases
Font scripts
JavaScript libraries
Miscellaneous
Performance
Programming languages
Security
SEO
Tag managers
Video players
Webmail
WordPress plugins
AI Crawlers
Does not block any of the 33 AI crawlers we track in its robots.txt. Checked Oct 2026.
Training crawlers
- Applebot-Extended Allowed
- CCBot Allowed
- ClaudeBot Allowed
- cohere-training-data-crawler Allowed
- Google-Extended Allowed
- GPTBot Allowed
- meta-externalagent Allowed
Search and AI assistants
- Amazonbot Allowed
- Claude-SearchBot Allowed
- DuckAssistBot Allowed
- OAI-SearchBot Allowed
- PerplexityBot Allowed
User-request fetchers
- ChatGPT-User Allowed
- Claude-User Allowed
- MistralAI-User Allowed
- Perplexity-User Allowed
The other 17 AI crawlers we track are not mentioned and stay allowed by default.
- robots.txt found
- llms.txt found
- llms-full.txt found
- ai.txt not found
- Oct 2026 mistralai-user added to robots.txt
- Oct 2026 claude-searchbot added to robots.txt
- Oct 2026 claude-user added to robots.txt
- Oct 2026 claudebot added to robots.txt
- Oct 2026 chatgpt-user added to robots.txt
- Oct 2026 amazonbot added to robots.txt
AI Crawlers
No AI crawlers blocked
Standards: llms.txt, llms-full.txt
Open Page Rank
2.83 / 10 · ranked #1,904,576 of all domains
11 referring domains · OpenPageRank
Page
- Language: English
- CDN: Cloudflare
- Weight: 1.3 MB · 44 requests
- Structured data: JSON-LD, OpenGraph, Twitter Cards
Homepage as crawled by the HTTP Archive, Oct 2026.
Organisation
- ListDefender
- Industry: Software and apps (declares
SoftwareApplication)
From the homepage's own structured data (Oct 2026).
Infrastructure
- Hosted on Cloudflare (104.21.2.189, IPv6)
- DNS: Cloudflare DNS
- Email: Google Workspace
- Email security: SPF (hard fail), DMARC reject
- SSL: Google Trust Services, ECDSA P256, wildcard, expires Nov 2026
- Common Crawl link graph: rank 9,523,950, 3 hosts
- Verified with: Google
DNS records as swept Oct 2026, certificate as seen Oct 2026. How the web is hosted