What powers magpile.com?
Last seen Jul 2026 · 15 technologies across 12 categories
Advertising
Analytics
CDN
Font Scripts
JavaScript Frameworks
JavaScript libraries
Miscellaneous
Programming languages
Security
Web Frameworks
Webmail
Widgets
AI Crawlers
Blocks 8 of 33 AI crawlers we track by name in its robots.txt. Checked Sep 2026. A recent check saw a policy file missing; awaiting confirmation.
Training crawlers
- Applebot-Extended Blocked
- Bytespider Blocked
- CCBot Blocked
- ClaudeBot Blocked
- Google-Extended Blocked
- GPTBot Blocked
- meta-externalagent Blocked
Search and AI assistants
- Amazonbot Blocked
The other 25 AI crawlers we track are not mentioned and stay allowed by default.
- robots.txt found
- llms.txt not found
- ai.txt not found
Content Signals:
search=yes ai-train=no declared in robots.txt
AI Crawlers
8 of 33 AI crawlers blocked
Standards: Content Signals
Open Page Rank
2.95 / 10 · ranked #1,680,313 of all domains
59 referring domains · OpenPageRank
Infrastructure
- Hosted on Cloudflare (104.21.7.118, IPv6)
- Email: Google Workspace
- Email security: no SPF record, no DMARC record
- SSL: Let's Encrypt, ECDSA P256, wildcard, expires Dec 2026
- Common Crawl link graph: rank 775,133, 3 hosts
DNS records as swept Aug 2026, certificate as seen Sep 2026. How the web is hosted
Full technology history →
Report an incorrect detection
Recent changes
- dropped Babel Jul 2026
- dropped core-js Jul 2026
- dropped Google Analytics Jul 2026
- dropped Nginx Jul 2026
- dropped React Jul 2026
- detected Cloudflare Browser Insights Jun 2026
- detected HTTP/3 Jun 2026
- detected Cloudflare Bot Management May 2026