What powers miners.tools?
Last seen Aug 2026 · 10 technologies across 8 categories
Analytics
Google Analytics
since Jul 2026
Matomo Analytics
since Jul 2026
Documentation
Hosting
GoDaddy
since Jul 2026
JavaScript frameworks
Svelte
since Jul 2026
Miscellaneous
Open Graph
since Jul 2026
PWA
since Jul 2026
Operating systems
Ubuntu
since Jul 2026
Static site generator
Web servers
AI Crawlers
Blocks 1 of 33 AI crawlers we track by name in its robots.txt, restricts 18 more. Checked Aug 2026.
Training crawlers
- Applebot-Extended Restricted
- Bytespider Restricted
- CCBot Restricted
- ClaudeBot Restricted
- cohere-training-data-crawler Restricted
- Google-Extended Restricted
- GoogleOther Restricted
- GPTBot Restricted
- meta-externalagent Restricted
Search and AI assistants
- PetalBot Blocked
- Amazonbot Restricted
- Applebot Restricted
- Claude-SearchBot Restricted
- OAI-SearchBot Restricted
- PerplexityBot Restricted
User-request fetchers
- ChatGPT-User Restricted
- Claude-User Restricted
- meta-externalfetcher Restricted
- Perplexity-User Restricted
The other 14 AI crawlers we track are not mentioned and only inherit the catch-all rules that apply to every crawler.
- robots.txt found
- llms.txt found
- llms-full.txt found
- ai.txt not found
AI Crawlers
1 of 33 AI crawlers blocked
Standards: llms.txt, llms-full.txt
Open Page Rank
0.34 / 10 · ranked #45,039,167 of all domains
0 referring domains · OpenPageRank
Page
- Language: English
- Weight: 0.2 MB · 15 requests
- Structured data: JSON-LD, OpenGraph, Twitter Cards
Homepage as crawled by the HTTP Archive, Aug 2026.