What powers mitdir.jp?
Last seen Jul 2026 · 20 technologies across 13 categories
Analytics
CDN
CMS
Development
Ecommerce
JavaScript frameworks
Marketing automation
Miscellaneous
PaaS
Performance
Programming languages
Tag managers
UI frameworks
AI Crawlers
Blocks 7 of 33 AI crawlers we track by name in its robots.txt, restricts 3 more. Checked Aug 2026.
Training crawlers
- Applebot-Extended Blocked
- Bytespider Blocked
- CCBot Blocked
- Google-Extended Blocked
- GPTBot Blocked
- meta-externalagent Blocked
- ClaudeBot Restricted
Search and AI assistants
- Amazonbot Blocked
- PerplexityBot Restricted
User-request fetchers
- ChatGPT-User Restricted
The other 23 AI crawlers we track are not mentioned and only inherit the catch-all rules that apply to every crawler.
- robots.txt found
- llms.txt found
- ai.txt not found
Content Signals:
search=yes ai-train=no declared in robots.txt
AI Crawlers
7 of 33 AI crawlers blocked
Standards: llms.txt, Content Signals
Open Page Rank
1.46 / 10 · ranked #6,893,993 of all domains
3 referring domains · OpenPageRank