What powers 
targetscan.org?
Last seen Aug 2026 · 4 technologies across 4 categories
CDN
JavaScript libraries
Operating systems
Web Servers
AI Crawlers
Blocks 32 of 33 AI crawlers we track by name in its robots.txt. Checked Sep 2026.
Training crawlers
- AI2Bot Blocked
- Applebot-Extended Blocked
- Bytespider Blocked
- CCBot Blocked
- ClaudeBot Blocked
- cohere-training-data-crawler Blocked
- Diffbot Blocked
- FacebookBot Blocked
- Google-CloudVertexBot Blocked
- Google-Extended Blocked
- GoogleOther Blocked
- GPTBot Blocked
- ImagesiftBot Blocked
- img2dataset Blocked
- meta-externalagent Blocked
- PanguBot Blocked
- SBIntuitionsBot Blocked
- Webzio-Extended Blocked
Search and AI assistants
- Amazonbot Blocked
- Applebot Blocked
- Claude-SearchBot Blocked
- DuckAssistBot Blocked
- OAI-SearchBot Blocked
- PerplexityBot Blocked
- PetalBot Blocked
- YouBot Blocked
User-request fetchers
- ChatGPT-User Blocked
- Claude-User Blocked
- GoogleAgent-Mariner Blocked
- meta-externalfetcher Blocked
- MistralAI-User Blocked
- Perplexity-User Blocked
The other 1 AI crawler we track is not mentioned and stays allowed by default.
- robots.txt found
- llms.txt not found
- ai.txt not found
- Aug 2026 chatgpt-user added to robots.txt
- Aug 2026 claude-searchbot added to robots.txt
- Aug 2026 claude-user added to robots.txt
- Aug 2026 claude-web added to robots.txt
- Aug 2026 applebot added to robots.txt
- Aug 2026 claudebot added to robots.txt
AI Crawlers
32 of 33 AI crawlers blocked
Audience
- 97% desktop
- 3% phone
United States
India
Where the site's Chrome visitors come from, largest estimated audience first.
Open Page Rank
4.69 / 10 · ranked #341,129 of all domains
110 referring domains · OpenPageRank
Core Web Vitals
| LCP | 1.4 s |
| INP | 50 ms |
| CLS | 0 |
| TTFB | 1100 ms |
Real user p75, Chrome UX Report, Aug 2026.
Page
- Weight: 0.1 MB · 7 requests
Homepage as crawled by the HTTP Archive, Sep 2026.
Infrastructure
- Hosted on MIT-GATEWAYS (18.4.1.182)
- DNS: targetscan.org
- Email: mit.edu
- Email security: no SPF record, no DMARC record
- SSL: Let's Encrypt, RSA 2048, expires Nov 2026
- Common Crawl link graph: rank 530,311, 2 hosts
DNS records as swept Sep 2026, certificate as seen Sep 2026. How the web is hosted