What powers twstar.com?
Last seen Aug 2026 · 19 technologies across 13 categories
Analytics
Blogs
CDN
Hosting
Issue trackers
JavaScript frameworks
JavaScript libraries
Miscellaneous
Programming languages
Security
UI frameworks
Web frameworks
Webmail
AI Crawlers
Blocks 10 of 33 AI crawlers we track by name in its robots.txt. Checked Sep 2026.
Training crawlers
- AI2Bot Blocked
- Applebot-Extended Blocked
- Bytespider Blocked
- CCBot Blocked
- ClaudeBot Blocked
- cohere-training-data-crawler Blocked
- Google-Extended Blocked
- GPTBot Blocked
- meta-externalagent Blocked
Search and AI assistants
- Amazonbot Blocked
The other 23 AI crawlers we track are not mentioned and only inherit the catch-all rules that apply to every crawler.
- robots.txt found
- llms.txt not found
- ai.txt not found
Content Signals:
search=yes ai-train=no ai-input=yes declared in robots.txt
- Sep 2026 ai2bot added to robots.txt
- Sep 2026 applebot-extended added to robots.txt
- Sep 2026 anthropic-ai added to robots.txt
- Sep 2026 bytespider added to robots.txt
- Sep 2026 ai2bot-dolma added to robots.txt
- Sep 2026 amazonbot added to robots.txt
AI Crawlers
10 of 33 AI crawlers blocked
Standards: Content Signals
Audience
United States
Where the site's Chrome visitors come from, largest estimated audience first.
Open Page Rank
0.44 / 10 · ranked #30,185,388 of all domains
2 referring domains · OpenPageRank
Core Web Vitals
| LCP | 1.1 s |
| CLS | 0.05 |
| TTFB | 300 ms |
Real user p75, Chrome UX Report, Aug 2026.
Page
- Language: English
- CDN: Cloudflare
- Weight: 2.5 MB · 166 requests
- Structured data: OpenGraph, Twitter Cards
Homepage as crawled by the HTTP Archive, Sep 2026.
Infrastructure
- Hosted on Amazon Web Services (3.231.35.217)
- Email: Google Workspace
- Email security: SPF (soft fail), no DMARC record
- SSL: Google Trust Services, ECDSA P256, expires Dec 2026
- Common Crawl link graph: rank 7,109,120
- Verified with: Google
DNS records as swept Sep 2026, certificate as seen Sep 2026. How the web is hosted