What powers thebyteline.com?
Last seen Oct 2026 · 23 technologies across 13 categories
Analytics
CDN
Font scripts
Hosting
JavaScript frameworks
JavaScript libraries
Miscellaneous
PaaS
Page builders
Performance
Security
UI frameworks
Webmail
AI Crawlers
Restricts 6 of 33 AI crawlers we track by name in its robots.txt. Checked Oct 2026.
Training crawlers
- ClaudeBot Restricted
- Google-Extended Restricted
- GPTBot Restricted
- Applebot-Extended Allowed
- Bytespider Allowed
- CCBot Allowed
- cohere-training-data-crawler Allowed
- FacebookBot Allowed
- meta-externalagent Allowed
Search and AI assistants
- OAI-SearchBot Restricted
- PerplexityBot Restricted
- Amazonbot Allowed
- Applebot Allowed
- DuckAssistBot Allowed
- YouBot Allowed
User-request fetchers
- ChatGPT-User Restricted
- Claude-User Allowed
- MistralAI-User Allowed
- Perplexity-User Allowed
The other 14 AI crawlers we track are not mentioned and only inherit the catch-all rules that apply to every crawler.
- robots.txt found
- llms.txt found
- llms-full.txt found
- ai.txt not found
- Sep 2026 applebot-extended added to robots.txt
- Sep 2026 bytespider added to robots.txt
- Sep 2026 ccbot added to robots.txt
- Sep 2026 claudebot added to robots.txt
- Sep 2026 claude-user added to robots.txt
- Sep 2026 chatgpt-user added to robots.txt
AI Crawlers
6 of 33 AI crawlers restricted
Standards: llms.txt, llms-full.txt
Open Page Rank
1.22 / 10 · ranked #9,626,249 of all domains
8 referring domains · OpenPageRank
Page
- Language: English
- CDN: Vercel
- Weight: 3.0 MB · 172 requests
- Structured data: JSON-LD, OpenGraph, Twitter Cards
Homepage as crawled by the HTTP Archive, Oct 2026.
Organisation
- Byteline
- Industry: News and media (declares
NewsMediaOrganization) - Profiles: X
From the homepage's own structured data (Oct 2026).
Infrastructure
- Hosted on Amazon Web Services (216.150.1.1)
- DNS: GoDaddy DNS
- Email: Google Workspace
- Email security: SPF (soft fail), no DMARC record, DNSSEC signed
- SSL: Let's Encrypt, RSA 2048, expires Nov 2026
- Common Crawl link graph: rank 15,343,195
- Verified with: Google
DNS records as swept Oct 2026, certificate as seen Oct 2026. How the web is hosted