What powers newsjazeera.com?
Last seen Sep 2026 · 16 technologies across 12 categories
Advertising
Analytics
Blogs
CDN
Cookie compliance
Font scripts
JavaScript libraries
Miscellaneous
Performance
Programming languages
Security
Web servers
AI Crawlers
Blocks 18 of 33 AI crawlers we track by name in its robots.txt. Checked Oct 2026.
Training crawlers
- Applebot-Extended Blocked
- Bytespider Blocked
- CCBot Blocked
- ClaudeBot Blocked
- cohere-training-data-crawler Blocked
- Diffbot Blocked
- Google-CloudVertexBot Blocked
- Google-Extended Blocked
- GoogleOther Blocked
- GPTBot Blocked
- meta-externalagent Blocked
Search and AI assistants
- Amazonbot Blocked
- PetalBot Blocked
User-request fetchers
- ChatGPT-User Blocked
- Claude-User Blocked
- meta-externalfetcher Blocked
- MistralAI-User Blocked
- Perplexity-User Blocked
The other 15 AI crawlers we track are not mentioned and stay allowed by default.
- robots.txt found
- llms.txt not found
- ai.txt not found
Content Signals:
search=yes ai-train=no declared in robots.txt
- Oct 2026 bytespider added to robots.txt
- Oct 2026 ccbot added to robots.txt
- Oct 2026 chatgpt-user added to robots.txt
- Oct 2026 claudebot added to robots.txt
- Oct 2026 claude-web added to robots.txt
- Oct 2026 claude-user added to robots.txt
AI Crawlers
18 of 33 AI crawlers blocked
Standards: Content Signals
Open Page Rank
0.28 / 10 · ranked #55,151,345 of all domains
1 referring domains · OpenPageRank
Infrastructure
- Hosted on Cloudflare (104.21.70.217, IPv6)
- DNS: Cloudflare DNS
- Email: no MX record returned
- SSL: Let's Encrypt, ECDSA P256, wildcard, expires Dec 2026
- Common Crawl link graph: rank 111,605,881
DNS records as swept Oct 2026, certificate as seen Oct 2026. How the web is hosted
Full technology history →
Report an incorrect detection
Recent changes
- dropped Cloudflare Bot Management Sep 2026
- dropped Google Tag Manager Sep 2026
- detected Priority Hints Sep 2026
- detected Blogger Jul 2026
- detected cdnjs Jul 2026
- detected Cloudflare Jul 2026
- detected Cloudflare Bot Management Jul 2026
- detected Font Awesome Jul 2026
- detected Funding Choices Jul 2026
- detected Google AdSense Jul 2026
- detected Google Analytics Jul 2026
- detected Google Font API Jul 2026
- detected Google Tag Manager Jul 2026
- detected Java Jul 2026
- detected jQuery Jul 2026
- detected Open Graph Jul 2026
- detected OpenGSE Jul 2026
- detected Python Jul 2026
- detected reCAPTCHA Jul 2026
- detected RSS Jul 2026
- detected Google Tag Manager Jun 2026
- dropped Google Tag Manager Dec 2025