What powers rgbots.co.uk?

Last seen Aug 2026 · 7 technologies across 6 categories

Analytics

CDN

Cloudflare
since Feb 2025

Live chat

WhatsApp Business Chat
since Aug 2026

Miscellaneous

HTTP/3
since Jul 2026
Open Graph
since Feb 2025

Performance

Priority Hints
since Aug 2026

Webmail

Google Workspace
since Feb 2025

AI Crawlers

Blocks 8 of 33 AI crawlers we track by name in its robots.txt. Checked Aug 2026.

Training crawlers

  • Applebot-Extended Blocked
  • Bytespider Blocked
  • CCBot Blocked
  • ClaudeBot Blocked
  • Google-Extended Blocked
  • GPTBot Blocked
  • meta-externalagent Blocked

Search and AI assistants

  • Amazonbot Blocked

The other 25 AI crawlers we track are not mentioned and stay allowed by default.

Content Signals: search=yes ai-train=no declared in robots.txt

See who blocks each AI crawler across the web

AI Crawlers

8 of 33 AI crawlers blocked

Standards: llms.txt, Content Signals

Full breakdown

Page

  • Language: English
  • CDN: Cloudflare
  • Weight: 0.1 MB · 9 requests
  • Structured data: JSON-LD, OpenGraph, Twitter Cards

Homepage as crawled by the HTTP Archive, Aug 2026.

Full technology history →
Report an incorrect detection

Recent changes

Look up any website for free

No paywall, no lead-gen, no account. Paste a domain and see what powers it.

No account neededHistory since 2018