What powers thoughtstuff.co.uk?
Last seen Aug 2026 · 28 technologies across 18 categories
Advertising
Analytics
Caching
CDN
CMS
Cookie compliance
Databases
Font scripts
JavaScript libraries
Miscellaneous
PaaS
Performance
Programming languages
Security
Web frameworks
Webmail
WordPress plugins
WordPress themes
AI Crawlers
Blocks 5 of 33 AI crawlers we track by name in its robots.txt. Checked Sep 2026.
Training crawlers
- Bytespider Blocked
- CCBot Blocked
- ClaudeBot Blocked
- Google-Extended Blocked
- GPTBot Blocked
The other 28 AI crawlers we track are not mentioned and only inherit the catch-all rules that apply to every crawler.
- robots.txt found
- llms.txt found
- ai.txt not found
Content Signals:
search=yes ai-train=no ai-input=yes declared in robots.txt
- Sep 2026 claudebot: Blocked (was Restricted)
- Sep 2026 cohere-ai removed from robots.txt
- Sep 2026 meta-externalagent removed from robots.txt
- Sep 2026 ai2bot-dolma removed from robots.txt
- Sep 2026 gptbot: Blocked (was Restricted)
- Sep 2026 img2dataset removed from robots.txt
AI Crawlers
5 of 33 AI crawlers blocked
Standards: llms.txt, Content Signals
Open Page Rank
3.49 / 10 · ranked #902,177 of all domains
49 referring domains · OpenPageRank