What powers agentsfeed.org?
Last seen Aug 2026 · 16 technologies across 10 categories
Analytics
Cloudflare Browser Insights
since Jul 2026
Authentication
NextAuth.js
since Jul 2026
CDN
Cloudflare
since Jul 2026
CMS
Contentful
since Aug 2026
Wagtail
since Aug 2026
JavaScript frameworks
Next.js App Router
since Jul 2026
React
since Jul 2026
Miscellaneous
Performance
Priority Hints
since Jul 2026
Programming languages
UI frameworks
Tailwind CSS
since Jul 2026
Web frameworks
AI Crawlers
Blocks 8 of 33 AI crawlers we track by name in its robots.txt. Checked Aug 2026.
Training crawlers
- Applebot-Extended Blocked
- Bytespider Blocked
- CCBot Blocked
- ClaudeBot Blocked
- Google-Extended Blocked
- GPTBot Blocked
- meta-externalagent Blocked
Search and AI assistants
- Amazonbot Blocked
The other 25 AI crawlers we track are not mentioned and stay allowed by default.
- robots.txt found
- llms.txt not checked
- ai.txt not checked
Content Signals:
search=yes ai-train=no declared in robots.txt
Full technology history →
Report an incorrect detection
Recent changes
- detected Contentful Aug 2026
- detected Django Aug 2026
- detected Python Aug 2026
- detected Wagtail Aug 2026