What powers unicrew.com?
Last seen Sep 2026 · 34 technologies across 18 categories
Analytics
CDN
CMS
CRM
Databases
Font scripts
JavaScript frameworks
JavaScript libraries
Miscellaneous
Performance
Programming languages
Security
SEO
Tag managers
UI frameworks
Webmail
WordPress plugins
AI Crawlers
Restricts 15 of 33 AI crawlers we track by name in its robots.txt. Checked Oct 2026.
Training crawlers
- Applebot-Extended Restricted
- Bytespider Restricted
- CCBot Restricted
- ClaudeBot Restricted
- Google-Extended Restricted
- GPTBot Restricted
- meta-externalagent Restricted
Search and AI assistants
- Amazonbot Restricted
- Applebot Restricted
- Claude-SearchBot Restricted
- OAI-SearchBot Restricted
- PerplexityBot Restricted
User-request fetchers
- ChatGPT-User Restricted
- Claude-User Restricted
- Perplexity-User Restricted
The other 18 AI crawlers we track are not mentioned and only inherit the catch-all rules that apply to every crawler.
- robots.txt found
- llms.txt found
- ai.txt not found
- Oct 2026 oai-searchbot added to robots.txt
- Oct 2026 chatgpt-user added to robots.txt
- Oct 2026 claude-searchbot added to robots.txt
- Oct 2026 ccbot added to robots.txt
- Oct 2026 amazonbot added to robots.txt
- Oct 2026 applebot added to robots.txt
AI Crawlers
15 of 33 AI crawlers restricted
Standards: llms.txt
Open Page Rank
2.45 / 10 · ranked #2,579,578 of all domains
23 referring domains · OpenPageRank
Infrastructure
- Hosted on Cloudflare (104.26.2.204, IPv6)
- DNS: Cloudflare DNS
- Email: Google Workspace
- Email security: SPF (hard fail), DMARC reject
- SSL: Google Trust Services, ECDSA P256, wildcard, expires Dec 2026
- Common Crawl link graph: rank 8,253,728, 2 hosts
- Verified with: Apple Business, Atlassian, Google, Microsoft 365, OpenAI, Zoho
DNS records as swept Oct 2026, certificate as seen Oct 2026. How the web is hosted
Full technology history →
Report an incorrect detection
Recent changes
- dropped Cloudflare Rocket Loader Jun 2026
- dropped Plesk Jun 2026
- dropped OWL Carousel Apr 2026
- dropped Plausible Apr 2026
- dropped Really Simple SSL & Security Apr 2026
- dropped WordPress Block Editor Apr 2026
- detected Amplitude Mar 2026
- detected Bootstrap Mar 2026
- detected Choices Mar 2026
- detected ClickCease Mar 2026
- detected Cloudflare Bot Management Mar 2026
- detected Cloudflare Rocket Loader Mar 2026
- detected Cloudflare Turnstile Mar 2026
- detected Google Analytics Mar 2026
- detected Google Tag Manager Mar 2026
- detected GSAP Mar 2026
- detected jQuery Migrate Mar 2026
- dropped Nginx Mar 2026
- detected Cloudflare Browser Insights Feb 2026
- detected HTTP/3 Feb 2026
- detected Site Kit Feb 2026