What powers
asianlii.org?
Last seen Aug 2025 · 6 technologies across 4 categories
AI Crawlers
Blocks 13 of 33 AI crawlers we track by name in its robots.txt. Checked Nov 2026.
Training crawlers
- Applebot-Extended Blocked
- Bytespider Blocked
- CCBot Blocked
- ClaudeBot Blocked
- cohere-training-data-crawler Blocked
- Diffbot Blocked
- Google-CloudVertexBot Blocked
- Google-Extended Blocked
- GoogleOther Blocked
- GPTBot Blocked
- meta-externalagent Blocked
Search and AI assistants
- Amazonbot Blocked
- PetalBot Blocked
The other 20 AI crawlers we track are not mentioned and stay allowed by default.
- robots.txt found
- llms.txt not checked
- ai.txt not checked
Content Signals:
search=yes ai-train=no declared in robots.txt
- Oct 2026 diffbot added to robots.txt
- Oct 2026 duckassistbot removed from robots.txt
- Oct 2026 cohere-ai added to robots.txt
- Oct 2026 youbot removed from robots.txt
- Oct 2026 *: Allowed (was Restricted)
- Oct 2026 chatgpt-user removed from robots.txt
AI Crawlers
13 of 33 AI crawlers blocked
Standards: Content Signals
Open Page Rank
5.31 / 10 · ranked #193,842 of all domains
179 referring domains · OpenPageRank
Infrastructure
- Registered May 2005 (21 years) via DNC Holdings, Inc., renews May 2034
- Hosted on AARNET-AS-AP - Australian Academic and Research Network (AARNet) (138.25.65.96)
- DNS: Cloudflare DNS
- Email: austlii.edu.au
- Email security: no SPF record, no DMARC record
- SSL: GlobalSign, RSA 2048, wildcard, expires Mar 2027
- Common Crawl link graph: rank 556,178, 2 hosts
DNS records as swept Nov 2026, certificate as seen Nov 2026, registration from the registry's RDAP service. How the web is hosted