Methodology

How PoweredBy detects, counts, and tracks web technologies, where the authority, traffic, and performance numbers next to them come from, and the choices we made.

What we count

Everything on PoweredBy counts registrable domains (per the Public Suffix List, including its private section): nike.com is one site, and www.nike.com folds into it, while someone.github.io counts as its own site. Subdomains are tracked underneath each domain and shown on its page, but market-share numbers never count a domain twice.

Where detections come from

Detections are produced by the open-source Wappalyzer ruleset maintained by HTTP Archive, the same fingerprints HTTP Archive runs in real Chrome browsers for its monthly crawl of the web's homepages. We pin an exact snapshot of that ruleset, so our numbers are consistent with the underlying crawl by construction. Our history is built from HTTP Archive's monthly datasets (mobile crawl, root pages), which we normalize, quality-check, and load into our own database.

Corpus history, read this before comparing months

Apr 2018 to Jul 2018: the crawl targeted roughly 450K sites (Alexa-based list), and technology detection was in its rollout period, about 12% of pages produced no detections at all. We treat those as failed crawls, not as evidence a site dropped its stack.
Aug 2018 onward: the corpus switched to the Chrome UX Report (CrUX) origin list and jumped to 1.3M+ pages, growing steadily since (5M+ by 2019, 15M+ today). Detection quality stabilized (~1% empty pages).
CrUX popularity ranks exist from May 2021 onward. We use them internally (ordering long lists, choosing which pages search engines should index); the authority figures shown on pages are Open Page Rank.

When do we say a site "dropped" a technology?

Only after two consecutive successful render-grade crawls of the same host both fail to detect it, and every host of the domain that carried the technology has independently confirmed the absence. Blocked, errored, or robots-disallowed fetches never count as absence, and a host disappearing from the crawl corpus doesn't either. This is deliberately conservative: false "X dropped Y!" stories are worse than a few days of staleness.

Historical technology names

Fingerprint rulesets rename and retire technologies over the years (Piwik became Matomo; IBM WebSphere Commerce became HCL Commerce; Google+ died). We maintain a curated mapping so history stays continuous under today's names, and genuinely discontinued products keep their own pages - their adoption and disappearance are part of the record.

Authority scores: Open Page Rank

The authority score shown for domains is Open Page Rank: 0 to 10, computed from referring domains across the open web link graph, with an exact rank among more than 121 million scored domains. We load the full dataset and refresh it as new releases land. It also orders every "websites using X" list: highest authority first, then Chrome UX popularity, then everything else. A domain without a score simply shows no authority block.

Audience and Core Web Vitals: Chrome UX Report

Audience countries, device splits, and Core Web Vitals come from the Chrome UX Report (CrUX), Google's monthly dataset of real Chrome usage, which covers about 12 million domains. Three things to keep in mind:

Countries are where a site's Chrome visitors browse from, not where its servers sit.
Web Vitals (LCP, INP, CLS, TTFB) are p75 values from real users, not lab scores.
Per-technology vitals are the median site among that technology's CrUX-covered sites. They describe the sites that chose the technology, not overhead the technology causes.

Search traffic, keywords, and backlinks

Estimated monthly organic visits, ranking-keyword counts (Google top 30), top keywords, and top backlinks come from Keywords Everywhere, for the US Google market. These are estimates from a search index, not the site's own analytics: treat them as order-of-magnitude signals. We refresh them in the background on a rolling cycle, prioritized by site authority, with full keyword and backlink panels for higher-authority domains and visit estimates for the rest, so figures can lag the live index by a few months. Keyword and backlink lists show the top 25 each, backlinks one per source domain; technology pages show the same visit estimates for the sites they list.

Page facts

Language, CDN, page weight and request counts, WordPress themes, and structured-data flags come from the same monthly HTTP Archive crawl as our detections, so they describe the site's homepage as that crawl fetched it. Page weight is what the crawl's mobile Chrome downloaded for the root page, which can differ from what your browser loads.

Freshness

HTTP Archive publishes monthly; we ingest each new month shortly after it lands. Any site can be rescanned live from its own page (the Rescan button), and a site we have never seen is scanned the moment someone looks it up. Every domain page shows when its data was last seen. Authority, audience, and search figures refresh on their own cadences, noted in their sections above.

Known limitations

Homepage-only detection: technologies used solely on inner pages can be missed.
Server-side stacks with no client-visible fingerprint (headers, cookies, markup) are undetectable by anyone using this approach.
Version numbers exist only where a fingerprint exposes them; absence of a version is not absence of the technology.
Detections before Aug 2018 cover a much smaller corpus (see above), treat early-2018 first-seen dates as "at least since".
Authority, audience, vitals, and search figures cover only domains present in their source datasets; a missing block means no data, not zero.
Search traffic, keyword, and backlink figures are estimates for the US Google market, not measurements from the site itself.

Attribution

Crawl data: HTTP Archive (thank you). Popularity ranks: Chrome UX Report. Domain rules: the Public Suffix List. Fingerprints: HTTP Archive's Wappalyzer fork (GPL-3.0, used server-side). Authority scores: Open Page Rank. Search estimates: Keywords Everywhere. Cite us as: "PoweredBy, the free technology profiler from Keywords Everywhere."