crawlora-net/techstack-census

We Fingerprinted the Top 1,000,000 Websites — open data

Aggregates over a homepage technology-stack fingerprint of the Tranco top-1,000,000 domain list, snapshot 2026-08-25 (run_id=top1m-2026-08-25-full-v2, detector version 2026.10.47). One record per domain — CDN, CMS, ecommerce platform, web server, SSL/TLS issuer, DNS provider, analytics, and more — detected from homepage HTML and response headers, aggregated into the tables below, plus a stratified ~10,000-domain row-level sample.

Companion to the study [We Fingerprinted the Top 1,000,000 Websites. The Top 1,000 Run a Completely Different Stack.](https://crawlora.net/blog/top-1-million-websites-techstack-2026?utm_source=huggingface&utm_medium=referral&utm_campaign=techstack-census-2026). The same underlying census is browsable and filterable as a queryable Tech Stack dataset.

Headline findings

• —The stack flips as you go down the rank list. Enterprise bot management runs 26.3% of the top 1,000 sites by Tranco rank, falling to 9.8% of the 100K–1M tail — a 2.7× gradient. React meta-frameworks (Next.js, Nuxt.js, Astro, Gatsby, SvelteKit, Remix) fall the same way, 16.4% → 6.0%. WordPress and free-tier Cloudflare CDN adoption run the opposite direction — 10.6% → 19.8% and 20.8% → 42.2% respectively.
• —84.7% of the 1,000,000 domains resolve a working homepage (847,491 reachable). A reachable=false record means the fetch itself failed — it is never counted as "zero technology detected." See METHODOLOGY.md.
• —Chinese-TLD (`.cn`) sites are measurably harder for a Western-signature-trained detector to see. Reachable .cn sites average 3.56 detected technologies vs. 7.84 for the rest of the reachable universe — a 2.2× gap — and show zero detected technology at 5.09% vs. 0.54% elsewhere (9.4×). This is a detector-coverage limitation, not a claim about the underlying sites — see the "What this dataset is NOT" section of METHODOLOGY.md before citing it.
• —Let's Encrypt and Google Trust Services combined secure 72.4% of every site carrying an SSL/TLS issuer signature — the free-CA era, measured directly.
• —This exact top-1K/10K/100K/1M rank segmentation is BuiltWith's paid "Market Analysis" product, $295–995/month. This release computes the same kind of cut from the open Tranco census, free.

Files

Methodology (condensed — see METHODOLOGY.md for the full version)

Homepage HTML and response headers for the Tranco top-1,000,000 domains were fingerprinted against a technology-signature database (CDN, CMS, ecommerce, analytics, web server, and more) — the same category of detection BuiltWith and Wappalyzer use. HTTP-only (no browser rendering); Tranco-seeded only. Collected via the Crawlora web-data API.

Caveats

• —`reachable=false` does not mean "no technology detected." It means the homepage fetch itself failed. Every adoption-rate figure in this release excludes unreachable domains from its denominator.
• —The rank-band table is a pinned snapshot, not a live query — the public /datasets/techstack/* API sorts by rank but has no rank-range filter, so this specific cut is reproducible only from a full-census pass, not the REST API.
• —`.cn` figures describe detector coverage, not the underlying Chinese web — see METHODOLOGY.md's "What this dataset is NOT" section before citing the visibility-gap numbers.
• —This is a frozen dated snapshot, not a live feed. It will not reflect stack changes made after 2026-08-25. For current data, query the live API.
• —The row-level sample is ~1% of the full census (9,850 of 1,000,000 domains), stratified by rank band, not the full dataset — see data/sample-schema.md for exactly which fields are and aren't included and why.

License & citation

Released under [CC BY 4.0](LICENSE) — use it for anything, just attribute.

Crawlora (2026). We Fingerprinted the Top 1,000,000 Websites — aggregate technology census by Tranco rank. https://github.com/Crawlora-org/techstack-census-data (CC BY 4.0)

See `CITATION.cff` for machine-readable citation metadata, and `CHANGELOG.md` for the refresh history and cadence.