commoncrawl/web-graph-testing-v1

1---2tags:3- webgraph4- cc-main-2025-oct-nov-dec5viewer: false6---7 8# Common Crawl neo4j Web Graph (Crawl from October, November and December 2025)9 10This dataset contains one of the Common Crawl Foundation's Web Graphs, stored in a neo4j graph database dump. 11We have been computing these web graphs since 2018, and currently every crawl has a web graph covering the previous 3 crawls.12 13This dataset is a test and contains the web graph computed over the crawls of October, November and December 2025. 14More information [here](https://www.commoncrawl.org/blog/host--and-domain-level-web-graphs-october-november-and-december-2024).15 16## Status17 18This project is in beta-testing. Please give it a try with the one Web Graph we've converted: we provide both the domain and host version.19 20The host Web Graph contains each hostname as a separate node, and links between them as edges.21The domain Web Graph is built by aggregating the host graph at the pay-level domain (PLD) level based on the public suffix22list maintained on publicsuffix.org.23 24## Hardware Requirements25 26We recommend 2–4 CPU cores or more, 16–32 GB of memory, and ample storage -- 512GB to 1TB.27 28## Getting Started 29 30Detailed instructions on how to spin up a neo4j instance and load one of the web graphs' dump are [here](https://github.com/commoncrawl/cc-web-graph-neo4j#docker-container). 31 32## Contact33 34All information are in our [GitHub repository](https://github.com/commoncrawl/cc-web-graph-neo4j).35Feel free to reach out to us if you have questions or feedback. 36 37