datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
ccrawl-recrawl-domains
Common Crawl Domain Recrawl
Live fetches of the home page of every ranked domain in Common Crawl's web graph, rendered to Markdown as they are fetched
What is it?
Common Crawl's web graph ranks domains by how central they are, but it does not tell you what those domains actually serve today. This dataset walks that ranking from the top and fetches each domain's home page now, storing the response as one Parquet row with the body, the headers, the timing and the… See the full description on the dataset page: https://huggingface.co/datasets/open-index/ccrawl-recrawl-domains.ccrawl-recrawl-urls
Common Crawl URL Recrawl
Live refetches of pages from Common Crawl's URL index, with the body inline and the text already extracted
What is it?
Common Crawl publishes which URLs it saw and when, but the page bodies live in WARC archives that are awkward to query and are as old as the crawl that made them. This dataset takes the URL index for a single monthly crawl and fetches the pages again now, storing each response as one Parquet row with the body, the… See the full description on the dataset page: https://huggingface.co/datasets/open-index/ccrawl-recrawl-urls.ccru-knowledge-instruct
CCRU Knowledge-Instruct Dataset
Synthetic instruction-tuning dataset generated from a curated corpus of texts related to the CCRU (Cybernetic Culture Research Unit), accelerationism, and adjacent continental philosophy.
Dataset Summary
Attribute
Value
Examples
278,463
Format
Chat instruction (system / user / assistant)
Domain
CCRU theory, accelerationism, hyperstition, continental philosophy
Generation model
huihui-ai/Qwen3.5-9B-abliterated-MLX-4bit… See the full description on the dataset page: https://huggingface.co/datasets/wayjeeair/ccru-knowledge-instruct.ccr_train_1000